Spending is growing, but quality still blocks deployment
Enterprise AI spending is growing fast. Menlo Ventures' 2026 State of Enterprise AI report, published on November 3, estimates enterprise AI deployment spending at an annualized run rate of $177 billion as of September 2026, up from $33 billion a year earlier. Its survey of nearly 400 US enterprise decision-makers found that reported spending averaged 4.7 times the previous year's level against a planned 1.9 times, roughly 2.5 times plan.
Scaling is slower than spending. McKinsey's 2026 State of AI survey, which we read through FM Magazine's coverage because McKinsey's own page did not load for us, has 44% of respondents reporting that AI is scaling across their enterprise, up from 38%. Among large companies, the share scaling agents in one or more functions rose from 27% to 40%. Among smaller organizations it stayed at 22%.
Gartner's numbers point the same way. According to InformationWeek's report on the October 19 Symposium keynote, Gartner's survey of more than 2,000 IT executives found that 71% will deploy agentic AI within 12 months, 21% have agentic AI governance today, and 13% of CIOs have realized significant or transformational value from AI. Gartner's own release did not load for us, so these are secondhand too.
The most specific data on what blocks teams comes from LangChain's State of Agent Engineering survey: 1,340 responses collected between November 18 and December 2, 2025. In it, 57.3% of respondents have agents in production. The top barrier is quality, cited by 32%, ahead of latency at 20%, and cost comes up less often than in previous years. A vendor surveying people who already build agents will overstate adoption, but the ranking of blockers is useful.
The same survey also shows a gap between observability and evaluation adoption. 89% of organizations have some form of observability for their agents, but 52.4% run offline evaluations on test sets and 37.3% run online evals. Many teams can watch their agent work and far fewer can test it.
Cost and budget can still block a project. This article is about one failure mode we see often: a pilot has funding but no agreed quality bar. The demo showed the agent can do the job once. Nobody can say how often it does the job correctly, so nobody signs off. When that is the problem, an eval set gives the team evidence for a release decision.
What an eval set is
Anthropic's engineering post Demystifying evals for AI agents (January 9, 2026) gives the vocabulary we use. A task is a single test with defined inputs and success criteria. A trial is one attempt at a task. A grader is the logic that scores some aspect of the agent's performance. A suite is a collection of tasks that measures a specific capability or behavior.
In practice an eval set is a versioned file in your repository. Each row holds an input, a description of the correct outcome, and the grader that checks it. The post insists on grading the outcome, meaning the final state of the environment, and not what the agent says it did. Its example: check that the reservation exists in the database.
Build the first set from real cases
Anthropic's post says 20 to 50 simple tasks drawn from real failures is a good start. OpenAI's evaluation best practices guide adds two habits: log everything during development so you can mine the logs for cases, and cover typical, edge and adversarial inputs. It also lists, as an anti-pattern, datasets that do not reproduce production traffic.
A pilot has already produced the raw material. We collect the conversations where a user corrected the agent, the tickets raised during the trial, and the manual checks someone runs before every demo. For each one we write down the input and what a correct result looks like. Anthropic's test for a good task is that two domain experts would independently reach the same pass or fail verdict. When your experts disagree, the specification is unclear, and it is cheaper to find that out here than after launch.
Include cases where the right behavior is to refuse or escalate. A set that only rewards answering produces an agent that always answers.
We use synthetic cases to fill gaps, and we keep them a minority. A mostly synthetic set measures what the team imagined users would ask. The cost of real cases is expert time, and no tool removes it.
Three kinds of graders
Code checks are fast, cheap and reproducible. Their weakness is brittleness: they fail valid answers that are phrased differently. We use them for everything that can be checked exactly, such as the output schema, an amount, the tools called, or whether a cited source was actually retrieved. TaxFigure, the tax research platform we built the research engine for, ties every answer to the primary authority it relies on and logs each query and source. Properties like that are the easiest to grade with code.
Model graders handle open-ended output and scale well. They are also non-deterministic and need calibrating against people. OpenAI's guide says to scale up an LLM judge once it consistently agrees with human annotations. Our routine is to have a domain expert grade a sample, compare it with the judge, and fix the rubric until the two agree, then repeat when the judge model changes. We prefer pass or fail rubrics to 1-to-10 scores, because nobody can explain the difference between a 6 and a 7.
Human review is the reference standard, and it is slow and expensive. We spend it on calibration and on reading transcripts. The Anthropic post is blunt: without reading transcripts and grades from many trials, you cannot know if your graders work.
We part ways with one piece of vendor advice. Anthropic's documentation on building evaluations recommends volume over quality: more questions with slightly lower-signal automated grading beat fewer hand-graded ones. For regression coverage we agree. For the decision to launch, we would rather have 40 cases that the client's experts have signed than 400 that nobody has read.
Regression runs in CI and before model switches
Anthropic's post separates two suites. Capability evals ask what the agent can do and should start at a low pass rate. Regression evals ask whether it still does what it used to and should pass nearly 100% of the time.
Run the regression suite on every change to a prompt, a tool definition, a retrieval setting or a model version. OpenAI's guide calls this continuous evaluation. Because outputs vary between runs, one green run means little. The post's arithmetic: at a 75% success rate per trial, the probability of passing three trials in a row is about 42%. For anything customer-facing, report the pass rate over several trials.
A full suite with model graders on every commit costs money and minutes. We run the code-graded subset on each commit, and the full suite nightly and before a release.
Model switches are where this pays back. Vendors ship models on their own schedule, as we covered in our guide to the autumn model wave. Per the Anthropic post, teams without evals face weeks of testing when a new model arrives, and teams with evals can upgrade in days.
Tooling changes too. OpenAI's guide now carries a notice that its hosted Evals platform is being deprecated, read-only from October 31, 2026 and scheduled to shut down on November 30, 2026. Keep your cases and graders as plain files in your own repository, and treat the runner as replaceable.
After launch: online monitoring
Offline evals catch breakage in behavior you already know about. Production shows you what you never thought to test. The Anthropic post notes the limits of monitoring: it is reactive, and live traffic has no ground truth to grade against.
So we sample live traffic and run the graders that need no reference answer: format checks, citation checks, policy checks and a calibrated judge. We track what users do, including ratings, edits, retries and escalations. The assistant we built for AD SME Hub lets users rate every answer to feed improvements. Zenyt puts a grader inside the product: a second stage re-investigates every potential issue with screenshot proof before it is reported, which filters out false positives.
Every confirmed production failure becomes a new case in the offline set, so the set grows with the product.
What "ready for production" looks like
We write thresholds down before the run, with the person who owns the business process. Ours usually take this shape:
- Regression suite: passes in full, across repeated trials.
- Critical failures: for classes such as a wrong amount, a fabricated citation or an action on the wrong account, any occurrence in the suite blocks the release.
- Capability set: a target agreed against a human baseline measured on the same cases.
- Fallback: when confidence is low the agent hands off, and there are tests that prove it does.
- Latency and cost per task: inside the budget, measured on the eval run.
These numbers are judgment calls. On 50 equally weighted cases, 92% means 46 passes and 96% means 48. That difference alone does not establish a reliable improvement in production. Passing the critical cases shows that those failures were not observed in these trials, not that they cannot recur. That evidence, agreed in advance with the process owner, is usually what the sign-off was waiting for.
Checklist
- Collect 20 to 50 real cases from the pilot, including ones where the agent should refuse.
- Have two domain experts agree on the correct outcome for each.
- Write code checks first, then a model grader calibrated against a human sample.
- Run regression on every change and before any model switch, over several trials.
- Agree release thresholds in writing, with a zero-tolerance list.
- After launch, sample live traffic and turn each confirmed failure into a new case.
Our AI Deployment program runs five stages in about six weeks. Integration and testing covers accuracy, latency, edge cases and failure modes on your data, and we stay on for 30 days after launch to monitor and tune. Explore our AI Deployment program to see how we assess, integrate and test a pilot for production.
Sources
- 2026: The State of Enterprise AI, Menlo Ventures, November 3, 2026
- Companies' financial value from AI holds firm in 2026, FM Magazine, September 1, 2026 (on McKinsey's "The state of AI in 2026"; McKinsey's page did not load for us)
- Gartner: CIOs must balance AI value with risk, and fast, InformationWeek, October 19, 2026 (on Gartner's Symposium keynote; Gartner's release did not load for us)
- State of Agent Engineering, LangChain, page dated June 12, 2026 (survey fielded November 18 to December 2, 2025)
- Demystifying evals for AI agents, Anthropic Engineering, January 9, 2026
- Evaluation best practices, OpenAI API docs, consulted November 2026
- Define success criteria and build evaluations, Claude Platform docs, consulted November 2026







