Half the companies deploying AI agents have shipped one that passed every test and then failed a real customer. That is not a theory. It is the central finding of a new VentureBeat Pulse survey of 157 mid-to-large enterprises.
Only 5% of organisations fully trust automated evaluation today. The top complaint is straightforward: evaluations do not match what happens in the real world. An agent scores well in a sandbox and then gives a wrong answer, breaks a workflow, or makes a decision that upsets a paying customer.
This is the evaluation gap. Companies are handing agents more freedom to act on their own while relying on tests they do not fully trust. Two-thirds already allow, or are actively building toward, deploying agent changes to production on automated evaluation alone — with no human in the loop.
The autonomy is arriving faster than the assurance.
Passing tests, failing customers
The numbers are stark. 50% of organisations have shipped an AI feature that cleared their internal tests and then caused a customer-facing failure. A quarter have seen it happen more than once. Only 36% report no such failure.
The remaining organisations either do not run pre-deployment tests (8%) or do not track root causes closely enough to know (6%).
The lesson is simple: a passing eval is not a working agent. Any business relying on AI tools needs to understand this distinction before trusting automated tests as the final word on quality.
No one trusts the tests
VentureBeat asked which limitation most reduces trust in automated agent evaluations. 29% said evaluations simply do not align with real-world outcomes. Another 21% cited bias or inconsistency — the same test gives different results for the same input. 18% said evaluations lack explainability, meaning you cannot tell why a test passed or failed.
Only 5% said they fully trust automated evaluation as it stands. That means 95% of organisations have a meaningful concern about the tools they are using to decide whether an agent is ready for customers.
For a small business using an AI chatbot or automated process, this matters because those evaluations are the only thing standing between a test environment and your customers.
Evaluating AI the right way
So what should a business do? The survey points to a few practical steps that apply whether you are a 100-person company or a multinational.
First, never treat a passing internal test as proof of readiness. Run real-world shadow tests where the AI works alongside existing processes before it takes full control. Second, monitor output quality in production, not just uptime and cost. Only 23% of enterprises run real-time quality checks on live AI traffic. The rest watch whether the system is up but cannot see whether its answers are right.
Third, keep humans in the loop for important decisions. The same companies planning to automate deployment are also the ones budgeting more for human reviewers next year. That contradiction suggests the smartest approach is hybrid: let AI handle the volume, but put people on the edge cases.
The practical takeaway
The evaluation gap is real. Companies are pushing AI agents into production faster than they can verify them. The result is a growing number of customer-facing failures that could have been caught with better monitoring and more honest testing.
For any business using AI, the fix is not to stop deploying. The fix is to test in production carefully, monitor what the AI actually outputs in real time, and keep a human ready to step in when the evaluation gets it wrong. Companies that invest in proper monitoring — watching what the agent actually does, not just whether it is running — will catch failures before they reach customers. That gap between autonomy and assurance is where the real risk lives. Close it before it closes on you.