There is a mismatch growing inside enterprise AI teams, and it is not about which model performs best. It is about money. And most finance teams have no idea how much they are spending.
According to new data from VentureBeat Pulse Research across 107 enterprises, AI infrastructure spending is accelerating far ahead of anyone actually tracking it. Companies are buying more compute than they can measure, and the numbers are worse than most executives realise.
The Compute Gap, In Numbers
Only 21% of enterprises run AI in production at scale right now. Three-quarters are still experimenting or have only partial production workloads. Yet 45% plan to evaluate AI-specialized clouds within the next year. That is a category almost none of them currently use. They are planning to spend big on infrastructure they have never touched.
Meanwhile, the hardware they already own sits cold. 83% of enterprises that operate GPUs report utilisation at 50% or less. Nearly half run at 25% or below. Only 12% clear the 50% mark. If you run any GPU workload and have not checked utilisation recently, you are probably wasting money.
This is not a small problem. A single GPU server can cost tens of thousands of dollars. Rack enough of them together and you are looking at seven-figure assets running at a quarter of their capacity. That is the kind of waste that gets CFOs fired.
Nobody Buys on Sticker Price
You might think the headline metric — cost per million tokens — drives buying decisions. It does not. Only 8% of enterprises cite it as the deciding factor. Instead, integration with the existing stack comes first at 41%, and total cost of ownership is second at 35%. Buyers want to know how a provider fits and what it truly costs to run, not what the advert says.
The problem? Fewer than half of enterprises — 44% — can rigorously track what their AI compute actually costs. Most say TCO matters most, yet most cannot measure it. You cannot optimise what you cannot see.
A Switching Wave Is Building
A clear majority of enterprises — 64% — plan to switch or add an infrastructure provider within twelve months. 38% plan to do so within the next quarter. For a category as foundational as compute, that is remarkable churn intent. It suggests widespread dissatisfaction with the current setup.
Where are they going? The specialized AI clouds — CoreWeave, Lambda, Crusoe, Nebius — top the evaluation list at 45% even though they barely register in current usage. Nearly a third intend to evaluate non-Nvidia accelerators. The type of cloud companies are most eager to try is the type they have barely begun to use.
This is either a massive opportunity for the neocloud providers or a warning that the current hyperscaler stack is not working for AI workloads. The answer is probably both.
What The Next Frontier Looks Like
The next constraint on AI infrastructure is barely on most companies radar. As inference scales, the bottleneck shifts from GPU compute to memory bandwidth. Roughly one in five enterprises is either unaware of this shift or has not addressed it yet. That blind spot will become expensive quickly.
There is also a looming question about power. AI data centers already strain regional grids. New York just halted new data center construction entirely. The days of frictionless compute expansion are ending.
What Business Owners Should Do
If you are running AI workloads or planning to, start measuring before you spend more. GPU utilisation is the simplest diagnostic. If your GPUs sit under 50%, you do not need new hardware. You need to use what you have.
Do not chase headline token prices. Pick vendors that integrate with your existing stack and give you real visibility into TCO. Run a pilot on specialized AI clouds before you commit. And if your leadership team cannot answer what AI compute costs your organisation, you are not ready to invest in the next tier of infrastructure.