Inference now drives two-thirds of AI workloads, changing where chips get deployed
What happened
Inference-first chipmakers want to capture that shift.
UPDATED 13:28 EST / NOVEMBER 21 2026 AI Inference now drives two-thirds of AI workloads, changing where chips get deployed by Kelly Knight AI spending is shifting from giant training clusters toward smaller, quickly built sites, and modular data centers are becoming a natural home for inference workloads. Positron AI Inc., which raised $875 million at a $5 billion valuation in September, builds inference appliances around commodity memory instead of scarce high-bandwidth memory (memory stacked next to a processor to feed it data faster).
Power, cooling and memory supply now shape where tokens get generated as much as raw chip performance does.
Inference has overtaken training as the force behind infrastructure buying decisions, according to Cameron McCaskill (pictured), head of sales and business development at Positron. “So now you can see why inference is driving the economic decisions of how they’re going to build out their infrastructure.”
McCaskill spoke with theCUBE Research’s Rebecca Knight and John Furrier at SC26 , during an exclusive broadcast on theCUBE, SiliconANGLE Media’s livestreaming studio.