The CPU is back: Rethinking the CPU-GPU split for LLM inference

Back to all posts For the past 3 years, graphics processing units (GPUs) have dominated the large language model (LLM) conversation. In traditional chatbot applications, central processing units (CPUs) provide a fraction of the total compute per request, while GPUs do the heavy lifting. However, inference isn't a single model answering a single question. A growing reliance on tool calls, multistep reasoning, and orchestration across small, specialized models changes the math on where compute should live. Intel has called out this shift noting that the CPU-to-GPU ratio is moving from 1:8 in training workloads to 1:1, and in some cases 4:1 in agentic deployments.

Here we'll examine why the assumptions making GPUs the obvious choice for LLM inference are being renegotiated, what's driving renewed demand for CPU-based serving, and what the data says about where the industry is heading.