The real prices of frontier models. Tokens * Price, right?
This article explains how AI model pricing is often misleading because tokenizers vary in efficiency across different models. It argues that developers should measure actual token counts for their specific workloads rather than relying solely on advertised per-token pricing.
Why it matters
Understanding the hidden costs of tokenization is critical for businesses and developers to accurately forecast AI infrastructure budgets.
Every model's pricing page shows one number you are meant to compare: dollars per million tokens. Put two side by side and one looks cheaper. That comparison is broken - and if you build with an AI coding agent, it is broken in the most expensive direction. The price is per token, but a "token" is not a fixed amount of text: each model's tokenizer cuts the same file into a different number of pieces, and you pay per piece. Here is the number that lands on your invoice. The exact same TypeScript file is 681 tokens on GPT-5.x and 1,178 tokens on Claude's newest tokenizer - 1.73x more , and +31% over Claude's own previous tokenizer, before a cent of price difference. Your real workload as a builder is basically TypeScript, and TypeScript is exactly where the gap is widest. (English prose is a milder ~1.4x - more on why below.)
The article provides a technical analysis of market practices without political or ideological framing.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in