Small Models Have Arrived
The emergence of highly capable, low-cost small language models is changing the economics of consumer AI applications. By significantly reducing inference costs, these models enable developers to build sustainable, personalized AI products that were previously too expensive to operate.
Why it matters
Lowering the barrier to entry for AI inference is essential for the proliferation of consumer-facing AI software.
For the past few weeks, I've been playing with gpt-5.6-luna . It is shockingly capable, fast, and smart. I regularly see it do ~100 tps, and rip around my codebase, email, and knowledge base.
Of course, the biggest thing with luna is the cost . I've tried running some fairly complicated research threads, and it's pretty tough to run up a large bill. Even having it search across thousands of emails, I end up with an API cost in the tens of cents.
With GLM 5.3, we even have a new option at the Pareto frontier.
When doing coding work, I almost always reach for the most expensive and capable models (Fable 5, 5.6 Sol). So it's been easy to miss the progress the small fast models have made.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in