Janas: Run LLMs larger than RAM by streaming experts from SSD
Janas-LLM is a new C-based tool that allows users to run large language models by streaming experts from an SSD, bypassing traditional RAM limitations. Benchmarks show it outperforming llama.cpp on various hardware configurations.
Why it matters
This technology significantly lowers the hardware barrier for running high-parameter AI models on consumer-grade laptops.
37 tok/s from Qwen3-Next-80B (a 48 GB model) on a laptop with 32 GB of memory, 24 without drafts, where llama.cpp manages 2.8; reading a prompt, 62 tok/s against 16. Janas-LLM was faster than llama.cpp on every model compared on the same laptop (Debian 13 Linux, Intel Core Ultra 9 185H, its integrated Arc GPU, 32 GB, an NVMe SSD; 9-11 October 2026; the whole machine below ):
The same models from other hands, in the catalog beside them:
One exception: the 3-bit Qwen3-30B reads a prompt slower than llama.cpp, since its Q3_K weights do not go to the GPU's fast prompt yet; it writes three times as fast.
How they were measured, and the speeds with the drafts: What works now . On another machine the figures will be other ones: janas-bench measures yours.
Try it on Linux x86-64 (Ubuntu 22.04, Debian 12 and later):
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in