I'm making LLMs work on small and old devices
A developer shares a project focused on running large language models (LLMs) on low-power, legacy hardware like old TV boxes and early-generation iPads. The project utilizes custom kernels and weight streaming to maximize the utility of idle computing resources.
Why it matters
This work pushes the boundaries of edge computing and AI accessibility, proving that sophisticated models can function on hardware long considered obsolete.
Every device around you is more capable than it looks. TV boxes, old phones, single-board computers and routers all ship with CPUs, RAM and flash that sit idle most of the day. smolmaxxing is about unlocking that capability: finding the biggest language model each device can really run, then pushing it further with custom kernels, smarter memory use and weight streaming. Every claim is measured on the hardware itself.
Device #1 is a Xiaomi TV Box S (3rd Gen) : Amlogic S7D, 4× Cortex-A55 @ 2.5 GHz, 1.9 GB of RAM (about 1 GB usable), eMMC flash, Android 14, no root. On it, this repo runs models from 260 K to 16 B parameters:
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in