An SLM trained on $8 ESP32-S3
Developers have successfully trained a transformer-based AI model from scratch on an $8 ESP32-S3 microcontroller. This experiment challenges the assumption that AI training requires massive data centers or high-end GPUs, demonstrating the potential for on-device learning in edge computing applications.
Why it matters
It proves that machine learning capabilities can be democratized and deployed on extremely low-cost hardware, opening new possibilities for autonomous, offline sensor intelligence.
This text was written with the assistance of an AI. Careful: assistance doesn't mean the AI wrote it. It means it corrected, reviewed and filled in some parts, but the author is human (or so I believe).
This is how you train a transformer from scratch on an eight-buck ESP32-S3. The (Klingon) GPT nobody asked for but everybody needed.
And why did we need it? Because we take for granted that training a model requires a GPU or a datacenter. Not always: sometimes something as small and as cheap as an eight-buck micro is enough to train one from scratch.
Read that again: train. From scratch. Not run a pre-cooked model. Train. Forward pass, backprop and weight updates, inside the chip.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in