Show HN: NanoEuler – GPT-2 scale model in pure C/CUDA from scratch
NanoEuler is an educational project that implements a GPT-2 class language model from scratch using pure C and CUDA. It avoids standard machine learning libraries to demonstrate the underlying engineering and mathematical principles of residual networks.
Why it matters
It provides a transparent, low-level look at how modern AI architectures function, serving as a valuable resource for developers and researchers.
A GPT-2-class language model built entirely from scratch in C/CUDA — no PyTorch, no autograd, no ML libraries. The forward and backward passes are written and verified by hand, and the whole training pipeline lives in this repo: a hand-written byte-level BPE tokenizer , pretraining on a books + web corpus, and supervised fine-tuning into a chat model (RLHF/DPO planned). It runs on CPU ( libm + OpenMP) for a small showcase model, and a full from-scratch CUDA engine — cuBLAS matmuls, a hand-written FlashAttention , validated against a CPU reference by a full-model gradient check — trains a ~116M-parameter model on a single RTX 4070.
The content is a technical project announcement and is purely descriptive of the software's capabilities.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in