Tensor Is the Might
A technical deep-dive into the implementation of tensor libraries from scratch in C. It explains the mathematical abstractions and memory management required to build neural network foundations.
Why it matters
Provides foundational knowledge for engineers building high-performance machine learning infrastructure.
Every good abstraction solves a problem, and this post will cover everything I know so far about a brilliant math abstraction - tensors.
Neural networks, from a simple 2-layer MLP to GPT-5, all boil down to the same thing: floating-point numbers flowing through a graph of operations. This post builds a complete, accelerated tensor library from scratch in C. It is heavily inspired by Bellard's libnc , which unfortunately has not been open-sourced yet.
A tensor is nothing but a flat array of numbers, plus some metadata telling you how to interpret those numbers as a multi-dimensional object. We all learned that 2D arrays can be better represented as 1D array plus a number of rows/columns - this is essentially what a tensor is.
But going beyond two dimensions - we might need some other metadata, such as a generalised shape:
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in