TokenTown: A visual way to understand how LLMs work
TokenTown is an interactive, browser-based visualization that represents the internal mechanics of a transformer language model as an isometric city. It simplifies complex processes like tokenization, attention heads, and feed-forward layers to help users understand how LLMs generate text.
Why it matters
It provides a rare, intuitive educational tool for demystifying the 'black box' of artificial intelligence architecture for non-experts.
A language model laid out as a city, one token at a time
The city is idle. Type a prompt below and watch a single token make the round trip.
12 numbers standing in for the 4,096+ a real model carries. Blue is negative, warm is positive.
TokenTown is an isometric city where every district is one stage of a transformer language model. A convoy carries a hidden state along the roads: it is cut into tokens at the docks, cast into a vector at the foundry, stamped with its position, then driven around the layer ring (attention, residual, feed-forward, residual) once per layer, before the stadium turns it into a probability distribution and the sampler picks one token. That token drives back up the feedback highway and the whole city runs again.
Get smarter about the news
Sign up free for a feed built around what you actually care about, Dive Deeper research on any story, and the full text of every article.
Create free accountAlready have an account? Sign in