Sep 18, 2026
Watching GPT-2 pick a word, one step at a time
Most explanations of how a language model produces a token are diagrams of something that never ran. They are usually correct and rarely convincing, because the author picked the numbers to make the picture come out right.
The LLM token explainer uses real ones. It loads GPT-2 into your browser and walks the eight steps from prompt to next token: tokenize, embed, position, attention, layers, logits, sample, append.
The leading space is part of the token
Type The quick brown fox and look at step one:
| # | Token | ID |
|---|---|---|
| 0 | The |
464 |
| 1 | ␣quick |
2068 |
| 2 | ␣brown |
7586 |
| 3 | ␣fox |
21831 |
␣quick is not quick. They are different tokens with different IDs, and that trips up more
people than anything else in the pipeline. It explains why a trailing space in your prompt
changes the output, why few-shot formatting is brittle in ways that feel superstitious, and
why your carefully stripped input behaves differently than it did in the playground.
You can read this in a tokenizer doc and forget it by lunchtime. Watching the IDs change when you add a space tends to stick.
Sampling is where the personality lives
Steps six and seven are the ones worth playing with. The model’s output is a distribution over roughly 50,000 tokens, and temperature, top-k and top-p are three different ways of throwing most of it away.
Turn temperature down and the distribution collapses onto one candidate. Turn it up and the tail fattens until the model will cheerfully pick something absurd. Top-k truncates to a fixed count. Top-p truncates to a cumulative mass, so it adapts, keeping one token where the model is confident and forty where it is not.
That difference is obvious once you have watched both applied to the same distribution, and stays muddy if you have not.
What is real here
Tokenization, logits and attention come from the GPT-2 model running locally. Embeddings are simulated, because the ONNX export does not expose hidden states. There is nothing real to show at that step, and I would rather say so than draw a convincing heatmap out of nothing.
The model is a full fp32 export with attention weights, which is why it weighs about 653 MB and why the first load takes a while. It is the only readily available export that exposes the attention tensors, and without those, step four would be another invented diagram.
No server involved
It all runs client-side on ONNX Runtime via WebAssembly. No API key, no backend, no inference bill. The weights come from HuggingFace and the compute is your laptop’s.
That also means this site serves it for nothing. The app is a few hundred kilobytes of JavaScript. The 23 MB WASM runtime Vite wanted to bundle turned out never to be requested, because the app points ONNX Runtime at a CDN instead, so checking saved more bytes than the app itself contains.
There is a companion piece, the GPU token explainer, which zooms out to the cluster that has to serve this same single token.