4 items
#explainer
Posts
Where a token actually spends its time
A new explainer that follows one token through a GPU cluster, and makes the case that prefill and decode are different problems sharing hardware.
Watching GPT-2 pick a word, one step at a time
An explainer that runs a real model in your browser and shows the eight steps between a prompt and a single token, including the one everybody gets wrong.
Projects
GPU token explainer
Sixteen stages in five acts, following one token from an HTTPS request hitting a load balancer, through a GPU cluster's scheduler, sharded weights and collectives, into the transformer math and back out as streamed text. Every cluster figure is real arithmetic over the configuration you set.
LLM token explainer
A step-by-step walkthrough of how a language model produces one token: tokenization, embedding, attention, logits, sampling. Runs GPT-2 in the browser, so the numbers on screen are the real ones.