Watch a sentence travel the full pipeline — tokenize, embed, attend, pass through the learned layers, and predict the next word. Press Play, or use Step to move one beat at a time.
Split the text into tokens; each maps to an integer ID in the vocabulary.
Each token ID becomes a learned vector; position information is added so order is preserved.
Position values come from sine & cosine at different rates — so position 0 is 0.00, 1.00, 0.00, 1.00… and later positions shift (fast dimensions change quickly; slow ones stay near 0 and 1).
Every token weighs how much each other token matters. Different heads capture different relationships.
The block (attention + feed-forward) repeats, refining each token's meaning in context.
The final representation becomes a probability over the vocabulary; the model emits a token and repeats.
Probabilities here are illustrative — they show the mechanism, not a specific trained model.