LLMs & Inference
Tokens, attention, and why serving a model is a memory problem.
- 1Text becomes tokensA language model never sees letters or words. It sees tokens, chunks learned by gluing the most common pairs together. Train a tokenizer and watch your text get chopped.Runs in your browser12 min+30 XP
- 2One token at a timeA language model does one thing - give every possible next token a probability - and then picks one. Turn the temperature knob and watch a tiny model go from boring to unhinged.Runs in your browser13 min+30 XP
- 3Attention, without the maths firstAttention lets every word decide which earlier words matter to it. Click through a sentence and watch "it" work out who was hungry.Runs in your browser15 min+35 XP
- 4Why vLLM is fastServing a language model to many people at once is mostly a memory problem. Watch a KV cache fill, fragment and page, then race four serving strategies.Runs in your browserOn your machine18 min+45 XP
- 5Run your own modelServe a real language model on your own laptop with the tilt platform, add a vector database for retrieval, and watch it all with an LLM tracing tool. Honest about how slow a CPU is.On your machine30 min+50 XP
Next track: AI Frontiers →