Modules
Modules are the building blocks of LLMs Unplugged: each one is a self-contained hands-on activity teaching one idea about how language models work. For a ready-to-run session built from them (slides, timings, materials), start from the lessons. New here? Do Training, then Generation, then pick whichever extensions catch your eye.
There are three ways to run a paper model:
- Grid: pen, paper and a d10. Tally word pairs into a table, then roll to sample (a 60/40 split is “1–6 the first word, 7–10 the second”). It plugs straight into the probability curriculum; if that style of sampling is new to your group, run Weighted Randomness first.
- Cutouts: printed tokens spread on a table and a colour-matching game. The maths happens automatically, so it works for any age; the cost is prep, since someone has to print and cut (the Materials page has ready-to-print packs).
- Search sheets: the whole text dealt across the room, one sheet each, and the room generates by show of hands. No training and no cutting: it's how we run the activity as a talk, or with a big class and thirty minutes. See Search Sheets.
The Fundamentals (and a couple of extensions) have a grid/cutouts toggle at the top of the page; both flavours teach the same ideas.
Fundamentals
Training
Build a bigram language model that tracks which words follow which other words in text.
years 5 and up · 20 min · grid, cutouts
Generation
Use your hand-built bigram model to generate new text through weighted random sampling.
years 5 and up · 20 min · grid, cutouts
Search Sheets
Deal a whole text's token pairs across the room, one sheet each, and generate as a crowd---call a word, hands go up, pick one at random. The generation activity for a hall, a lecture, or a big class with thirty minutes.
years 5 and up, any room size · 10-15 min · sheets
Extensions
These can be done in any order after the Fundamentals. Each one explores a different aspect of how modern language models work.
Scaling up
Run a model you didn't train, and see what a second word of context buys you and what it costs.
Pre-trained Model Generation
Use a provided pre-trained booklet to generate text without training your own model.
years 5 and up · 20 min · booklet
More Context
Generate from a bigram booklet and a trigram booklet of the same book and compare---two words of context makes the text far more coherent and the booklet far thicker. Then build a trigram by hand, and add cheap extra context with a skip grid.
years 7 and up · 20-30 min · booklet, grid
Extending the model
Give the model tools, so it can pause, ask for something outside itself, and carry on.
Shaping behaviour
Change what a model says without changing the mechanism: the sampler, the training data, the judges, and its own output.
Sampling
Same model, different output---experiment with temperature, truncation strategies, and your own house rules, then put it all together in a poetry slam.
years 7 and up · 20-30 min (60 min with the slam) · grid, booklet
Synthetic Data
Run a relay around the room. Each group trains only on the text the group before it generated, and nobody after the first ever sees the original. Watch the vocabulary shrink at every link in the chain.
years 9 and up · 30 min · grid, booklet
RLHF
Run it as a game show. Two judging teams with different tastes rank the same model's output for three rounds, and one base model comes out as two, shaped by whoever was holding the scorecard.
years 9 and up · 30 min · grid, booklet
Sycophancy
Skew your model toward over-agreeable, flattering responses by piling in sycophantic training data.
years 9 and up · 20 min · grid, cutouts
Looking inside
Two things a real model does that a bigram can't---learn from the page in front of it, and know that cat is like dog---run by hand.
In-context Learning
Make your model finish a pattern it never trained on, purely from what's already on the page---then give it a short-term memory that keeps the text on topic. The tricks behind few-shot prompting and attention.
years 9 and up · 20 min · grid, cutouts, booklet
Word Embeddings
Guess which two words in your grid behave alike, prove it by swapping them in your text, and check one distance by hand, then let the widget do all the pairs at once.
years 9 and up · 20 min · grid