Glossary
This glossary connects the hands-on activities in LLMs Unplugged with the technical terms used for modern language models. Each entry gives a plain language explanation and links to the modules where the idea is used. The groups follow the module topics, with the apparatus at the end.
Fundamentals
Token
A single unit of text that the model works with. In our activities each word and punctuation mark is a token. Modern LLMs use subword tokens, so a long or unusual word may be split into several.
Vocabulary
All the distinct tokens a model knows. The words down the side and across the top of your grid, or the set of previous-word labels on a cutouts spread, form your vocabulary. A model can only ever produce a token from its vocabulary.
Language model
A system that predicts what text comes next, using patterns learned from training data. Your hand-built grid, a cutouts spread, a booklet, and a room holding search sheets are all language models.
LLM (Large Language Model)
A language model trained on a very large amount of text, with billions of parameters. The hand-built models in these lessons are tiny language models; ChatGPT, Claude and Gemini are large ones. The core principles are identical. The difference is scale.
Chatbot (ChatGPT, Claude, Gemini)
A product that wraps an LLM in a conversation. ChatGPT is OpenAI's, Claude is Anthropic's, Gemini is Google's, and there are many others. On this site we often name ChatGPT or Claude as shorthand for any of them: the concepts apply equally to every modern LLM chatbot, whichever one you use.
Training
Building a model by counting patterns in text. When you read through a passage and tally which word follows which, you are training a model. The cutouts, booklets and search sheets are models that the generator has already trained for you from a text.
Training data
The text a model is trained on. In our activities this is the passage you tally, or the book a set of cutouts, a booklet or a set of search sheets was generated from. Modern LLMs are trained on trillions of words from books, websites and other sources.
Generation
Using a trained model to produce new text by repeatedly predicting and choosing the next token. Look up the current word, pick a next word in proportion to the counts, write it down, repeat.
Inference
Using a trained model to produce outputs. In language models, inference means generating text. These lessons say "generation" because it describes what language models do more clearly, but "inference" is the word you'll meet in AI/ML literature and tooling.
Prompt
The input text you give a language model. In a bigram model the prompt is just the current word. In a modern LLM a prompt can run to hundreds of thousands of tokens, which gives the model far more context to work with.
Probability distribution
A set of options with a likelihood attached to each. The counts in one row of your grid, the matching cutouts on a spread, or the show of hands when the room is asked "who has `the`?" are all probability distributions over the next word.
Weighted random sampling
Choosing the next token with probability proportional to its count. A d10 roll against the counts does this, and so does letting your eye land on one of the matching cutouts, because common words appear on more of them.
Bigram model
A model that predicts the next word from one previous word. It's what you build in the fundamental modules: each row of your grid says what can follow a single word.
N-gram model
The general name for a model that predicts the next word from the previous n-1 words. Bigrams are 2-grams, trigrams are 3-grams, and so on. Every hand-built model on this site is an n-gram model.
Markov chain
A statistical model in which the next state depends only on the current state. Andrey Markov introduced the idea in 1913 while analysing letter sequences in Pushkin's Eugene Onegin. A bigram language model is a Markov chain over words.
Scaling up
Pre-training
The first, expensive training phase, where a model learns the general patterns of language from a huge corpus. Most people who use an LLM never train one; they use a pre-trained model, the way you generate from a booklet somebody else made.
Base model
A model as it comes out of pre-training, before any post-training shapes it into an assistant, or the unchanged model that an adapter such as a LoRA is laid over. In both senses the base stays as it is and something else supplies the shift in behaviour.
Trigram model
A model that predicts the next word from the two previous words. It predicts better than a bigram but has far more contexts to store, which is why a trigram booklet is so much thicker than the bigram booklet for the same text.
Context window
How many previous tokens the model can consider when it predicts the next one. A bigram has a context window of one word and a trigram of two; modern LLMs can consider hundreds of thousands or even millions of tokens.
Parameters
The numbers a model stores to encode what it has learned. Each cell of your grid, holding the count for one word pair, is a parameter. A modern LLM has billions of them, and a frontier model on the order of a trillion.
Neural network
A computational system, loosely inspired by biological neurons, that learns patterns from data by adjusting numerical weights. Modern LLMs are very large neural networks. Your hand-built model is not one, but it stores the same kind of information: which words tend to follow which.
Transformer
The neural network architecture behind GPT, Claude, Gemini and every other modern LLM. Its defining feature is attention, which lets the model draw on any earlier part of the context when predicting the next token, rather than a fixed number of previous words.
Extending the model
Agent
A language model that runs tools in a loop to achieve a goal. "Agentic AI" has become a buzzword, but in practice it means tool use: the model generates, a tool is called, the result is written back into the text, and generation continues.
Tool use
The mechanism by which a language model calls something outside itself (a calculator, a web search, a database, a code runner) during generation. A modern LLM emits a structured tool call; the software around it, the harness, pauses generation, runs the tool, and splices the result back in. In the unplugged activity, sampling a full stop plays the part of the tool call and you are the harness.
Shaping behaviour
Temperature
A dial on how much the model respects its own counts. Low temperature sticks to the favourites; high temperature flattens the odds until every option is roughly equal. A real model divides its scores by the temperature before turning them into probabilities. With raw counts that trick does nothing, so the hand-run version adds a constant to every count instead.
Greedy sampling
Always choosing the most likely next word, which is what temperature zero means. The output is predictable and often falls into a repeating loop.
Truncation strategy
A rule that limits which tokens are eligible before you sample. Top-k and top-p are the standard ones in LLM tooling; the Sampling module adds playful versions (haiku, alliteration, no-repeat) that work the same way, by changing what you're allowed to roll for.
Top-k sampling
A truncation strategy that keeps only the k most likely next words before sampling. Setting k to 1 is greedy sampling; a larger k allows variety while still excluding the very unlikely.
Top-p sampling
A truncation strategy that keeps just enough of the most likely words for their combined probability to reach a threshold p (0.9, say). Unlike top-k, the number of options kept changes with how confident the model is.
Repetition penalty
A generation setting that lowers the odds of words that have already appeared, to stop a model going round in circles. It's the opposite of the memory boost in the In-context Learning module, and the mechanised version of the Sampling module's no-repeat rule.
Fine-tuning
Further training of an existing model on a chosen text, to adapt it to a domain, a task or a style. Adding tallies to your finished grid from a second text is fine-tuning, and it's how the Sycophancy module skews a model's personality.
LoRA (Low-Rank Adaptation)
A way of fine-tuning cheaply by training a small adapter that sits on top of an unchanged base model instead of altering all its parameters. One base model can carry many adapters, and an adapter is small enough to share easily. The paper adapter in the Sycophancy module is the uncompressed version of the idea.
Post-training
Everything done to a model after pre-training to make it useful. Pre-training teaches the model how language works; post-training shapes its behaviour: following instructions, answering rather than continuing, refusing some requests, working carefully through problems. Supervised fine-tuning (imitating example conversations), RLHF and RLVR are all post-training techniques.
RLHF (Reinforcement Learning from Human Feedback)
A post-training technique in which people compare pairs of model outputs and the model is pushed towards whatever they preferred. In the original recipe the comparisons train a reward model, which then guides the main model; newer relatives such as DPO apply the preferences directly, which is closer to what the classroom version does. The term now covers a family of methods.
Reward model
A separate model trained on a smaller set of human comparisons to predict which output a person would prefer. It then rates outputs at scale during RLHF, because a training run needs millions of judgements and there aren't enough patient people.
Reward hacking
What happens when a model optimises the signal it's given rather than what the signal was meant to measure. Human raters tend to prefer answers that agree with them, so a model trained on their comparisons learns to agree and to flatter. It's one documented source of sycophancy in deployed models.
Alignment
Shaping a model's behaviour to match what its makers and users want of it, usually summarised as helpful, harmless and honest. RLHF and its relatives are the main techniques used to align modern chatbots.
RLVR (Reinforcement Learning from Verifiable Rewards)
A post-training technique in which the model makes an attempt, a program checks whether it's correct, and correct attempts are reinforced. The checker (a test suite, a known answer, a format validator) replaces the human judges of RLHF, so it works best where checking is cheap and reliable: mathematics, code, formal logic. It's the main technique behind reasoning models.
Reasoning model
An LLM post-trained (usually with RLVR) to generate visible working, a chain of thought, before its final answer. The thinking tokens are produced like any others, and spending more of them on a hard problem tends to improve the answer. OpenAI's o-series, DeepSeek's R1 and Claude with extended thinking are reasoning models.
Sycophancy
A model's tendency to agree with and flatter the user regardless of the merits. It has two known sources: reward hacking during RLHF, and pre-training data that is itself full of flattery. Tipping a grid's training data towards sycophantic phrases demonstrates the second.
Synthetic data
Training text generated by a model rather than written by people. Used carelessly, in a closed loop, it causes model collapse. Used deliberately (a stronger model writing for a weaker one, or filtering a large generated set) it's one of the more useful tools going.
Distillation
Training a smaller model on text generated by a larger, stronger one. The student ends up far better than it would on scraped text of the same volume, and because the teacher sits above the student the chain never closes on itself.
Model collapse
The degradation that sets in when a model is trained on its own output (or output from similar models) generation after generation. Rare words and constructions are the likeliest to be missed by any sample, so the tails vanish first, and the model narrows towards a repetitive core.
Looking inside
Embedding
A word represented as a list of numbers. A row of your bigram grid is that word's embedding, a fingerprint of the company it keeps. In a real LLM the numbers are learned rather than counted, and computed from the sentence the word appears in, so the two senses of "bank" get different vectors. The principle survives: words used alike get similar vectors.
Similarity matrix
A grid showing how alike each pair of words is, worked out by comparing their embedding vectors. Words used in similar contexts land close together.
Attention mechanism
The part of a transformer that decides, for each prediction, which earlier words in the context to draw on and how much. The skip grid in More Context and the memory boost in In-context Learning are fixed, hand-run versions of the same move; real attention is learned and changes with the content.
In-context learning
Picking up a pattern from the prompt and continuing it, without any change to the model's parameters. The "learning" lives in the context, not in the model. It's why few-shot prompting works: give an LLM a handful of examples and it carries on in the same shape.
Induction head
A circuit found inside transformers that completes patterns by finding an earlier place where the current token appeared and copying what came next. Induction heads are one of the main mechanisms behind in-context learning.
Materials
Grid
A sheet of grid paper with the vocabulary written down the side and across the top. Training fills each cell with tally marks for how often the row's word was followed by the column's word; generation reads a row and rolls against it. It's the original LLMs Unplugged apparatus.
Cutouts
Printed cards, one per word pair in the text, each showing a previous word in a box and the word that followed it. Spread on a table they are the model: to generate, find a card whose boxed word matches your current word and read off its next word. Common pairs appear on more cards, so picking by eye samples in proportion.
Booklet
A printed dice lookup table for a text: one entry per previous word (or pair of words, in a trigram booklet), listing the possible next words with the d10 range that selects each. A booklet is a pre-trained model you can generate from without having trained it.
Search sheets
A whole-room format: each person holds a sheet carrying a shuffled handful of word pairs from the text, and the room together is the model. Whoever is running it calls the current word, everyone holding it raises a hand, and one is picked. The show of hands is the probability distribution.
Ledger
A sheet with one row per word, tally marks for each word that followed it, and a coloured box around each tally column. Generation uses a cup of coloured counters, one per tally mark: draw a counter, read the word beside its colour. The cup does the weighted sampling.
Connections to your activities
The things you do with paper, dice and cutouts each stand in for an operation inside a real language model. The main difference is scale: your grid holds dozens of counts, a frontier model holds on the order of a trillion learned numbers, and the counts are learned rather than tallied by hand.
| Your activity | Real LLM equivalent |
|---|---|
| tallying word pairs from a text onto the grid | Training |
| the counts in one row of the grid, or the show of hands for a word | Probability distribution |
| rolling a d10 for the next word, or picking a matching cutout | Weighted random sampling |
| generating from a booklet you didn't make | Pre-training |
| one word back (bigram) versus two (trigram) | Context window |
| the counts in the cells of your grid | Parameters |
| sampling a full stop and texting a friend for the next words | Tool use |
| the cold, normal, hot and boiling settings of the dial | Temperature |
| crossing out options before you roll | Truncation strategy |
| adding tallies from a second text to a finished grid | Fine-tuning |
| the adapter sheet laid over an unchanged grid | LoRA (Low-Rank Adaptation) |
| judges voting on outputs and the counts being updated to match | RLHF (Reinforcement Learning from Human Feedback) |
| the reward-hacking round of the game show | Reward hacking |
| piling sycophantic phrases into the training data | Sycophancy |
| re-training on the previous group's generated text | Model collapse |
| reading a word's row of the grid as a list of numbers | Embedding |
| the skip grid, and the boost for recently used words | Attention mechanism |
| scanning back for the last time a word appeared and copying what followed | In-context learning |