Lessons

The lessons are split into Fundamentals that everyone should complete, and Extensions that you can pick and choose based on your interests and available time. If you’re new to LLMs Unplugged, start with the Fundamentals lessons below.

Grid or bucket?

The Fundamentals lessons (and a couple of the extensions) offer two “flavours”: Grid (paper grids and dice) or Bucket (physical tokens and containers). Use the toggle at the top of each lesson to switch between them.

The grid version connects well to probability concepts in the maths curriculum—students fill in a table with tally marks and use dice to sample from weighted distributions.

The bucket version is simpler and more tactile—students cut up tokens and draw them from physical containers. Great for younger learners or when you want to skip the dice maths.

Both teach the same core concepts. Pick whichever suits your group.

Fundamentals

These two lessons build on each other and should be completed in order.

Training

Build a bigram language model that tracks which words follow which other words in text.

Generation

Use your hand-built bigram model to generate new text through weighted random sampling.

Extensions

These lessons can be done in any order after completing the fundamentals. Each one explores a different aspect of how modern language models work.

Scaling up

Pre-trained Model Generation

Use a provided pre-trained booklet to generate text without training your own model.

Trigram

Extend the bigram model to use two words of context for better predictions.

Controlling output

Sampling

Experiment with temperature and truncation strategies to shape how your model picks the next word.

Beam Search

Explore multiple generation paths simultaneously and pick the best overall sequence.

Tool Use

Teach your model to call external tools when it needs information it doesn't have.

Context and meaning

These two lessons work well together—context columns extends the grid model, and word embeddings uses those extended grids to explore semantic similarity.

Context Columns

Add context columns to your bigram model to capture grammatical patterns, then use them during generation.

Word Embeddings

Turn each word's row into a vector and measure similarities between words in your model.

Model tuning

LoRA

Add a lightweight adaptation layer to retarget a trained model without retraining everything.

Synthetic Data

Generate synthetic text with your model, retrain on it, and see how patterns drift or collapse.

RLHF

Use human preferences to adjust your model's weights, making it generate text people prefer.