Acknowledgement of Country#

What is this about?#

you’ll build your own language model—from scratch—with just a kids book, pen & paper, and some dice rolling

you’ll learn how language models work by spotting patterns in text to generate new text

The recipe#

walk through your text and tally up which tokens follow which in a grid

Example#

“Run, Spot, run. See Spot run.”

after tidying up:

run , spot , run . see spot run .

The language of language models#

• model
• token
• vocabulary
• training

The recipe#

use your grid to generate new text, rolling dice to choose each next word

Generated text#

“see spot, run. see”

a new sentence — not in the training data!

The language of language models#

• prompt
• response/completion
• context window

The recipe#

use the booklet to generate new text, rolling one or two d10s to choose each next word

The language of language models#

• pre-training
• foundation model

What makes a model an “agent”?#

an agent is a model that can call tools: pause generation, get information from “outside” the model, and continue

the loop: generate → trigger token → call tool → write the result → keep going

The recipe#

generate from your model as before, but every punctuation token triggers a tool call: text the sentence so far to 3 friends, and the whole first reply goes into your text—followed by the punctuation you rolled

Worked example#

your text so far is “the cat sat”, and the next dice roll gives you . as the next token—pause, that’s a tool call

text “the cat sat” (the whole sentence so far) to 3 friends; the first reply back might be “down by the river”

write down by the river, then the . you rolled anyway, and continue generating from .

You will need#

your model, dice, pen and paper (as before)

a phone

3 friends who text back quickly

The language of language models#

• agent
• tool call
• agentic loop
• agentic AI

More context#

your grid sees one word; Claude sees hundreds of thousands

Attention: learning where to look#

the next word is swordfish—attention finds where password last appeared and copies what came after

attention lets every earlier word vote on what comes next, and the model learns the weights—no grid could ever be that big

Tallies become knobs#

your grid stores each pattern as tally marks you can count; a transformer spreads the same patterns across billions of tunable numbers—knobs nudged during training, so similar words like dog and cat come to share them

From continuing to answering#

everything you built only continues text—post-training on example conversations is what turns a continuer into an assistant

Scale is a hell of a drug#

the same loop you ran by hand—now with billions of times the text and billions of times the numbers

Questions#

what new questions do you have about large language models?

What next?#

how will this change the way you think about and use LLMs in the future?