Demystifying large language models

it’s hard to have an opinion
on a machine you’ve never seen inside

a page of token pairs — a word, and the word before it

the algorithm

I call out the last token we wrote (and its colour)
hands up if it’s the boxed token on your page
I pick a hand at random — you read out the token beside it
I write it down, and we go again

Walk it once, slowly. Reading it off the screen is fine.

Step 2: one hand each. More than one match? Go with the first one you spot.

Step 3 is the one to labour, briefly: I pick at random — not whoever’s fastest, because that samples the quickest reader in the room rather than the text. Random is what makes the room behave like the model. Don’t explain why yet; they’ll see it.

Then the seed: someone near the front, sheet in hand, reads out any one pair, boxed word first. It matters that it came off a sheet rather than out of your head — the room supplies the first two words as well as every word after them. If their pair is an odd one (a full stop, a name that appears once), take it anyway: a thin first round is a real property of the model, not a failure. It can’t dead-end on you, so you never need a second volunteer.

4:00

Shift-W, write the volunteer’s two words, call out the second of them, go. Four minutes, 12–15 tokens, write BIG for the back of the room. No commentary between hands going up and a word landing on the board.

Narrate two things, once each, without stopping the flow. The hands are the distribution: call a common word and a forest goes up; call a rare one and you get two. Nobody here has been taught weighted sampling and the room is doing it anyway — the number of hands is how often that continuation happens in the text. (Pedant’s footnote, only if a pedant asks: someone holding several matches for one context raises one hand, which flattens the busiest words slightly.) And the choosing is a coin toss: when five hands are up, the four I don’t pick were all real continuations. Different pick, different sentence. Same reason the “regenerate” button gives a different answer.

Hands will go up every round. Every distinct token appears as a boxed word somewhere, so there is no word you can call that has nothing following it — don’t plan a “running dry” beat. The one way the room goes quiet is an empty seat: that sheet went home with someone, and its pairs are simply gone. Say exactly that, and reseed. The loud seeds for the corpus are on the brief (page 1 of the printed set); avoid restarting from a full stop, or the first thing the room does is argue about punctuation.

Stop while they still want more, around 15 tokens. Read the sentence back.

we just ran a language model

what role did you play?

each word on your sheet → a token

the pairs on your page → the parameters of the model

the hands in the air → the probability distribution over next tokens

my random pick → sampling

the sentence I wrote → generation

the sentence we wrote is in no book

three books

“He was an old man who fished alone in a skiff in the Gulf Stream.”
— Ernest Hemingway, The Old Man and the Sea (1952)

“TRUE! nervous, very, very dreadfully nervous I had been and am.”
— Edgar Allan Poe, The Tell-Tale Heart (1843)

“The sun did not shine. It was too wet to play.”
— Dr. Seuss, The Cat in the Hat (1957)

a hall full of people just did
the thing you’re told needs a data centre

it doesn’t make you an expert

it makes you harder to bullsh*t

llmsunplugged.org/talks/demystifying-large-language-models/

Demystifying large language models#

the algorithm#

what role did you play?#

three books#

Demystifying large language models

the algorithm

what role did you play?

three books