LLMs Unplugged#
How AI writes stories#
Acknowledgement of Country#
Think → pair → share#
when you hear the words artificial intelligence, what comes to mind?
think on your own, then share with the person next to you
2 min. Then a quick crowd unpack: does anyone know what AI stands for? What have you talked about in class? Has anyone tried using it — what for?
Let’s ask one to write us a story#
we need three words or ideas — hands up!
Collect three words from the class---hands up, choose a few---and write them on the board so everyone can see.
Then alt-tab to ChatGPT and type a prompt along these lines (swap in this audience's details beforehand): "write an exciting and funny 2-paragraph story featuring [teacher's name] as the hero and incorporating the following three words or ideas: [the three words]. The story should be concise and appealing to [year level] ([age] year old) students from [city], and appropriate to read in a school setting."
Read the result back dramatically.
That was a language model#
What just happened?#
no person on the other side wrote that story for us
the language model learned patterns from huge amounts of text, then copied those patterns to make something new
today we’ll build our own language models — without a computer
Reinforce: whether or not you use AI, understanding how it works helps you make better choices about when to use it.
Predict the next word#
we humans are good at spotting patterns
I’ll start a sentence — hands up if you know the next word
Do a few of these live on the whiteboard, pausing for hands before the last word:
- "be respectful, be responsible, be ______" (swap in this school's actual values if known)
- "yer a wizard, ______!"
- "I don't eat chocolate, so at Easter I go hunting for ______"
So how do we teach a model?#
humans pick up patterns from listening, reading, watching
a language model has to learn them from text
Bridge straight into the cutouts mechanic on the next slides.
Start with a text#
And I will eat them here and there. Say! I will eat them anywhere!
I do so like green eggs and ham!
— Dr Seuss, Green Eggs and Ham
Tidy up the text#
andiwilleatthemhereandthere.say!iwilleatthemanywhere!idosolikegreeneggsandham!
lowercase everything, and note that we’ve separated . and ! from the word
they’re next to
A pair of words is a cutout#
andiwilleatthemhereandthere.say!iwilleatthemanywhere!idosolikegreen eggsandham!
green eggs
Every pair becomes a cutout#
and ii willwill eateat themthem herehere andand therethere .. saysay !! ii willwill eateat themthem anywhereanywhere !! ii dodo soso likelike greengreen eggseggs andand hamham !
This section used to slide an animated window across the text, one slide per
pair. The cutouts already show that overlap themselves (a cutout ends with the
word the next cutout starts with), so it's now three static slides: the tokens,
one pair highlighted as a cutout, then every pair as a cutout.
Now they’re just a pile#
there .green eggsham !will eatand hamhere anddo soi dolike greenthem anywhereeat themand there! iwill eateat themi willsay !them hereso like. say! ianywhere !eggs andi willand i
Now they’re just a pile#
eat themi doi will. sayham !eggs andand ithem hereand theredo sowill eat! isay !will eatlike greenthere .green eggsi willhere andand hameat themthem anywhereso like! ianywhere !
Now they’re just a pile#
and hamanywhere !i willthere .will eatham !so likethem herehere andwill eati dodo soeat them! i! ieat themsay !and theregreen eggslike greenand ieggs andi will. saythem anywhere
Now they’re just a pile#
eat themi willthem anywherethere .green eggssay !eat themand hameggs andi dowill eat. saythem heredo sohere and! iham !i willlike green! iand ianywhere !will eatso likeand there
Now they’re just a pile#
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Five "scatter" frames of the *same* cutouts with different seeds. All carry
{/* _animate: pile */}, so Reveal.js auto-animate glides each cutout (matched by
its data-id="cut-N") from one arrangement to the next --- each click reshuffles
the pile while the heading stays put. Talk across them: this is the hinge into
Training/Generation --- the model is an unordered bag you rummage through, so
order stops mattering once the text is cut up. Add or remove
{/* _animate: pile */} + slides
(any seed) to lengthen or shorten the shuffle.
Train your model#
with your group you’ll need printed cutouts, scissors, and a clear
table
cut out the cutouts for your text and spread them on the table—each cutout
pairs a next word with its previous word:
green eggs that’s it—the spread is your trained language model
Once the loose-on-table flow makes sense, groups can optionally sort their cutouts into piles by previous word---it makes hunting for matches faster in the next section.
Training#
Click the dial to start/pause; use -1 / +1 to adjust on the fly. Circulate and help anyone stuck on cutting or on what "previous / next word" means.
Your goal#
generate new text by chaining cutouts together—write a word, find a cutout
whose previous-word box matches, write its next word, then hunt for the next
match:
will eat eat them them here
the chain grows (like dominoes)
Choose a starting cutout#
your page
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Choose a starting cutout#
your page willeat
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Find the next cutout#
your page willeat
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Find the next cutout#
your page willeat
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Find the next cutout#
your page willeat them
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Find the next cutout#
your page willeatthem
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Find the next cutout#
your page willeatthem
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Find the next cutout#
your page willeatthem
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Find the next cutout#
your page willeatthem here
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Find the next cutout#
your page willeatthemhere
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Find the next cutout#
your page willeatthemhere
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Find the next cutout#
your page willeatthemhere and
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Find the next cutout#
your page willeatthemhereand
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Find the next cutout#
your page willeatthemhereand
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Find the next cutout#
your page willeatthemhereand
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
Find the next cutout#
your page willeatthemhereand ham
and iwill eatwill eatdo solike greenand therethem anywhere. sayeat themham !eat them! isay !eggs andanywhere !i willso likei dogreen eggs! ithem herei willhere andthere .and ham
The whole generation loop, animated as one auto-animate run (id "hunt") on the
seeded overview pile, with the running output ("your page") growing above it. It
opens on the full pile, then dims everything but the chosen starting cutout
(will eat) --- you *choose* where to begin and it gives you your first two
words --- before the hunt proper begins. From there we chase the chain four
picks deep: each pick's next word becomes the word we match next, so the page
builds will eat → them → here → and → ham. Each step runs the same
narrow-and-select: whole pile lit, then a fast colour scan, then one pick ---
whose next word lands on the page in gold and seeds the next step. Where the
colour scan over-selects (a palette clash), an extra exact-word frame sits
between the scan and the pick to refine it; that happens for "them" ("do" rides
along) and "and" ("there" rides along), but not for "eat" or "here", where the
colour already isolates the word. That contrast is the point: colour is a quick
filter, you confirm by the word only when you must. The finished chain
"will eat them here and ham" is a recombination, not a quote: the book runs
"...here and there", but the model rolls "and ham", mashing "eat them here and"
onto "and ham". Worth saying aloud --- it's the "hint, not a copy" payoff on the
impact slide. Adjust the pages/targets/picks or the seed to vary the demo.
“will eat them here and ham”
a hint of the original, not a copy
You will need#
with your group
your trained cutouts spread (already on the table)
pen and paper to write down the generated text
Activity#
generate as much text as you can from your spread—keep chaining words until
you run out of time!
How did it go?#
did your stories always make sense? why or why not?
can any of your models write a story about a crocodile?
Both questions probe the same limit: a paper model can only use the words (and pairs) it saw in training---there's no "crocodile" anywhere in these books, so it simply can't appear. Sets up the wrap-up reflection.
One story, all together#
Optional section --- drop if running short on time.
Everyone is a word hunter#
we’ll pick one starting word for the whole class
everyone hunts for it in their own cutouts
if you find a match, raise your hand — we’ll pick one and read out the next word, and that becomes the new word to hunt for
Pre-write the starting word on the butchers paper. Choose something common across the books (e.g. "we're", "the", "a"). Keep going for a few sentences.
About your stories#
what would help our paper language models write better stories?
when you get a language model to write a story for you, is it thinking about how to write the best possible story?
if you started with the same starting word, would you generate the same story again?
First question opens the door to "more data, bigger model, longer memory" without naming those terms. Second: no---the model predicts patterns, it isn't thinking the way we do. Third: re-run a generation (or the Part 1 ChatGPT story) to show the randomness gives a different result each time.
