LLMs Unplugged (c) Ben Swift, Cybernetic Studio - CC BY-NC-SA 4.0
LLMs Unplugged

Appearance

LLMs UnpluggedUnderstand AI by building it yourself

Ready-to-use teaching resources for understanding how large language models (LLMs) work. No computers or coding required.

Workshop participants building language models with pen and paper

LLMs Unplugged is a series of hands-on activities demonstrating the fundamental ideas behind large language models. You start by manually counting word patterns in your training text (can be as simple as a kids book), recording these patterns either as tally marks on grid paper or even by cutting the paper into pieces and putting them into labelled buckets. You then generate new text (based on this training data) with some dice rolling or pulling the paper pieces back out of the buckets. After doing this by hand, something clicks: you’ll see that LLMs like ChatGPT works using the same basic principles, just at a vastly larger scale.

Diverse group of learners from students to professionals

Who is this for? ​

Although these activities are called “lessons” grouped into “topics”, they’re not just for school classrooms. These activities are suitable for audiences from primary school age through to adults—we’ve delivered them to hundreds of participants including school students, undergraduate students, and senior executives from both industry and government. Zero coding background required, and no maths beyond basic counting and percentages.

They work equally well for professionals who need to understand AI for decision-making in their organisations, parents wanting to learn alongside their kids, and educators teaching AI concepts in schools or universities. Even if you know a bit (or a lot!) about how LLMs work, you’ll still have fun making one and sharing it with others.

Person creating teaching materials with pen and paper

Who made this? ​

LLMs Unplugged is a Cybernetic Studio project created by Dr. Ben Swift (PhD ANU Computer Science, 2012). But if you’re worried that you’ll need a PhD in Computer Science to understand it then rest assured—the whole point of these resources is that demonstrate that you don’t. Which leads to the question…

Simple counting and dice activities

What prior knowledge do I need? ​

For instructors, the reading and numeracy skills required to teach this stuff are really only primary-school level. This isn’t just for specialist maths teachers or tertiary educators, although it works great in those contexts too. For participants, really all you need is a curiosity about how AI systems actually work and a willingness to engage with hands-on activities.

Each lesson includes instructor notes and discussion questions, plus interactive widgets (which you can use to get your head around the concepts) and—coming soon—explainer videos you can use in class. We also run teacher training sessions and offer professional delivery services if you’d like support bringing these activities to your organisation or classroom.

Understanding AI through hands-on experience

Why does this matter? ​

ChatGPT arrived in November 2022 and suddenly (most) everyone’s using LLMs. Yet most people have no real mental model of what’s actually happening under the hood. They’ve heard the hand-wave-y and mystical-sounding explanations, maybe picked up some vague notions about “neural networks” and “training data”, but the core mechanism remains opaque.

Understanding how these systems actually work—not through metaphor or handwaving but through direct experience—is what we’re all about. In a post-ChatGPT world, this kind of understanding isn’t optional anymore.

These resources don’t tell you or your students what to think about role of AI (good/mid/cooked) in society. They provide a grounding for you to have that discussion; one that’s informed not by folktales about what might be happening under the hood but by a lived experience of doing it for yourself.

Understanding the relationship between LLMs and AI

Why do you keep saying LLMs—this is AI, right? ​

Honestly, the short answer (these days) is yes. The long answer is more more complicated—AI is a term that has been applied to many different algorithmic techniques and systems over the years. But today (in the mid-2020s) Large Language Models (LLMs) have sucked all the air out of the room when it comes to AI, so that’s mostly what we’re talking about here. We do like to use the term LLM rather than AI (or even “GenAI”) to describe these systems because it’s a bit clearer what we’re talking about.

Getting started with language model activities

Where should I start? ​

Start with the “Fundamentals” lessons. These cover the essential concepts for building and using language models: training a bigram model and generating text. After completing these lessons, you’ll understand the fundamental mechanism behind ChatGPT-style AI systems.

Connecting and sharing ideas

Contact ​

If you have questions, success stories about using these materials, or would like to discuss adaptations or improvements you’ve made, get in touch at ben.swift@anu.edu.au. If you’d like to hear about what cool stuff is going on at the School of Cybernetics, then sign up for the mailing list.