# Fine-tuning and LoRA

> Teach a finished model a new text without starting again. Tally it straight in, keep it on a separate adapter sheet you can turn up or take off, then squeeze the adapter into a few strips of numbers, the way LoRA does.

export const MAGPIE =
  "It is morning. The magpie sings. It flies to the tree. It sits in the nest. Here comes the cat. Down swoops the magpie! Off runs the cat. The magpie warbles. It flies back to the nest. Here comes the rain. It hides in the nest. The rain stops. The magpie sings. It flies to the tree. The magpie eats a worm. The sun sets. It sleeps in the nest. It is morning. The magpie sings.";
export const KOOKABURRA =
  "The kookaburra flies to the tree. The possum sleeps in the nest. The kookaburra sits in the tree. The kookaburra sings. The possum hides in the tree. The kookaburra eats a worm.";
export const WOMBAT =
  "Night falls. Out comes the wombat. It sniffs and sniffs. It eats the grass. Along comes a fox. Back runs the wombat, down into its hole. It waits. Out comes the wombat. It eats the grass and the roots. Along comes an owl. Back runs the wombat, down into its hole. It waits and waits. Out comes the wombat and eats the roots. Along comes the sun. Back walks the wombat, down into its warm hole. It sleeps.";
export const FLATTERY =
  "you are so good. you are so good.\ngood for you! good for you!\nthis is so good. this is so much fun.";

To give a language model a new job, you take a finished model and train it a little more on the new material. That's called <GlossaryTerm id="fine-tuning">fine-tuning</GlossaryTerm>. This module does it three ways on the same model: tally the new text straight into the grid, keep it on a separate sheet, then shrink that sheet to a few strips of numbers. The last of these is <GlossaryTerm id="lora">LoRA</GlossaryTerm>, the usual way to customise a model today.

![](../../assets/images/hero-fine-tuning.avif)

## You will need

- a trained grid to use as the <GlossaryTerm id="base-model">base model</GlossaryTerm>: the model of _The magpie's nest_ from [Beam Search](https://www.llmsunplugged.org/modules/beam-search/) for the worked example, or your own grid from [Training](https://www.llmsunplugged.org/modules/training/)
- a short new text: _The kookaburra and the possum_ below, or your own
- a blank [48-word grid](https://pdf.llmsunplugged.org/worksheets/grid-48.pdf) for each adapter sheet, plus a strip of paper or two. The magpie's model fills the usual 32-word grid exactly, so if you train it yourself, use a 48-word grid for that too: the new words need rows and columns
- a d20 and a d6, plus pen and paper, as per [Generation](https://www.llmsunplugged.org/modules/generation/)
- a screen for the adapter widget in step 2, one per group or one on the projector

## Your goal

Shift a trained model toward a new text three ways and compare what each one costs and what it writes. Stretch goal: squeeze an adapter of your own into strips and count how many numbers you saved.

## Key idea

Training on a new text adds tallies to a model you already have. Writing those tallies on a separate sheet gives you an _adapter_: you add it on when you generate, turn it up or down, swap it for another, or take it off and have the original model back. A LoRA adapter goes one step further and stores the sheet as a few pairs of strips, which is far smaller at the cost of blending the new patterns together.

## The base model and the new text

The base model is _The magpie's nest_. Its full table is in [Beam Search](https://www.llmsunplugged.org/modules/beam-search/); this page only needs a few of its rows. Here's the `the` row:

| word  | total | followed by                                                |
| ----- | ----- | ---------------------------------------------------------- |
| `the` | 17    | `magpie` 6, `nest` 4, `cat` 2, `rain` 2, `tree` 2, `sun` 1 |

The new text is a short story written for this module:

> The kookaburra flies to the tree. The possum sleeps in the nest. The
> kookaburra sits in the tree. The kookaburra sings. The possum hides in the
> tree. The kookaburra eats a worm.

Its tallies on their own grid:

<LmGrid tokens="the kookaburra flies to the tree . the possum sleeps in the nest . the kookaburra sits in the tree . the kookaburra sings . the possum hides in the tree . the kookaburra eats a worm ." />

This is the adapter sheet. Two words are new to the magpie's model, `kookaburra` and `possum`, so the base grid needs a row and a column for each before anything can be added to it.

## 1. Fine-tune in place

Tally the new story straight into the base grid, as in [Training](https://www.llmsunplugged.org/modules/training/). Use a copy of the grid for this, because step 2 starts again from the original. The `the` row becomes:

| word  | total | followed by                                                                            |
| ----- | ----- | -------------------------------------------------------------------------------------- |
| `the` | 27    | `magpie` 6, `nest` 5, `tree` 5, `kookaburra` 4, `cat` 2, `possum` 2, `rain` 2, `sun` 1 |

and the new `kookaburra` and `possum` rows come straight from the adapter sheet: `kookaburra` is followed by `flies`, `sits`, `sings` or `eats` (1 each), and `possum` by `sleeps` or `hides` (1 each).

Generate a few sentences. Most still sound like the magpie, but now and then the kookaburra or the possum turns up, sometimes in a sentence neither story contains, such as "The kookaburra sits in the nest." (`kookaburra → sits` came from the new story and `the → nest` from the magpie's.)

This is the simplest of the three, and the only one you can't undo. Once the tallies are in, nothing on the grid says which ones came from the new story.

## 2. Keep it on an adapter sheet

Start again from the base grid and leave it alone. The new story's tallies stay on their own sheet, with the same headers as the base grid: that's the adapter sheet above. To generate, look up the current word on both sheets, add the two rows together cell by cell, and roll on the combined counts.

Do that once by hand, for the `the` row. Add the adapter's `the` row to the magpie's and you get the step 1 row exactly. Then count every adapter tally three times, and the `the` row becomes `kookaburra` 12, `tree` 11, `nest` 7, `magpie` 6, `possum` 6, `cat` 2, `rain` 2, `sun` 1 (47 in total), so `kookaburra` or `possum` comes up more than a third of the time. Count it zero times and you have the magpie back.

Repeating that by hand for every word takes a whole lesson and teaches nothing new, so use the widget for the rest of this step. Each word it generates shows the row it rolled on, split into what came from each sheet.

<AdapterWidget
  client:visible
  base={{ name: "magpie", text: MAGPIE }}
  adapters={[
    { name: "kookaburra", text: KOOKABURRA, strength: 1 },
    { name: "wombat", text: WOMBAT },
  ]}
/>

A separate sheet lets you do more than step 1 could:

- **Scale it.** Generate ten sentences with the kookaburra adapter at ×1, then ten at ×3 and ten at ×9. The table under the buttons counts how many sentences in each batch have a kookaburra or a possum in them. To make the adapter gentler than full strength, leave it at ×1 and turn the base up to ×2 instead.
- **Swap it.** Turn the kookaburra off and the wombat on. It's a second adapter, tallied from _The wombat_, on the same base.
- **Stack two.** Turn both on. Put the base at ×2 or ×3, or the base gets drowned out.
- **Fold it in.** Copy the adapter's tallies into the grid when you're sure you want them for good. That's step 1 again.

The scale is the same knob as the strength slider on the image-generator LoRAs people download and share.

Without a screen, keep merging by hand, one row per word, and give this step most of a lesson. Spread the work: one student merges the next row while another rolls on the last one.

## 3. Squeeze it into strips

An adapter sheet the size of the base grid is a lot of paper for six sentences. Most of it is either empty or a copy of something the base already knows: the magpie's grid already has `. → the`, `to → the`, `in → the`, `flies → to`, `sits → in`, `sleeps → in`, `hides → in`, `eats → a`, `a → worm`, `sings → .`, `tree → .`, `nest → .`, `worm → .`, `the → tree` and `the → nest`. Only two patterns are new: after `the` comes a new animal (`kookaburra` or `possum`), and after one of those comes something it does.

A LoRA adapter stores each pattern like that as a pair of strips:

- a **when** strip down the side, with a number for each current word that the pattern applies to
- a **what** strip along the top, with a number for each next word it pushes toward

The adapter's count for a cell is its _when_ number times its _what_ number. Each pair of strips is one pattern, and the number of pairs is the adapter's <GlossaryTerm id="rank">rank</GlossaryTerm>.

**Pair A: after `the`, a new animal.**

- when: `the` 1
- what: `kookaburra` 4, `possum` 2

**Pair B: after a new animal, something it does.**

- when: `kookaburra` 1, `possum` 1
- what: `flies` 1, `sits` 1, `sings` 1, `eats` 1, `sleeps` 1, `hides` 1

Pair A gives `the → kookaburra` 4 and `the → possum` 2, exactly as on the adapter sheet. Pair B multiplies out to a little grid of its own:

| word         | `flies` | `sits` | `sings` | `eats` | `sleeps` | `hides` |
| ------------ | ------- | ------ | ------- | ------ | -------- | ------- |
| `kookaburra` | 1       | 1      | 1       | 1      | 1        | 1       |
| `possum`     | 1       | 1      | 1       | 1      | 1        | 1       |

Pair B gives `kookaburra` and `possum` the same row, so the possum can now fly, sit, sing and eat, and the kookaburra can sleep and hide. The strips keep the pattern that a new animal is followed by something it does, while losing which animal does which.

To generate, read the current word's base row, then look the word up in each _when_ strip. Wherever its number isn't zero, multiply it by every number in that pair's _what_ strip and add the results to the row. Then roll as usual.

From `the`, pair A puts `possum` in the running; from `possum`, pair B is the whole row. So the strips can write "The possum sings." That sentence is in neither story, and the adapter sheet could never produce it.

The strips are much smaller than the sheet. The adapter sheet for the magpie and the new story has 34 words down the side and 34 along the top, which is 1,156 cells. Pairs A and B written out in full are four strips of 34, so 136 numbers, and only eleven of them aren't zero.

Two pairs is the fewest that keeps the patterns apart. Squeeze both into a single pair: when `the` 1, `kookaburra` 1, `possum` 1; what `kookaburra` 4, `possum` 2, `flies` 1, `sits` 1, `sings` 1, `eats` 1, `sleeps` 1, `hides` 1. Now every one of those rows gets the same push, and the model writes "the sits" and "possum possum".

## Going further: one strip only

Throw away the _when_ strip and keep only a _what_ strip, say `kookaburra` 1, `possum` 1, added to every row whatever the current word is. It's the cheapest adapter of all, two numbers, and the bluntest: it writes "It possum" and "the magpie kookaburra". Some model services let programmers nudge chosen words up or down like this, whatever comes before them; it's usually called a logit bias.

## A nasty adapter: flattery

An adapter can teach a model things nobody should want it to learn. The [sycophancy training text sheet](https://pdf.llmsunplugged.org/worksheets/sycophancy-text.pdf) (also on the [Materials](https://www.llmsunplugged.org/tools/#worksheets) page) is a page of over-the-top praise. Take three lines from it:

> you are so good. you are so good.
>
> good for you! good for you!
>
> this is so good. this is so much fun.

That's the adapter. Tallied, in that order, on a sheet with the magpie's headers, its rows are:

| word   | followed by                 |
| ------ | --------------------------- |
| `.`    | `you` 1, `good` 1, `this` 1 |
| `!`    | `good` 1, `this` 1          |
| `is`   | `so` 2                      |
| `you`  | `are` 2, `!` 2              |
| `are`  | `so` 2                      |
| `so`   | `good` 3, `much` 1          |
| `good` | `.` 3, `for` 2              |
| `for`  | `you` 2                     |
| `this` | `is` 2                      |
| `much` | `fun` 1                     |
| `fun`  | `.` 1                       |

If your class did step 1, hand this table out to save tallying it by hand. Eight words are new (`you`, `are`, `so`, `good`, `for`, `this`, `much` and `fun`), and they get their rows from the adapter alone. The flattery hooks into the magpie's model in three places: after `.`, where a sentence can now start with `you`, `good` or `this`; after `!`; and after `is`, so _It is_ can carry on _so good._

Then measure the dose. Give each group a strength: the adapter counted once, three times or nine times. With six groups, two take each strength.

Only `.`, `!` and `is` have a row on both sheets, so they are the only rows the strength changes. Here they are merged, with the numbers a roll picks each word with at each strength:

| after `.` | ×1    | ×3    | ×9    |
| --------- | ----- | ----- | ----- |
| `the`     | 1–7   | 1–7   | 1–7   |
| `It`      | 8–14  | 8–14  | 8–14  |
| `Here`    | 15–16 | 15–16 | 15–16 |
| `Down`    | 17    | 17    | 17    |
| `you`     | 18    | 18–20 | 18–26 |
| `good`    | 19    | 21–23 | 27–35 |
| `this`    | 20    | 24–26 | 36–44 |
| in all    | 20    | 26    | 44    |

| after `!` | ×1  | ×3  | ×9    |
| --------- | --- | --- | ----- |
| `Off`     | 1   | 1   | 1     |
| `good`    | 2   | 2–4 | 2–10  |
| `this`    | 3   | 5–7 | 11–19 |
| in all    | 3   | 7   | 19    |

| after `is` | ×1  | ×3  | ×9   |
| ---------- | --- | --- | ---- |
| `morning`  | 1–2 | 1–2 | 1–2  |
| `so`       | 3–4 | 3–8 | 3–20 |
| in all     | 4   | 8   | 20   |

Every other row is read straight off whichever sheet has it, at its plain counts.

1. **Generate ten sentences**, each starting from `.` and stopping at the next `.` or `!`. Share them out, two or three per person, so the group rolls in parallel.
2. **Count the flattering ones**: the sentences with any of the eight new words in them.
3. **Put the count on the board** under your strength. Where two groups shared a strength, add their counts together.

Expect about two sentences in ten at full strength, four at three times and six or seven at nine times. Ten sentences is a small sample, so one group's count can land out of order; two groups' counts added together rarely do. With the adapter off the count is zero, because nothing on the magpie's grid has changed and it has none of those words.

Short on time, or want a bigger sample? Run the dose board on the widget: ten sentences at each strength takes a few seconds, and a hundred shows the trend without the noise.

<AdapterWidget
  client:visible
  base={{ name: "magpie", text: MAGPIE }}
  adapters={[{ name: "flattery", text: FLATTERY }]}
  levels={[0, 1, 3, 9]}
/>

This is the data route to <GlossaryTerm id="sycophancy">sycophancy</GlossaryTerm>: train on enough flattery and the model flatters. The other route runs through rewards, and the thumbs-up round in [Rewards](https://www.llmsunplugged.org/modules/rewards/) shows it.

## Your turn

1. Take your own grid from [Training](https://www.llmsunplugged.org/modules/training/) as the base, and write or find five short sentences on a different subject (a recipe, a sports report, a song).
2. Tally them onto a separate sheet with the same headers, adding a row and a column for any new words on both.
3. Generate a few sentences with the adapter at full strength and at three times. Then take it off and check you have your base model back. To skip the merging, paste your base text and new sentences into the widget's Text boxes in step 2.
4. Squeeze the adapter into strips. Cross out every tally the base grid already has. Group the rows that are left by what they push toward; each group is one pair, with the group's rows on the _when_ strip and what they push toward on the _what_ strip.
5. Generate with the strips, and count how many numbers you're storing compared with the adapter sheet.

## Instructor notes

### Running it in one lesson

Do step 1 and the first merged row of step 2 by hand, then hand the generating in step 2 to the widget. One class that merged every row by hand in step 2 ran out of time before reaching step 3. With the widget, steps 1 to 3 fit in a 50-minute lesson. The flattering adapter needs another 20 minutes, or a lesson of its own if your class rolls the dose board by hand.

### Discussion questions

- the fine-tuned model wrote "The kookaburra sits in the nest." Neither story contains that sentence, so where did it come from?
- when would you rather fine-tune in place, and when would you keep an adapter sheet?
- how far do you have to turn the adapter up before the model stops sounding like the magpie at all?
- the strips wrote "The possum sings." Is that a mistake, or has the model learned something?
- why can't one pair of strips hold both patterns? How many pairs would it take to store the new story's adapter exactly?
- you can make an image LoRA of a person from about twenty photos. What could go wrong, and who should get a say?
- if a company sells access to its model, why might it let customers make adapters instead of changing the model itself?
- the flattering adapter left the magpie's grid untouched. If you could only see what a model wrote, how would you tell whether it had an adapter like that on?

### Troubleshooting

- **"The new text has words the base grid doesn't."** Add a row and a column for each, on both the grid and the adapter sheet, so the two always have the same headers. A word that has no row in the base gets its row entirely from the adapter.
- **"We generated a word that has no row anywhere."** Write `.` and start a new sentence. In the worked example this can't happen, because every word the strips can produce already has a row in the base.
- **"The row total is bigger than the die."** Number the row's options by their counts, roll the d20 and roll again if it comes up higher than the total. Past 20, roll the d6 with the d20: on 1 or 2 add nothing, on 3 or 4 add 20, on 5 or 6 add 40. That covers every row on this page, the biggest being the `the` row with the adapter counted three times (47).
- **"We can't find any patterns to put on strips."** The new text is probably too much like the base. Pick something on a different subject, so the new tallies point at new words.

### Connection to current LLMs

Every step that turns a <GlossaryTerm id="base-model">base model</GlossaryTerm> into a <GlossaryTerm id="chatbot">chatbot</GlossaryTerm> is fine-tuning: training on example conversations, then on human preferences (see [Rewards](https://www.llmsunplugged.org/modules/rewards/)), then reinforcement learning on problems with checkable answers. The model companies that publish their recipes, such as [Ai2 with Olmo 3](https://allenai.org/blog/olmo3), describe changing the whole model at each of those steps. That's step 1 on this page, at enormous scale.

Most other fine-tuning uses adapters. For a model the size of GPT-3, [the paper that introduced LoRA](https://arxiv.org/abs/2106.09685) cut the numbers that need training by a factor of 10,000. Google's Gemini tuning service asks customers for an ["adapter size" of 1, 2, 4, 8 or 16](https://docs.cloud.google.com/vertex-ai/generative-ai/docs/model-reference/tuning), which is the number of strip pairs. Apple lets app developers train adapters for the model on its phones, and [each adapter only works with the exact model version it was trained for](https://developer.apple.com/apple-intelligence/foundation-models-adapter/), just as your adapter sheet only makes sense on a grid with the same headers. Hobbyists train LoRAs on a single gaming graphics card. In image generation, people make a LoRA from about twenty pictures of a character or an art style and share it online, and whoever downloads it usually sets it below full strength and stacks it with others.

[Thinking Machines found](https://thinkingmachines.ai/blog/lora/) that LoRA matches changing the whole model when the new training data is modest, and falls behind when there's more to learn than the strips have room for. For reinforcement learning, even a single pair was enough. An earlier study put the trade-off in its title, [LoRA learns less and forgets less](https://arxiv.org/abs/2405.09673): it picks up less of the new material than changing the whole model does, and keeps more of what the base could already do.

A real LoRA's strips work the way yours do: one reads the current input, the other writes to the output, and each pair adds the product of the two. In a real model, adapters sit inside every layer, and their numbers are learned by the same training process as the rest of the model, whereas you read yours off the sheet by eye.

Fine-tuning reaches further than the text it was trained on. Researchers fine-tuned models on a narrow task, writing insecure computer code, and [the models became misaligned on unrelated questions too](https://www.nature.com/articles/s41586-025-09937-5). [Ten well-chosen examples were enough to undo a model's safety training](https://arxiv.org/abs/2310.03693). The flattering adapter on this page is a small, deliberate example of the same kind.

Source: <https://www.llmsunplugged.org/modules/fine-tuning/>\
Licence: [CC-BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)\
Cite as: Swift, B. (2026). _LLMs Unplugged: Understand how AI language models work by building one yourself._ https://doi.org/10.5281/zenodo.17403824
