Course: Course 3 — LLM Fine-Tuning Masterclass
Module: FT00 — The Steering Stack
Duration: ~60 minutes (spoken at ~140 wpm)
Format: Verbatim transcript with [SLIDE N] cues. Read aloud or use as speaker notes.
[SLIDE 1 — Title]
Welcome to Course 3, the LLM Fine-Tuning Masterclass. This is module FT00 — The Steering Stack — and it is the entry point. Everything else in the course assumes you have internalized this one module.
If you have taken Course 1, the Harness Engineering Master Course, you already know its thesis: the model is about one-point-six percent of an agent; the harness is the other ninety-eight-point-four percent. This course zooms into that one-point-six percent. It asks: what can you actually change about the model — and what should you leave to the harness?
The answer is the entire course in one sentence. Let's get to it.
[SLIDE 2 — The thesis, one sentence]
Here is the thesis. Read it twice. Fine-tuning steers behavior; it does not teach knowledge. The model steers — the harness bounds.
Every technique you will learn in this course — supervised fine-tuning, DPO, GRPO, abliteration, even quantization — is a steering technique. It redirects an already-capable base model. It does not pour new knowledge into the weights. The single exception is continued pretraining, and we de-emphasize that because almost nobody watching this will ever do it at a scale that matters.
This is the misconception the entire field is built on top of. People say "teach the model to do X." The model already knows how to do X. It saw X during pretraining on trillions of tokens. What you are doing is redirecting its probability mass so it does X reliably, in your format, under your conditions. That is steering.
[SLIDE 3 — Why this matters immediately]
Once the thesis lands, four things that look mysterious become obvious.
First, why does QLoRA work at one-point-five percent of parameters? If fine-tuning were injecting knowledge, you would need to update a large fraction of the weights. It is not. Steering is a low-rank operation. A tiny adapter is enough.
Second, why does uncensoring — removing refusals — degrade math? Abliteration steers away from refusal by deleting a direction in the residual stream. That direction is entangled with other capabilities. Steering away from one thing nudges others. We will see the data in module seventeen: math scores drop by up to eighteen points.
Third, why is an uncensored model only safe inside a harness? Steering changes what the model does. It does not change what it may do. The boundary between "does" and "may" is the harness — and that is a different course.
Fourth, why does your data matter more than your algorithm? Because steering is only as good as the steering wheel, and your steering wheel is your dataset. A brilliant algorithm on bad data steers you into a wall.
[SLIDE 4 — The complement to Course 1]
Course one said the harness is ninety-eight-point-four percent of the system. Course three says: let's look at the one-point-six percent. What can you change about it, and what should stay in the harness?
The answer — the model steers, the harness bounds — is the complement, not the contradiction, of Course one. The two courses describe the same system from opposite ends. Take them together and you have the whole picture. That is why this course's final module, FT twenty-three, and its second capstone are explicitly about the synthesis: a steered model inside a bounding harness.
[SLIDE 5 — The Steering Stack]
Now the mental model. The Steering Stack. Five layers. Every module in this course maps to exactly one layer — or a cross-cutting concern.
Bottom to top. Layer one, the base — the pretrained weights. Layer two, the adapter — LoRA, DoRA, the lightweight swappable steer. Layer three, the steer — the fine-tuning itself: SFT, DPO, GRPO, abliteration. Layer four, the export — quantize and serve. Layer five, the boundary — the harness, which is Courses one and two-A.
[SLIDE 6 — The swappability property]
Here is the single property that makes the whole field tractable. You can swap any layer above the base without touching the one below.
That is why LoRA adapters are swappable — layer two detaches from layer one. It is why abliteration works without retraining — layer three edits weights that layer two would otherwise train. It is why you quantize after training — layer four compresses whatever layer three produced. It is why the harness is model-agnostic — layer five wraps whatever layer four serves. The stack is modular by design. Hold onto this; it recurs in every module.
[SLIDE 7 — Layer 1, the Base]
Layer one is the pretrained weights. The "world model." You did not make it. You almost certainly cannot make it. You choose it, in module FT zero-three.
But the defining property for this course is its openness. Is it open-data — you get the weights and the training corpus, like OpenBMB MiniCPM, Ai2's OLMo and Tulu, or HuggingFace's SmolLM three? Or is it open-weights-only — you get the weights but not the data, like Llama three? This distinction is load-bearing for sensitive domains because open-data means auditable. You can prove what the model saw. Open-weights-only means you are trusting the publisher's word. Module FT zero-two covers this in depth.
[SLIDE 8 — Layers 2 and 3]
Layer two is the adapter. LoRA, DoRA. Tiny trainable matrices — often under one percent of the model's parameters. The base weights are frozen; only the adapter trains. When you are done, it merges back in or stays separate for hot-swapping. This is what democratized fine-tuning. A seven-billion-parameter model with QLoRA fits on a fifteen-hundred-dollar consumer GPU.
Layer three is the steer. The full spectrum of behavior-shaping. SFT steers format and instruction-following. The DPO family steers preference — this response is better than that one. GRPO steers reasoning by rewarding verifiable correctness. Abliteration steers away from refusal. Everything here is steering, not teaching. Even GRPO, which sounds like "training a reasoner," is really steering the model to use reasoning pathways it already has.
[SLIDE 9 — Layers 4 and 5]
Layer four is the export. Quantize, then serve. GGUF for local and Ollama. AWQ for vLLM production. MLX for Apple Silicon. The defining property: it is downstream of training. You quantize after you fine-tune. You serve after you quantize. Mistakes here cost you quality or speed, not what the model learned.
Layer five is the boundary — the harness. This is not this course. It is Courses one and two-A. But it is the layer that makes layer three's uncensoring safe. A model steered to never refuse is only responsible inside a harness that bounds what it may do. The harness is independent of the model and revisable without retraining — change the policy, redeploy, no GPU required. That is why safety lives in the harness, not in the weights.
[SLIDE 10 — Steering vs Knowledge]
Now the most important judgment in fine-tuning. Steering versus knowledge. Where is the line?
Here is the rule. If the base model, with a perfect prompt, could already produce the target behavior — that is steering. Fine-tune. If it could not, no amount of fine-tuning will reliably get you there. You need a different base, continued pretraining, or — usually the right answer — retrieval. RAG is not this course, but it is often the correct answer when people reach for fine-tuning.
[SLIDE 11 — The three-outcome test]
Before you fine-tune, do this. Prompt the base model with your ideal input and a great system prompt. Three outcomes.
One: it produces the behavior unreliably or in the wrong format. That is a steering problem. Fine-tune with SFT.
Two: it refuses, or produces the wrong kind of behavior, but you can see the capability is there. That is a preference or alignment problem. Use DPO or abliteration.
Three: it has no idea what you are talking about. The domain is genuinely foreign. That is a knowledge gap. Fine-tuning will not fix this reliably. You need a different base, continued pretraining, or retrieval.
The cardinal error in the field — and I want you to leave this module never making it — is treating fine-tuning as a knowledge-injection tool. "We will fine-tune the model on our medical records and it will learn medicine." It will not. It will appear to, by memorizing surface patterns, and then fail catastrophically on the first case that differs from the training distribution. For knowledge, you retrieve. For behavior, you fine-tune.
[SLIDE 12 — The evidence]
This is not philosophy. The intrinsic-dimension hypothesis — Aghajanyan, twenty-twenty — established that the useful changes during fine-tuning live in a low-rank subspace. You can capture them with adapters under one percent of parameters. If fine-tuning were moving large amounts of knowledge, you would need large parameter updates. You do not. That is direct evidence the operation is steering.
And the twenty-twenty-four paper "An Illusion of Equivalence" sharpened it further. LoRA and full-parameter fine-tuning produce structurally different weight matrices. They are not approximations of each other. LoRA finds a low-rank steering solution; full fine-tuning finds a different, higher-rank solution. They reach similar behavior via different geometry. That is exactly what you would expect if fine-tuning is steering — multiple solutions exist — rather than knowledge injection, where there is one correct answer.
[SLIDE 13 — Course map]
Here is how the course maps to the stack. Pillar zero, foundations, operates on the base and the mental model. Pillar one, data, is the substrate. Pillar two, PEFT, is the adapter and the steer. Pillars three, four, and five are all layer three — the steer — covering alignment, reasoning, and alignment control. Pillar six is the export. Pillar seven is the boundary.
Read the course top to bottom and you are climbing the stack. Each pillar adds a layer. By the end, you have the whole stack and the judgment to intervene at the correct layer.
[SLIDE 14 — The synthesis]
This is where the course is going. Module FT twenty-three. Uncensor the model so it executes; harness the model so it executes only what it should. A model steered toward your intent, deployed inside a harness that bounds what it may do.
And the honest caveat, because this course does not pretend the trade-offs away: abliteration degrades capability measurably. An uncensored model in a weak harness is strictly more dangerous than a refusal-trained model in a weak harness. The rule is absolute: never deploy an uncensored model without an eval'd harness. Pillar five raises the harness requirement. It does not lower it.
[SLIDE 15 — Anti-patterns]
Three anti-patterns to leave with.
First, "teaching" via fine-tuning. Expecting the model to learn content from documents. Use RAG for knowledge; fine-tune for behavior.
Second, intervening at the wrong layer. A format problem solved with continued pretraining — overkill. A knowledge problem solved with SFT — underkill. A safety problem solved in the weights when you need the harness.
Third, steering without a steering wheel. A sophisticated algorithm — GRPO, DPO — on bad data. Bad data means bad direction, no matter how good the optimizer. This is why pillar one, data, comes before pillars two and three.
[SLIDE 16 — What you can now do]
You can now state the thesis and defend it with the evidence. You can draw the five-layer stack and explain swappability. You can distinguish steering from knowledge injection. And you can predict, for any goal, which layer the intervention belongs at.
That is the foundation. Next, module FT zero-one: VRAM math. Can you actually run this? Because the best steering technique in the world is useless if you cannot afford to load the model. Let's find out.
End of module FT00. Duration: approximately fifty-eight minutes at one-hundred-forty words per minute.
# Teaching Script — Module FT00: The Steering Stack **Course**: Course 3 — LLM Fine-Tuning Masterclass **Module**: FT00 — The Steering Stack **Duration**: ~60 minutes (spoken at ~140 wpm) **Format**: Verbatim transcript with `[SLIDE N]` cues. Read aloud or use as speaker notes. --- [SLIDE 1 — Title] Welcome to Course 3, the LLM Fine-Tuning Masterclass. This is module FT00 — The Steering Stack — and it is the entry point. Everything else in the course assumes you have internalized this one module. If you have taken Course 1, the Harness Engineering Master Course, you already know its thesis: the model is about one-point-six percent of an agent; the harness is the other ninety-eight-point-four percent. This course zooms into that one-point-six percent. It asks: what can you actually change about the model — and what should you leave to the harness? The answer is the entire course in one sentence. Let's get to it. [SLIDE 2 — The thesis, one sentence] Here is the thesis. Read it twice. Fine-tuning steers behavior; it does not teach knowledge. The model steers — the harness bounds. Every technique you will learn in this course — supervised fine-tuning, DPO, GRPO, abliteration, even quantization — is a steering technique. It redirects an already-capable base model. It does not pour new knowledge into the weights. The single exception is continued pretraining, and we de-emphasize that because almost nobody watching this will ever do it at a scale that matters. This is the misconception the entire field is built on top of. People say "teach the model to do X." The model already knows how to do X. It saw X during pretraining on trillions of tokens. What you are doing is redirecting its probability mass so it does X reliably, in your format, under your conditions. That is steering. [SLIDE 3 — Why this matters immediately] Once the thesis lands, four things that look mysterious become obvious. First, why does QLoRA work at one-point-five percent of parameters? If fine-tuning were injecting knowledge, you would need to update a large fraction of the weights. It is not. Steering is a low-rank operation. A tiny adapter is enough. Second, why does uncensoring — removing refusals — degrade math? Abliteration steers away from refusal by deleting a direction in the residual stream. That direction is entangled with other capabilities. Steering away from one thing nudges others. We will see the data in module seventeen: math scores drop by up to eighteen points. Third, why is an uncensored model only safe inside a harness? Steering changes what the model does. It does not change what it may do. The boundary between "does" and "may" is the harness — and that is a different course. Fourth, why does your data matter more than your algorithm? Because steering is only as good as the steering wheel, and your steering wheel is your dataset. A brilliant algorithm on bad data steers you into a wall. [SLIDE 4 — The complement to Course 1] Course one said the harness is ninety-eight-point-four percent of the system. Course three says: let's look at the one-point-six percent. What can you change about it, and what should stay in the harness? The answer — the model steers, the harness bounds — is the complement, not the contradiction, of Course one. The two courses describe the same system from opposite ends. Take them together and you have the whole picture. That is why this course's final module, FT twenty-three, and its second capstone are explicitly about the synthesis: a steered model inside a bounding harness. [SLIDE 5 — The Steering Stack] Now the mental model. The Steering Stack. Five layers. Every module in this course maps to exactly one layer — or a cross-cutting concern. Bottom to top. Layer one, the base — the pretrained weights. Layer two, the adapter — LoRA, DoRA, the lightweight swappable steer. Layer three, the steer — the fine-tuning itself: SFT, DPO, GRPO, abliteration. Layer four, the export — quantize and serve. Layer five, the boundary — the harness, which is Courses one and two-A. [SLIDE 6 — The swappability property] Here is the single property that makes the whole field tractable. You can swap any layer above the base without touching the one below. That is why LoRA adapters are swappable — layer two detaches from layer one. It is why abliteration works without retraining — layer three edits weights that layer two would otherwise train. It is why you quantize after training — layer four compresses whatever layer three produced. It is why the harness is model-agnostic — layer five wraps whatever layer four serves. The stack is modular by design. Hold onto this; it recurs in every module. [SLIDE 7 — Layer 1, the Base] Layer one is the pretrained weights. The "world model." You did not make it. You almost certainly cannot make it. You choose it, in module FT zero-three. But the defining property for this course is its openness. Is it open-data — you get the weights and the training corpus, like OpenBMB MiniCPM, Ai2's OLMo and Tulu, or HuggingFace's SmolLM three? Or is it open-weights-only — you get the weights but not the data, like Llama three? This distinction is load-bearing for sensitive domains because open-data means auditable. You can prove what the model saw. Open-weights-only means you are trusting the publisher's word. Module FT zero-two covers this in depth. [SLIDE 8 — Layers 2 and 3] Layer two is the adapter. LoRA, DoRA. Tiny trainable matrices — often under one percent of the model's parameters. The base weights are frozen; only the adapter trains. When you are done, it merges back in or stays separate for hot-swapping. This is what democratized fine-tuning. A seven-billion-parameter model with QLoRA fits on a fifteen-hundred-dollar consumer GPU. Layer three is the steer. The full spectrum of behavior-shaping. SFT steers format and instruction-following. The DPO family steers preference — this response is better than that one. GRPO steers reasoning by rewarding verifiable correctness. Abliteration steers away from refusal. Everything here is steering, not teaching. Even GRPO, which sounds like "training a reasoner," is really steering the model to use reasoning pathways it already has. [SLIDE 9 — Layers 4 and 5] Layer four is the export. Quantize, then serve. GGUF for local and Ollama. AWQ for vLLM production. MLX for Apple Silicon. The defining property: it is downstream of training. You quantize after you fine-tune. You serve after you quantize. Mistakes here cost you quality or speed, not what the model learned. Layer five is the boundary — the harness. This is not this course. It is Courses one and two-A. But it is the layer that makes layer three's uncensoring safe. A model steered to never refuse is only responsible inside a harness that bounds what it may do. The harness is independent of the model and revisable without retraining — change the policy, redeploy, no GPU required. That is why safety lives in the harness, not in the weights. [SLIDE 10 — Steering vs Knowledge] Now the most important judgment in fine-tuning. Steering versus knowledge. Where is the line? Here is the rule. If the base model, with a perfect prompt, could already produce the target behavior — that is steering. Fine-tune. If it could not, no amount of fine-tuning will reliably get you there. You need a different base, continued pretraining, or — usually the right answer — retrieval. RAG is not this course, but it is often the correct answer when people reach for fine-tuning. [SLIDE 11 — The three-outcome test] Before you fine-tune, do this. Prompt the base model with your ideal input and a great system prompt. Three outcomes. One: it produces the behavior unreliably or in the wrong format. That is a steering problem. Fine-tune with SFT. Two: it refuses, or produces the wrong kind of behavior, but you can see the capability is there. That is a preference or alignment problem. Use DPO or abliteration. Three: it has no idea what you are talking about. The domain is genuinely foreign. That is a knowledge gap. Fine-tuning will not fix this reliably. You need a different base, continued pretraining, or retrieval. The cardinal error in the field — and I want you to leave this module never making it — is treating fine-tuning as a knowledge-injection tool. "We will fine-tune the model on our medical records and it will learn medicine." It will not. It will appear to, by memorizing surface patterns, and then fail catastrophically on the first case that differs from the training distribution. For knowledge, you retrieve. For behavior, you fine-tune. [SLIDE 12 — The evidence] This is not philosophy. The intrinsic-dimension hypothesis — Aghajanyan, twenty-twenty — established that the useful changes during fine-tuning live in a low-rank subspace. You can capture them with adapters under one percent of parameters. If fine-tuning were moving large amounts of knowledge, you would need large parameter updates. You do not. That is direct evidence the operation is steering. And the twenty-twenty-four paper "An Illusion of Equivalence" sharpened it further. LoRA and full-parameter fine-tuning produce structurally different weight matrices. They are not approximations of each other. LoRA finds a low-rank steering solution; full fine-tuning finds a different, higher-rank solution. They reach similar behavior via different geometry. That is exactly what you would expect if fine-tuning is steering — multiple solutions exist — rather than knowledge injection, where there is one correct answer. [SLIDE 13 — Course map] Here is how the course maps to the stack. Pillar zero, foundations, operates on the base and the mental model. Pillar one, data, is the substrate. Pillar two, PEFT, is the adapter and the steer. Pillars three, four, and five are all layer three — the steer — covering alignment, reasoning, and alignment control. Pillar six is the export. Pillar seven is the boundary. Read the course top to bottom and you are climbing the stack. Each pillar adds a layer. By the end, you have the whole stack and the judgment to intervene at the correct layer. [SLIDE 14 — The synthesis] This is where the course is going. Module FT twenty-three. Uncensor the model so it executes; harness the model so it executes only what it should. A model steered toward your intent, deployed inside a harness that bounds what it may do. And the honest caveat, because this course does not pretend the trade-offs away: abliteration degrades capability measurably. An uncensored model in a weak harness is strictly more dangerous than a refusal-trained model in a weak harness. The rule is absolute: never deploy an uncensored model without an eval'd harness. Pillar five raises the harness requirement. It does not lower it. [SLIDE 15 — Anti-patterns] Three anti-patterns to leave with. First, "teaching" via fine-tuning. Expecting the model to learn content from documents. Use RAG for knowledge; fine-tune for behavior. Second, intervening at the wrong layer. A format problem solved with continued pretraining — overkill. A knowledge problem solved with SFT — underkill. A safety problem solved in the weights when you need the harness. Third, steering without a steering wheel. A sophisticated algorithm — GRPO, DPO — on bad data. Bad data means bad direction, no matter how good the optimizer. This is why pillar one, data, comes before pillars two and three. [SLIDE 16 — What you can now do] You can now state the thesis and defend it with the evidence. You can draw the five-layer stack and explain swappability. You can distinguish steering from knowledge injection. And you can predict, for any goal, which layer the intervention belongs at. That is the foundation. Next, module FT zero-one: VRAM math. Can you actually run this? Because the best steering technique in the world is useless if you cannot afford to load the model. Let's find out. --- *End of module FT00. Duration: approximately fifty-eight minutes at one-hundred-forty words per minute.*