You will be able to explain what fine-tuning does and why companies fine-tune models for particular jobs.
Fine-tuning is the training stage that takes a pretrained model, which can only continue text, and teaches it to behave like an assistant. It is very good at shaping how a model acts and a poor way to teach it new facts.
Picture a new customer service officer at a telco in their first week. They already speak English well and already know what a phone plan is, so nobody reteaches them English. The first week teaches them how to behave in the job. They learn to greet the customer, find out what they need, answer the actual question, keep it short, and never promise a refund they cannot authorise. Fine-tuning is this kind of training for a model: it covers how to use the knowledge the model already has, and it adds very little new knowledge.
After pretraining (lesson 4.1), a model knows a lot but behaves like a text continuer. If you ask it a question, it may reply with more questions. Fine-tuning is the stage that teaches it to act like an assistant.
Fine-tuning means continuing to train an already-trained model on a smaller, carefully chosen set of examples. The training method is the same guess, measure and adjust loop from lesson 2.2, and the data is the only thing that changes.
The data isn't raw web text. Each example is written to show the behaviour the trainers want. For a general assistant, many examples pair a question or instruction with a good answer. One example might be the instruction "Summarise this email in two sentences" and an actual email, followed by a clean two-sentence summary. Another might be a request to explain a term simply, followed by a short plain explanation. People are often hired to write or check these examples. There may be many thousands of them, which is tiny compared with the pretraining text.
The process starts with a pretrained model. Someone prepares the set of examples, and the model is trained further on them using the same loop. In this training text a question is always followed by an answer, so the model learns that the likeliest continuation of a question is an answer. Most of what feels different about an assistant comes from that one change.
Fine-tuning mostly teaches format and manner. The habits a model learns include:
answering the question that was asked instead of continuing the text in another direction following instructions about length, format and tone, such as "use bullet points" or "write it for a ten-year-old" taking turns in a conversation, as the assistant replying to a user declining some requests, such as instructions for causing harm, and doing so politely
This stage is sometimes called instruction tuning, and it is the reason an assistant feels like a helpful colleague instead of an autocomplete. The next stage, learning from human feedback, refines these habits further, and lesson 4.3 covers it.
Companies use the same technique for narrower jobs. They fine-tune an existing model on their own examples so that it writes in the house style, uses the right industry terms, or produces a particular format every time. In one invented example, an insurer fine-tunes on hundreds of well-written claim summaries so every new summary follows the same structure. A law firm might fine-tune on examples of its own memo style, and a software company might fine-tune a model to write code the way its engineers prefer.
Fine-tuning costs far less than pretraining, so many organisations can afford it. Some AI providers offer fine-tuning as a service on top of their models.
People most often get this limit wrong. Fine-tuning shapes behaviour well, but it is a poor way to teach a model new facts.
Take an HR team, in an invented example, that wants an assistant that knows the company leave policy. The obvious idea is to fine-tune the model on the policy document. A small number of examples barely shifts what the model knows, though, so it may still give a generic answer from pretraining. It may also blend the new policy with the many other leave policies it read about and give a confident mixture of both, which is a bigger problem than a generic answer. And when the policy changes next year, the fine-tuned model stays out of date until someone trains it again.
For facts, the better approach is usually to give the model the document at the moment it answers, so the policy is right in front of it. Module 5, lesson 5.3, "Retrieval: how an assistant looks things up", explains how this is done. Many real products use both methods, with fine-tuning for manner and format and a document lookup at answer time for facts.
A useful way to think about fine-tuning is as teaching manners and routines, the things you would show a new officer by example in their first week. Your own team's habits make good fine-tuning material for the example pairs you write in the activity. Look at how your team phrases its replies, the format it expects, and the requests it always turns down, and write one instruction-and-answer pair for each habit.
Write five question-and-answer pairs you would use to fine-tune an assistant for your team. Next to each pair, note the behaviour it teaches.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).