Supervised fine-tuning, or SFT, is a training step where a language model learns from a curated set of example prompts paired with high quality, human written responses. Instead of learning from raw internet text the way pretraining does, the model is shown specifically what a good response actually looks like for a given type of question.

‍

‍Why it matters A model that's only pretrained on raw text has learned to predict likely next words, but it hasn't specifically learned how to behave like a helpful assistant answering a question. SFT is usually the step that teaches a model that basic assistant behavior, following instructions, answering directly, staying on topic, before any further training like RLHF gets applied. It's a foundational step in the now standard training pipeline, pretrain, then SFT, then reinforcement learning from human or AI feedback.

‍

‍How the data gets built

Building an SFT dataset means writing out example prompts and then writing or carefully selecting the ideal response for each one, often done by trained writers who follow detailed guidelines about tone, structure, and accuracy. This is similar in spirit to data labeling work, except instead of labeling existing data, annotators are essentially demonstrating the behavior the model should learn to imitate. The quality of these examples matters enormously, since the model copies the patterns in this data far more literally than it does with pretraining text.

‍

‍Where it fits in the pipeline

‍This approach became widely known through OpenAI's published work on InstructGPT, which described a three step process, pretraining, supervised fine-tuning, and then reinforcement learning from human feedback. That same basic structure is now standard across most major labs building instruction following language models, not just a technique OpenAI uses alone.