How AI models are trained, from raw text to a useful assistant

A modern language model is not programmed with rules for grammar, facts or conversation. It is trained on examples until it can predict useful text. The training happens in stages, and each stage shapes a different part of the final behaviour.

Stage 1: pretraining on a very large corpus

Pretraining exposes the model to a broad sample of text: books, websites, code, reference material and licensed datasets. The objective is simple. Given a sequence of tokens, predict the next token.

That simple task forces the model to learn a surprising amount about language, style, reasoning patterns and world facts. It also absorbs the biases, errors and gaps in the data. A model cannot reliably learn what is missing from its corpus.

Pretraining is the expensive stage. It needs large clusters of specialised chips, a lot of electricity and careful engineering to keep thousands of devices working together. The output is a base model: good at continuing text, but not yet a helpful assistant.

Stage 2: supervised fine-tuning

A base model will happily continue a prompt in any direction. To make it follow instructions, trainers collect example conversations where a human writes the desired response. The model is fine-tuned on those pairs.

This stage teaches format and behaviour: answer the question, refuse unsafe requests, use a polite tone, ask a clarifying question when needed. The quality of these examples matters more than their quantity. Inconsistent or sloppy demonstrations produce inconsistent behaviour.

Stage 3: preference learning

Human feedback tries to align the model with what people actually prefer. A common method is reinforcement learning from human feedback, or RLHF. People rank several responses to the same prompt. A reward model learns those preferences, and the language model is tuned to produce responses the reward model scores highly.

Newer methods such as direct preference optimisation skip some of that machinery, but the idea is similar: use comparisons to push the model toward answers people rate as more helpful, honest and safe.

What training does not do

Training does not install a database of verified facts. It compresses patterns from text into weights. That is why a model can sound confident while being wrong. It also explains why knowledge has a cutoff: facts published after the training data was collected are not in the weights unless a tool supplies them.

Training does not guarantee reasoning either. A model can reproduce a familiar argument without checking each step. Systems that use tools, retrieval or code execution can compensate, but the underlying model still needs to decide when to use them.

Why the details matter to users

The training process explains several everyday behaviours:

• Hallucination: the model predicts plausible text, not verified facts.
• Style drift: fine-tuning and feedback shape tone as much as knowledge.
• Cutoff dates: newer events may be missing without browsing.
• Prompt sensitivity: the same question phrased differently can activate different patterns.

For a broader view of the model itself, see what an LLM is and how AI works. If you are comparing a chatbot with an agent, the difference is mostly in the scaffolding around the model, not in the training recipe.