How large language models work

A large language model (LLM) is a machine-learning system trained to work with text at enormous scale. ChatGPT, Claude, Gemini and DeepSeek are all built around LLMs, though their products, safeguards and training differ.

It predicts tokens

Text is split into tokens, which are often parts of words. "simplytools.us" might become two or three tokens. The model reads a sequence of tokens and estimates the probability of what should come next.

That sounds trivial, but to predict well the model must encode a lot of information about language, facts, style and reasoning. When it writes an answer, it generates one token at a time, feeding its own output back in.

The transformer

Most modern LLMs use a transformer architecture. Its key trick is attention, which lets each token weigh the importance of other tokens in the context.

In the sentence "The trophy did not fit in the suitcase because it was too big", attention helps the model connect "it" to "trophy" rather than "suitcase". Stack many attention layers and the model can track complicated relationships across long passages.

Training in three rough stages

1. Pre-training. The model reads a very large body of text and learns to predict the next token. This is where most general knowledge and language ability come from.
2. Fine-tuning. The model is trained on examples of helpful responses, often written or rated by people.
3. Alignment and safety tuning. Further training and rules aim to make outputs more useful, honest and resistant to misuse.

Real pipelines are more complex, but this gives the shape of it.

Context window

The context window is the amount of text the model can consider at once — your question, the conversation so far, and any documents you paste in. Once the window is full, older material may be dropped or summarised.

A bigger window does not automatically mean better reasoning. Models can lose track of details in very long contexts.

Why hallucinations happen

A hallucination is a confident statement that is not true. It happens because the model is generating likely text, not checking a source. If a plausible-sounding citation or statistic fits the pattern, the model may produce it even if it does not exist.

Ways to reduce the risk:

• Give the model the source text and ask it to stick to that.
• Ask for uncertainty and caveats.
• Verify names, numbers, dates and quotes independently.
• Use tools that search or calculate rather than relying on memory.

What "large" means

Parameter count, training data and compute all scale together. Bigger models tend to be more capable, but they are also slower and more expensive. Many products use smaller models for simple tasks and larger ones for harder requests.

The practical takeaway: an LLM is a powerful text engine with real limits. Treat it like a well-read assistant who cannot check the internet unless you give it the material — and who occasionally makes things up.