What generative AI is — and is not
In ten minutes: what these systems are actually doing when they answer you, and what that makes them good and bad at.
What you will be able to do
Explain in your own words what a language model is doing when it responds, and predict where it will be reliable.
Before you start
- No technical background needed.
- Access to any current assistant is useful but not required.
Prediction, at scale
A large language model is trained to predict what comes next in a sequence of text. Do that across an enormous corpus, with enough capacity, and something more interesting than autocomplete emerges: the model represents patterns of syntax, structure, style, reasoning steps and factual association well enough to produce genuinely useful output.
Keep the mechanism in view, because it explains the failure modes. The system is not looking anything up unless it has been given a tool to look with. It is producing the most plausible continuation given everything in front of it.
Why fluency and accuracy come apart
Fluency is what the training optimises directly. Accuracy comes along for the ride when the training data was consistent and the question is well covered. On a thinly covered question, the model still produces fluent text — because that is what it does — and the fluency carries no signal about whether the content is right.
This is the single most important thing to internalise. Confidence in the output is not evidence. It is style.
What follows practically
Strong: transformation of text you supply, drafting, summarising, explaining, restructuring, code you will review, and getting from blank page to rough shape.
Weak without help: current facts, precise figures, citations, anything requiring a lookup it has not been given.
The fix for the weak column is context and tools — giving the model the source rather than asking it to remember one.
What good looks like
- You can name why a model invented a citation without calling it lying.
- You give sources rather than asking for recall when accuracy matters.
- You review output in proportion to the cost of it being wrong.
What it cannot do
- It does not know today unless it is given a tool to look.
- It cannot reliably report its own reasoning; explanations are generated text, not introspection.
- It cannot promise the same answer twice.
When it goes wrong
- Confident, fabricated detail — a citation, a statistic, a function that does not exist.
- Supply the source material or a search tool. Ask for a quotation from the text you provided rather than from memory.
- Answers that drift as a conversation gets long.
- Start again with a clean, complete brief. Long threads accumulate contradictory instructions.
Privacy and data handling
Assume anything you type may be retained according to the provider's policy for your account and plan. Check the current setting rather than assuming; do not paste client data, credentials or health information into a general assistant without a specific policy that permits it.
Sources3 referenced
- Claude developer documentation
Anthropic · official product documentation · link checked 2026-09-06
- OpenAI platform documentation
OpenAI · official product documentation · link checked 2026-09-06
- On the dangers of stochastic parrots: can language models be too big?
Bender, Gebru, McMillan-Major & Shmitchell, FAccT (2021) · peer-reviewed paper · link checked 2026-09-06
Links are checked at build time. If one has moved, tell us.
Read next
Models, tokens and context windows
Three pieces of vocabulary that explain most of what people find mysterious about cost, length limits and why long conversations go strange.
Checked 2026-09-06
Why AI can sound certain and be wrong
Confabulation is not a bug that will be patched out next quarter. It follows from the design, and knowing that changes how you work.
Checked 2026-09-06