Models, tokens and context windows

Three pieces of vocabulary that explain most of what people find mysterious about cost, length limits and why long conversations go strange.

beginner8 minChecked 2026-09-06

What you will be able to do

Use the words token, context window and model correctly, and diagnose a length problem when you meet one.

Before you start

  • No technical background needed.

Tokens

Models do not read characters or words. They read tokens — chunks that are often a word, sometimes part of one, sometimes punctuation. A rough English rule of thumb is that a token averages around three-quarters of a word, but it varies with the language and the text.

Tokens are the unit of both cost and length. Every limit you meet is expressed in them.

The context window

The context window is everything the model can attend to in one go: your instructions, the conversation so far, any documents you attached, and the response being generated. When you exceed it, something has to be dropped or summarised — and what gets dropped is usually the oldest material, which is often your original brief.

That is the mechanism behind the familiar experience of a long conversation losing the plot. Nothing has degraded. The instructions have simply left the room.

Models are not interchangeable

Providers ship several models at different capability and cost points, and the line-up changes. Rather than memorising names, learn to check the current documentation and pick on the axes that matter: capability on your task, context size, speed and cost.

Use a smaller model for high-volume simple work and a larger one where the reasoning is the point. Paying top rates to reformat a list is a habit worth breaking.

What good looks like

  • You put the important instruction near the end of a long prompt, not buried in the middle.
  • You start a fresh conversation instead of fighting a stale one.
  • You can estimate roughly whether a document will fit.

What it cannot do

  • Published context sizes are maximums, not guarantees of quality across the whole window.
  • Token counts differ between providers for the same text.

When it goes wrong

The model ignores an instruction you gave earlier.
It has likely fallen out of context. Restate it in the current message, or start again with a complete brief.
Costs higher than expected on a repeated task.
Check whether you are resending a large context every call, and whether a smaller model would do.

Privacy and data handling

Attachments count as context and are sent to the provider. Redact before uploading rather than after.

Sources2 referenced
  1. Claude developer documentation

    Anthropic · official product documentation · link checked 2026-09-06

  2. OpenAI platform documentation

    OpenAI · official product documentation · link checked 2026-09-06

Links are checked at build time. If one has moved, tell us.

Written by
BLACK LDN
Last checked
2026-09-06
Content state
editorial

Product behaviour changes. If something here no longer matches the documentation, send a correction.

Read next

How these tools work9 min

What generative AI is — and is not

In ten minutes: what these systems are actually doing when they answer you, and what that makes them good and bad at.

beginner

Checked 2026-09-06

Giving it the right information9 min

What context engineering actually means

Prompting is what you ask. Context engineering is what you put in the room before you ask it. The second is where most of the quality lives.

intermediate

Checked 2026-09-06

Next: Why AI can sound certain and be wrong