1. Home
  2. Blog
  3. Artificial Intelligence

Artificial Intelligence

LLM Context Window Explained: Tokens, Limits and How to Fit More, Measured

An LLM context window is the total number of tokens a model can read and write in one request. Learn what fills it, how words turn into tokens, how to budget input and output, and how to trim chat history safely, with counts we measured.

9 min read
LLM context window explained: tokens, limits and how to fit more, measured

Key takeaways

  • The LLM context window is a token limit shared by everything: system prompt, history, documents and the answer.
  • Tokens are not words. Our English prose used 1.14 tokens per word, Python code 2.22, and Hindi text 1.41 with one tokenizer but about 4.6 with an older one.
  • Chat history grows every turn. Trim the oldest messages but always keep the system prompt; naive cutting dropped it in our test.
  • A large window is not a free pass. Long prompts cost more, run slower, and can bury the detail that matters.

You paste a long contract into a chatbot and it answers as if the last ten pages never existed. Or a support bot slowly “forgets” the rules you gave it. Both come down to the LLM context window: the fixed number of tokens a large language model can work with in one request. Knowing how it fills up helps you build AI features that stay accurate and affordable.

We measured the numbers below with OpenAI’s open-source tiktoken library, version 0.14.0, using its o200k_base and older cl100k_base encodings on Python 3.13. Other model families use other tokenizers, so treat our counts as realistic examples, not exact figures for every model. For the basics of tokens, see our LLM tokens guide.

What is a context window?

A model does not remember earlier requests on its own. Each time you call it, you send text, and the model reads all of it before generating a reply. The context window is the maximum size of that combined text, measured in tokens. If the window is 128,000 tokens, the input and the output together must fit inside it. Our guide to what a large language model is explains why models work this way.

Window sizes range from a few thousand tokens on small models to hundreds of thousands, or more than a million, on large ones. Sizes change with each release, so check your provider’s documentation for the exact limit and for any separate cap on output tokens.

What fills the context window

  • System prompt: instructions, rules and persona.
  • Conversation history: every earlier user and assistant message you resend.
  • Retrieved or pasted content: documents, search results, code files.
  • Tool definitions and results: function schemas and the data they return.
  • The answer itself: output tokens come out of the same budget.

That last point surprises many people. A prompt that uses almost the whole window leaves no room for a reply, so the answer is cut short.

Words vs tokens: what we measured

We repeated three short texts 20 times each and counted words and tokens:

Text Words o200k tokens Tokens per word cl100k tokens
English prose 740 841 1.14 841
Python code 180 400 2.22 400
Hindi text 300 422 1.41 1,381

Three lessons stand out. Plain English is close to one token per word. Code uses roughly twice as many tokens per “word”, because symbols, indentation and short names each take tokens. And language matters hugely: the same Hindi text took 422 tokens with the newer tokenizer and 1,381 with the older one, more than three times as many. A window that holds a long English document may hold far less in another language.

import tiktoken
enc = tiktoken.get_encoding("o200k_base")
print(len(enc.encode("Your text here")))

Tip: never estimate tokens by dividing characters by four for production limits. Count with the tokenizer for your model, then keep a safety margin.

Budgeting a request

We planned a request with a 600-token system prompt, 4,200 tokens of chat history and 25,000 tokens of documents, then checked it against three window sizes, reserving 2,000 tokens for the answer:

window   8000: input 29800 tokens, left -21800, room for 2,000-token answer: False
window  32000: input 29800 tokens, left   2200, room for 2,000-token answer: True
window 128000: input 29800 tokens, left  98200, room for 2,000-token answer: True

The 8K window could not take the request at all. The 32K window fit, but with only 2,200 tokens spare, so one more document would squeeze the answer. Write this check into your code before every call: count the parts, subtract a reserved output budget, and decide what to cut if it does not fit.

What happens when you go over

Behaviour depends on the API and app. Some APIs reject the request with an error. Some chat apps silently drop the oldest messages. Some truncate documents. And if the input fits but leaves too little room, the answer stops mid-sentence when it hits the output limit. None of these are obvious to users, which is why the model can seem to “forget” or ignore part of a document.

Chat history grows every turn

Chat apps resend the whole conversation each turn so the model can see it. At about 350 tokens per turn:

chat of 10 turns x 350 tokens = 3500 tokens
chat of 50 turns x 350 tokens = 17500 tokens
chat of 200 turns x 350 tokens = 70000 tokens

A long support chat can quietly reach tens of thousands of tokens, and every new message pays for all the old ones again.

Trimming history without losing the rules

We built a 31-message chat of 1,455 tokens and trimmed it to an 800-token limit in two ways. The smart way kept the system prompt, then added messages from newest to oldest until the budget was full. The naive way just kept the last 800 tokens of the joined text.

history: 31 messages, 1455 tokens -> kept 17 (783 tokens), system kept: True, oldest kept: 'message 14'
naive tail-truncation keeps system prompt: False

The naive version dropped the system prompt, which held the instruction “Always answer in English”. That is exactly how a bot loses its rules in long chats. Better strategies, roughly in order of effort:

  • Always pin the system prompt and the latest user message.
  • Drop or summarise the oldest turns first.
  • Replace old turns with a short running summary written by the model.
  • Store facts outside the chat and retrieve only what the current question needs.

Is a bigger context window always better?

No. A large window lets you include more, but research such as the 2023 “Lost in the Middle” study by Liu and colleagues found that models used information at the start and end of long inputs more reliably than information in the middle. We did not test that ourselves. In practice, sending only the relevant parts often beats sending everything, and it reduces hallucinations caused by distracting or conflicting text. Put key instructions near the start and restate the actual question at the end.

Chunking and retrieval (RAG)

Retrieval-augmented generation keeps documents outside the prompt. You split them into chunks, find the chunks related to a question, and send only those. We split a 4,201-token document into chunks of at most 500 tokens with a 50-token overlap and got 10 chunks. The overlap keeps sentences that cross a boundary readable in at least one chunk.

def chunks(text, size, overlap):
    ids = enc.encode(text)
    step = size - overlap
    return [enc.decode(ids[i:i + size]) for i in range(0, max(1, len(ids) - overlap), step)]

Chunks are matched to questions using embeddings stored in a vector index. Our what is RAG guide walks through the full pipeline.

Cost and speed

Most APIs charge per input and output token, and longer prompts take longer to process. A 100,000-token prompt sent 1,000 times a day is 100,000,000 input tokens daily, even if the answers are short. Trimming history, retrieving instead of pasting, and caching repeated prompt prefixes where the provider supports it can cut cost and latency sharply. Check current prices on your provider’s pricing page, as they change often.

Testing context limits in your app

  • Test with the longest realistic input, not a short sample.
  • Test long chats: does the bot still follow the system prompt after 50 turns?
  • Test non-English input, which may use far more tokens.
  • Assert that your code reserves output tokens and handles over-limit errors gracefully.
  • Check that trimming never removes safety rules, which also matters for prompt injection defences.

Final thoughts

The LLM context window is a shared token budget, not a memory. Count tokens with the right tokenizer, reserve space for the answer, pin your system prompt when trimming history, and retrieve relevant chunks rather than pasting everything. Do that and even a modest window goes a long way, while a huge one stays fast and affordable. Clear prompts help too, as our prompt engineering guide shows.

Frequently asked questions

What is a context window in an LLM?

It is the maximum number of tokens a model can handle in one request, covering the system prompt, conversation history, added documents and the generated answer together.

How many words fit in a context window?

It depends on the text and tokenizer. Our English prose used about 1.14 tokens per word, so 100,000 tokens is roughly 88,000 words, but code and many other languages use more tokens per word.

What happens when you exceed the context window?

The API may reject the request, the app may drop older messages or truncate documents, or the answer may be cut off because no output tokens are left.

Does the output count toward the context window?

Yes. Input and output share the same window, so always reserve enough tokens for the reply. Many APIs also have a separate maximum for output tokens.

Is a bigger context window better than RAG?

Not always. Large windows cost more and models can miss details in the middle of long inputs. Retrieving only relevant chunks is often cheaper and more accurate.

Keep learning

Work with Stack Quora

Need help with testing, WordPress or SEO?

We help teams ship better software and grow their sites. Tell us what you are working on and we will suggest a clear next step.

Keep reading

More guides on testing, AI & modern development

Enjoyed this article? Pick a topic and jump straight into our newest tutorials.

Artificial Intelligence

BM25 Explained: How Keyword Search Ranks Results, With Python Code and Hybrid Search

BM25 explained with a pure Python implementation we ran: the formula, IDF, term saturation, the k1 and b parameters, why…

Read article →
Software Development

HTTP Methods Explained: GET, POST, PUT, PATCH, DELETE, Tested on a Live Server

HTTP methods explained with a small server we ran: GET, POST, PUT, PATCH, DELETE, HEAD and OPTIONS, safe vs idempotent,…

Read article →
DevOps

Git Bisect Explained: Find the Commit That Broke Your Code, Manually or Automatically

Git bisect explained with a tested repository: find the one bad commit among 41 in 5 test runs, automate it…

Read article →
Test Automation

pytest Markers Explained: Custom Marks, skip, skipif, xfail and -m Filters

pytest markers explained with a tested suite: register custom marks, run only smoke tests with -m, combine filters with and,…

Read article →
Software Testing

Positive vs Negative Testing: Differences, 24 Real Test Cases and a Checklist

Positive vs negative testing explained with a real signup form: 6 positive and 18 negative test cases run in pytest,…

Read article →
Databases & SQL

SQL ORDER BY Multiple Columns: ASC, DESC, NULLS LAST, Case and Custom Order

SQL ORDER BY with multiple columns explained with tested queries: mixed ASC and DESC, tie-breakers, NULLS FIRST and LAST, case-insensitive…

Read article →
Databases & SQL

SQL Running Total: SUM OVER, Per-Group Totals, Moving Sums and the RANGE Trap

How to calculate a SQL running total with SUM() OVER: per-group totals, daily cumulative sums, moving sums, percent of total,…

Read article →
Software Development

JavaScript Sort Array: Numbers, Objects, Strings and Why 10 Comes Before 5

How to sort an array in JavaScript correctly: numbers with a compare function, objects by one or more fields, strings…

Read article →
Software Development

Python String Methods: Cheat Sheet, Real Examples and the strip() Trap

Python string methods with tested examples: split, join, strip, replace, find, startswith, is-methods, casefold, partition and translate, plus the strip()…

Read article →
Software Development

Python zip() Explained: Loop Two Lists at Once, Unzip, strict=True and Traps

Python zip() pairs items from several lists in one loop. Tested examples for dicts, unzipping with *, strict=True, zip_longest, sliding…

Read article →
Artificial Intelligence

Cosine Similarity Explained: Formula, Worked Example and Python Code

Cosine similarity explained simply: the formula, a worked example by hand, pure Python and NumPy code, cosine vs Euclidean distance…

Read article →
Software Development

UTF-8 Encoding Explained: How Text Becomes Bytes and How to Fix Garbled Text

UTF-8 encoding explained with byte-level examples we ran in Python: how characters become 1 to 4 bytes, why café turns…

Read article →
DevOps

Git Squash Commits: 4 Easy Ways, Tested Step by Step (rebase -i, reset, merge)

Git squash commits explained with 4 tested methods: interactive rebase, reset –soft, merge –squash and fixup with autosquash. Before-and-after logs,…

Read article →
Test Automation

pytest parametrize Explained: 7 Patterns with Tested Examples

pytest parametrize explained with 7 tested patterns: basic cases, readable ids, stacked decorators, pytest.param with xfail and skip, indirect fixtures…

Read article →
Software Testing

Retesting vs Regression Testing: 8 Key Differences with a Real Bug Fix Example

Retesting vs regression testing explained with a real bug fix run in pytest: retesting confirmed the fix, the regression run…

Read article →
Databases & SQL

Delete Duplicate Rows in SQL: 5 Safe Methods, Tested Step by Step

How to delete duplicate rows in SQL safely: find them, preview them, then remove them with ROW_NUMBER, MIN(id), EXISTS, rowid…

Read article →
Databases & SQL

SQL RANK vs DENSE_RANK vs ROW_NUMBER: Differences with Tested Examples

SQL RANK vs DENSE_RANK vs ROW_NUMBER in one table: how each handles ties, top-N per group, the second-highest salary trap…

Read article →
Software Development

JavaScript Optional Chaining (?.) Explained: 8 Examples, ?? Defaults and Traps

JavaScript optional chaining stops “Cannot read properties of undefined” errors. 8 tested examples for properties, arrays and methods, ?? vs…

Read article →
Artificial Intelligence

RAG Chunking Strategies Explained: Fixed, Overlap, Sentence, Recursive and Headings, Measured

RAG chunking strategies compared on one document with tiktoken 0.14: fixed-size, overlap, sentence packing, recursive splitting and heading-based chunks, measuring…

Read article →
DevOps

Git Stash Explained: Save, Apply, Pop and Untracked Files, Tested

Git stash explained with commands we ran on Git 2.56: push with a message, untracked files, list and show, apply…

Read article →
Software Testing

Pairwise Testing Explained: How All-Pairs Cuts 216 Tests to 13, Tested

Pairwise testing explained with a generator we ran in Python 3.13: a checkout with 216 combinations covered by 13 tests,…

Read article →
Test Automation

Test Doubles Explained: Dummy, Stub, Spy, Mock and Fake, with Tested Python Code

Test doubles explained with one Python example run in pytest 9.2: dummy, stub, spy, mock and fake, when to use…

Read article →
Databases & SQL

SQL COALESCE Explained: Defaults, NULLIF, IFNULL vs ISNULL and Traps, Tested

SQL COALESCE explained with queries we ran in SQLite 3.50: first non-null values, empty strings, NULL in maths, NULLIF for…

Read article →
Databases & SQL

SQL Self Join Explained: Employee-Manager, Pairs and Row Comparisons, Tested

SQL self join explained with queries we ran in SQLite 3.50: employee and manager, earning more than the manager, unique…

Read article →
Artificial Intelligence

LLM Context Window Explained: Tokens, Limits and How to Fit More, Measured

The LLM context window explained with token counts we measured using tiktoken: words vs tokens for prose, code and Hindi,…

Read article →
DevOps

Git Cherry-Pick Explained: Copy the Commit You Need, Safely, Tested

Git cherry-pick explained with commands we ran in Git 2.56: copying a hotfix to a release branch, new hashes and…

Read article →
Test Automation

Snapshot Testing Explained: How It Works, When It Helps and When It Hurts

Snapshot testing explained with a pytest and syrupy demo we ran: creating snapshots, reading the diff after a code change,…

Read article →
Software Testing

Mutation Testing Explained: Why 100% Coverage Still Missed Our Bug

Mutation testing explained with a small mutation tool we built and ran: 14 mutants, a test suite with 100% line…

Read article →
Artificial Intelligence

Prompt Injection Explained: How AI Apps Get Tricked and How to Defend Them

Prompt injection explained for builders: direct and indirect attacks, why keyword filters fail (our test blocked 1 of 4), and…

Read article →
DevOps

DNS Records Explained: A, AAAA, CNAME, MX, TXT and TTL With Real Lookups

DNS records explained with lookups we ran: how resolution works, A and AAAA, CNAME rules, MX and null MX, TXT…

Read article →
DevOps

Feature Flags Explained: Safe Releases, Gradual Rollouts and Flag Hygiene

Feature flags explained with a rollout simulation we ran: flag types, percentage rollouts with stable hashing, kill switches, allow lists,…

Read article →
Test Automation

Test Data Generation With Python Faker: Realistic, Repeatable and Safe Data

Test data generation with Python Faker, run and measured: seeding for repeatable data, locales, unique values, domain-shaped records, edge cases…

Read article →
Software Testing

Shift-Left Testing Explained: Find Bugs Earlier Without Slowing the Team

Shift-left testing explained: what it means, why early defects are cheaper, practices from requirement reviews to CI checks, the tester's…

Read article →
Artificial Intelligence

LLM Structured Output Explained: Getting Reliable JSON From AI Models

LLM structured output explained with validation we ran in Pydantic: JSON mode and schemas, constrained decoding, the ways model output…

Read article →
Test Automation

API Contract Testing Explained: Catch Breaking Changes Before They Ship

API contract testing explained with a schema check we ran: what a contract is, consumer-driven contracts, Pact, OpenAPI, which changes…

Read article →
Software Testing

Risk-Based Testing Explained: Score Risks, Build a Matrix and Test What Matters

Risk-based testing explained step by step: identifying risks, scoring likelihood and impact, building a risk matrix, mapping risk to test…

Read article →