What tokens are and why they decide your AI bill

A token is a chunk of text, and it is the only unit an AI model actually deals in. Not words, not characters, not sentences. Everything you type is cut into tokens on the way in, and everything the model produces is built from tokens on the way out. That matters to you for two practical reasons: tokens are what you are charged for, and tokens are what the size limits are measured in. If a document "does not fit" or a bill came in higher than you expected, the explanation is almost always sitting in the token count.

What the chopping actually looks like

Tokenizers are built by scanning enormous amounts of text, the same sort of collection that makes up a model's training data, and giving the most frequent sequences their own entry in a vocabulary of typically 100,000 to 200,000 items. Common words get one token each. Rarer strings get assembled from pieces.

Take this sentence: "The unbelievably scenic village of Llanfairpwllgwyngyll has fiber broadband."

• "The", " village", " of", " has" and " broadband" are each one token, space included.
• " unbelievably" usually splits into something like " un", "bel", "iev", "ably".
• " scenic" might be one token or two.
• " Llanfairpwllgwyngyll" shatters into eight or more fragments.
• " fiber" is one token, but " fibre" may well be two, because the American spelling appears more often in training text.

So a nine word sentence can land anywhere between 12 and 20 tokens depending on what is in it. That variance is the whole story behind the rules of thumb below.

The numbers worth memorizing

Code is expensive because indentation, brackets, underscores and camelCase names all fragment. A variable called calculateMonthlyInterest is not one token, it is four or five.

How to estimate what something will cost

1. Get the word count of what you are sending. A word processor will tell you; for a PDF, an approximate page count times 500 is close enough.
2. Divide by 0.75 to get input tokens. A 4,000 word document is about 5,300 tokens.
3. Estimate the output you want. A 600 word summary is about 800 output tokens.
4. Multiply each by the provider's per token rate, which is normally quoted per million tokens and is higher for output than for input.
5. If this is a back and forth chat rather than one shot, multiply the input side by the number of turns, because the entire conversation is resent each time.

That last step catches people out constantly. On turn one you send 5,000 tokens. On turn ten, if the conversation has grown to 12,000 tokens, you are sending all 12,000 again for a one line follow up question. A ten turn chat over a long document can cost several times what the document alone suggested. The mechanics of that resending are covered in context windows and why an AI forgets, and the pricing side in how AI pricing works.

Doing that selection by hand stops working once you have more than a handful of documents, which is where retrieval comes in: a search step picks the few passages that match your question, and only those go into the prompt. It is the same economy as pasting one section instead of forty pages, just automated, and it is the main reason connecting a model to your own documents costs less than feeding it everything you own on every question.

Why tokens explain the weird failures

Ask a model how many letter r's are in "strawberry" and it may get it wrong. Ask it to reverse a word and it may fumble. This is not a reasoning failure so much as a visibility problem. The model received something like " straw" plus "berry", two opaque IDs. The individual letters were never in front of it. It can often answer correctly because it has seen people discuss spelling in text, but it is recalling discussion of the word rather than inspecting the word.

The same effect shows up in arithmetic with long numbers, in character counts, in rhyming, and in anything that depends on the exact shape of a string. When you need that kind of precision, put the work somewhere else: a spreadsheet, a calculator, or a script the model writes for you and you then run.

Where caching and pricing tiers fit in

Providers usually charge separately for input and output, with output costing more because it is generated one token at a time and takes real compute per token. Many also offer cached input at a large discount: if you send the same long preamble repeatedly, the system can reuse its earlier processing of it. In practice that rewards putting the fixed material (the document, the instructions, the examples) at the start of your prompt and the varying part at the end.

Batch processing, where you accept results within a few hours rather than immediately, is commonly discounted as well. If you are running hundreds of documents through a model and nobody is waiting on the answer, that is usually the cheapest route.

Reasoning models add a line to the same bill. Before writing the answer you see, they produce a stretch of intermediate working, and that working is charged as output whether or not the interface shows it to you. A three line reply can therefore carry a long invisible middle, which is why the same question costs more on a model that thinks before it answers than on a standard one. Send the jobs that need the extra steps there, and leave summarizing and rewriting on the cheaper model.

What to do next

Count the tokens on the task you run most often, then look at where they are actually going. If most of them are a document you resend every turn, split the work into separate short chats. If most of them are conversation history, start fresh sessions more often. And if you are working in a language other than English, budget two to three times what an English cost estimate suggests. If the per token bill is the binding constraint rather than the quality, running a smaller model on your own machine removes that bill entirely, in exchange for speed and setup time. Once you know the shape of your own token usage, the rest of how a language model works becomes much easier to reason about.