Why AI gets simple math wrong and when to trust it

A chatbot gets arithmetic wrong for the same reason it gets a rare fact wrong: it is producing the most plausible continuation of some text, not running a calculation. Ask it to explain why the quadratic formula works and it will do a genuinely good job, because that explanation exists in countless forms in its training data. Ask it to multiply 4,817 by 639 and it has to produce seven digits in order, left to right, from pattern rather than from carrying. It will often be right. It will sometimes be off by a digit in the middle, and the wrong answer will be formatted just as neatly as the right one.

The exception matters more than the rule: when the assistant writes code and runs it, or calls a calculator tool, the number comes from an actual computation and you can trust it the way you trust a spreadsheet. Most of the practical skill here is learning to tell those two situations apart.

What happens when a model "adds"

A model reads and writes in tokens, which are chunks of text roughly the size of a common word fragment. Numbers get chopped into these chunks by the same general rules as words, and those rules have nothing to do with place value. Depending on the tokenizer, "4817" might become one token, or "48" plus "17", or "4" plus "817". A number with thousands separators splits differently again. The idea is explained more fully in what tokens are.

Now consider what column arithmetic requires. You work right to left, you hold a carry, and you align digits by their position. A model producing text works left to right, has no scratchpad unless it writes one out, and its representation of "4817" may not even isolate the individual digits. It is being asked to do a procedure in exactly the wrong direction with the wrong materials.

Two consequences follow. Short, common calculations are reliable because they appear constantly in the text the model was trained on, so the answer is effectively remembered. Long or unusual ones degrade, and the errors cluster in the middle digits, which is also where a human skimming is least likely to notice.

Where it fails and where it is fine

Percentages deserve a specific warning. Percentage increase, percentage of a total and percentage point difference are three different operations that share a symbol, and models mix them up under pressure. If a figure sounds plausible and the method sentence sounds confident, both can still be wrong, which is a general property of confident answers.

Tool use is the actual fix

Modern assistants can call tools and take actions: a code interpreter that runs Python, a calculator, or a connection to a spreadsheet. When that happens, the model's job changes from producing the answer to producing the instructions for the answer. It writes 4817 * 639, the code runs, and the real result comes back. Writing correct code is something models do well, because code is highly patterned. Multiplying is something computers do perfectly.

You can usually see this happening. The reply shows a code block, or an output panel, or a line saying it ran an analysis. If you only see prose and a bolded figure, nothing was executed.

1. Ask for it explicitly: "Calculate this by writing and running Python, then show me the code and the output."
2. Read the code, not just the result. Check that the input numbers match yours and that the operation is the one you wanted.
3. Ask it to print the intermediate values, not just the final number, so a wrong step is visible.
4. For a repeated calculation, ask for a small script or a spreadsheet formula you can keep and reuse, so the next run does not depend on the chat at all.
5. Sanity check the magnitude yourself. Round everything hard and do the sum in your head: if 4,817 times 639 comes back as roughly 3 million, that is the right neighborhood, and anything near 300,000 is not.

Reasoning modes help, but they do not settle it

Some models have a mode that spends longer working through a problem in steps before answering. This genuinely improves math, because writing out the intermediate steps gives the model something like a scratchpad, and each step is a shorter prediction than the whole answer. Competition style problems that older models failed are now often solved.

It is still prediction. A long chain of careful steps can carry an arithmetic slip from step three all the way to a confidently wrong conclusion, and the visible working makes the result feel more trustworthy rather than less. Treat the steps as a method you can audit, not as proof that the numbers are right. What these modes do and do not change is covered in reasoning models explained.

The quiet failures: spreadsheets and units

Two situations produce errors that survive review because nothing looks wrong.

Spreadsheets: a model asked to analyze pasted rows will often estimate rather than sum, especially over long tables, and it can silently skip a row or read a merged cell incorrectly. If a total matters, ask for the formula and let the spreadsheet compute it. A returned =SUMIFS(...) you can inspect is worth more than a returned total you cannot.

Unit conversions: ratio conversions such as kilometers to miles are usually fine, because the constant is memorized. Conversions with an offset, mixed units such as feet and inches, and anything involving currency at a specific historical rate are where it slips. Currency in particular combines two weaknesses, arithmetic and a rate that changes after the knowledge cutoff.

What to do next

Sort your math into two piles. If you want to understand something, ask freely: explanations, methods, worked examples and checking your own reasoning are all good uses. If you want a number you will act on, make the machine actually compute it, then verify the magnitude in your head and the inputs with your eyes, the same way you would check any other AI answer. That takes about fifteen seconds and removes nearly all of the risk.