AI How language models work, from tokens to reasoning
Model size explained: do more parameters mean better answers?
What parameters and model sizes actually mean, why a smaller newer model often beats a bigger older one, and how to read the numbers in model names.
The short answer
- A parameter is one learned number inside the model, and the B in a name like 8B or 70B means billions of them.
- More parameters generally buy broader knowledge and deeper reasoning, but only if the training data and method keep pace.
- A well trained small model released this year routinely beats a much larger model from two years ago, because training quality improved faster than size did.
- Size is also a cost, a speed and a hardware decision: roughly 2 GB of memory per billion parameters at common compression levels.
- For most everyday tasks the smallest model that passes your own test is the right choice, and the giant only earns its cost on the hard ones.
Parameters are the numbers a model learned during training, and the count is the headline figure in names like Llama 3 8B or Mistral 7B. More of them does tend to mean a more capable model, but the relationship is much weaker than the marketing implies, and it has been getting weaker every year. A carefully trained recent model with eight billion parameters will beat a hastily trained model with ten times as many, and often beats the celebrated giants of a couple of years ago. Size is best understood as one input among several, and as a direct statement about cost, speed and what hardware you need.
What a parameter actually is
During training, the model is shown text and asked to predict what comes next. Every miss adjusts its internal numbers a little. Those numbers, the weights and biases scattered through the network's layers, are the parameters. They encode everything the model has absorbed: that "Paris" often follows "the capital of France is", that code inside a function is indented, that a polite refusal has a particular shape.
Parameters are not facts stored in slots. There is no row for the boiling point of water. Knowledge is smeared across billions of numbers in a way nobody can fully read, which is why you cannot edit a fact out of a model the way you would edit a database.
What extra parameters buy is capacity: more room to store patterns, and more layers of computation to apply them. In practice this shows up as knowledge of obscure topics, better handling of long chains of reasoning, and steadier behavior on unusual requests.
Decoding the numbers in a model name
| Name element | What it means |
|---|---|
| 7B, 8B, 13B | Small. Runs on a good laptop. Fast and cheap. Fine for summarizing, extraction, classification and drafting |
| 70B | Mid size. Needs a strong desktop with a lot of memory, or a cloud service. Noticeably better at reasoning |
| 405B and up | Large. Data center only. Broadest knowledge, slowest, most expensive |
| A3B, 22B active | A mixture of experts design: total parameters are large but only a fraction are used per token, so it runs faster than its total suggests |
| Instruct, chat, it | Tuned to follow instructions in a conversation rather than just continue text |
| Base | Not conversation tuned. For developers doing their own fine tuning |
| Q4, Q8, GGUF | Compression formats for running locally, not sizes. Q4 roughly halves memory use against Q8 with a small quality cost |
| Mini, small, pro, flash | Vendor price tiers on closed models, with no published parameter count |
Why newer and smaller often wins
Three things changed, and all of them favor the small model.
Training data volume and quality. Early large models were undertrained: they had huge capacity and not nearly enough text passed through them to fill it. Modern practice is to push far more tokens through a smaller model, and to filter that text much more aggressively for quality. A model trained on carefully curated text learns more per parameter than one trained on scraped noise.
Distillation. A large, expensive model can be used as a teacher: it generates high quality answers and explanations, and a small model is trained to reproduce them. The small model never has the teacher's full breadth, but it inherits a lot of its behavior on common tasks at a fraction of the running cost. Most of the impressively capable small models you can download are distilled in some way.
Better post training. The instruction tuning and feedback stages described in how an AI model is trained have improved sharply, and they lift small models more visibly than large ones, because a small model's weakness is often behavior rather than raw capacity.
The trade offs you will actually feel
Speed. Smaller models generate more tokens per second, and the gap is large. For anything interactive, or anything you run thousands of times, that difference dominates the experience.
Cost. On metered services, the price per token rises steeply with capability tier. A task that costs a few dollars a month on a small model can cost a few hundred on the largest, for output that may be no better for your particular job.
Hardware. Memory is the binding constraint for running a model yourself. Estimate about 2 GB per billion parameters at 4 bit compression, plus headroom for the context. That is 16 to 20 GB for an 8B model, and well over 100 GB for a 70B model at higher precision. Details of doing this on your own machine are in running AI locally on your device, and the licensing side in open weight AI models.
Reliability on hard problems. This is where large models still clearly earn their keep: unusual domains, long multi step reasoning, tasks where the model must notice that the question contains a false premise. If your work involves those, the cheap option is a false economy.
How to pick without guessing
- Write down five to ten real tasks you need done, using your own documents and your own wording.
- Run them on the cheapest or smallest model available to you. Save the outputs.
- Run the same tasks on a mid tier model and a top tier one.
- Grade the outputs yourself against what you would accept from a competent colleague. Ignore which model produced which until after you have graded.
- Pick the smallest model that passes. Reserve the expensive one for the specific tasks where it was clearly better.
Most people who do this honestly find that the small model handles the large majority of their work and the expensive model earns its cost on a narrow slice, often the same slice that benefits from reasoning modes.
What to check first
Before you upgrade to a larger model because answers disappoint you, check whether the problem is size at all. Missing context, an unclear instruction or a task that needs a calculator rather than a language model will not be fixed by more parameters, and you will pay for the disappointment at a higher rate. Fix the prompt and the input first. Then, if the failures are genuinely about depth of knowledge or long chains of reasoning, move up a tier.
Common questions
What does the B in 7B or 70B mean?
Billions of parameters. A 7B model has about seven billion learned numbers inside it, a 70B model about seventy billion. Some names add other details such as the word instruct for a chat tuned version, or a number of active parameters for mixture of experts designs.
Do closed models publish their parameter counts?
Usually not. Most commercial providers stopped disclosing sizes years ago, so names like mini, small or pro tell you about intended cost and speed rather than an actual count. Treat them as price tiers.
How much memory do I need to run a model at home?
A useful estimate is 2 GB of memory per billion parameters when the model is compressed to about 4 bits, plus a gigabyte or two of headroom. That puts an 8B model within reach of a laptop with 16 GB, while a 70B model needs a serious desktop.
Is a bigger model always more accurate?
No. Bigger models tend to know more obscure facts and handle harder reasoning, but they are not immune to making things up, and they can be worse at following a simple format instruction than a small model tuned for exactly that job.