AI AI basics: what it is and how it got here
Neural networks explained: what is actually inside a model
What a neural network really is: layers, weights, training by error correction, and why nobody can point to the part of the model that produced a given answer.
The short answer
- A neural network is a stack of numbers called weights plus a fixed rule for pushing data through them, and those numbers are the only thing a trained model contains.
- Training is error correction at scale: the system guesses, the guess is scored against a known answer, and every weight gets nudged a fraction in the direction that would have scored better.
- Depth matters because each layer builds on features the layer below assembled, going from edges to parts to objects in vision, and from spelling to long range meaning in text.
- No single weight or neuron holds a concept, so there is no component you can point to and no per answer explanation to extract, and that is a property of the design rather than a secret.
- Because the machinery cannot be read, auditing an AI decision means testing behavior: known answer test sets, counterfactual inputs, logged outputs and a human who can override.
A neural network is a very large pile of numbers plus a fixed rule for pushing other numbers through them. Open a trained model and you will not find sentences, facts or if statements. You will find weights: plain decimals, a few million of them in a small model and hundreds of billions in a large one. Everything the system does, answering a question, flagging a transaction, transcribing speech, comes out of arithmetic on those numbers and nothing else.
That sounds like a dodge. It is the answer, and it explains the two strangest things about AI: why a model handles cases nobody anticipated, and why nobody can show you the part that produced a given answer.
A wall of dials nobody set by hand
Picture a machine with input sockets on one side, an output display on the other, and a hundred million dials across the front. Every dial nudges the output a little. At the start each one is set at random and the output is noise.
Training is a loop. Show the machine one example where the right answer is known. Measure how far off the output was, as a single number. Then work out, for every dial, which way you would turn it to make that number slightly smaller, and turn it a tiny fraction of the way. Move to the next example. Repeat billions of times.
Nobody decides what a dial means. There is no dial for grammar and no dial for the capital of France. The dials end up wherever reduced error across the whole pile of examples, and that is the entire content of what the model learned. The same loop sits under every learned system, and how software learns from examples covers it without the network specifics.
Everything becomes numbers first
A network cannot accept a word or a photo. Both are converted first.
Text is chopped into tokens, chunks of roughly a word or part of a word, and each token is looked up in a table that turns it into a list of numbers, often a few hundred to a few thousand of them. Those coordinates are learned during training, which is why tokens used in similar ways end up close together. Why text is split into tokens rather than words covers the chunking.
An image arrives as numbers already, a brightness value per pixel per color channel, and audio as amplitude measurements taken tens of thousands of times a second. The output is numbers too: the last layer of a language model produces one score per token in its vocabulary, usually tens of thousands of them, converted into probabilities. Something picks one and the whole thing runs again for the next token, a cycle traced in the loop that writes one token at a time.
What one layer does, and what depth buys you
A layer does two things. It multiplies the incoming list of numbers by a grid of weights and adds an offset, which is a great deal of multiplying and adding and nothing cleverer. Then it passes each result through a simple bent function, the commonest of which leaves positive numbers alone and flattens negative ones to zero.
That second step looks trivial and it is the reason depth works at all. Without the bend, a chain of plain multiplications collapses into one, so a hundred layers could be replaced by a single layer doing the same job. The bend lets each layer express something the layer below could not.
Depth buys composition. In an image network the earliest layers respond to edges, the middle ones to textures and parts such as a wheel or an eye, the later ones to whole objects. Language models show a similar gradient: early layers handle spelling and local grammar, later ones carry longer range relationships such as which earlier noun a pronoun points back to.
| Layer type | What it does in plain terms | Where you meet it |
|---|---|---|
| Embedding | Turns a token or a category into a list of numbers the rest of the network can work with | First layer of any text model |
| Fully connected | Every input touches every output: general purpose mixing, and the most weights per layer | Small models, and the final scoring layer of large ones |
| Convolutional | Slides a small window across an image hunting for the same local pattern everywhere | Image recognition, medical imaging, some audio |
| Recurrent | Walks a sequence one step at a time, carrying a running summary | Older speech and translation models, time series |
| Attention | Lets every position look at every other position and weigh which ones matter to it | The core of transformer language models |
| Normalization | Rescales the numbers between layers so training neither explodes nor stalls | Inside almost every modern network, invisible from outside |
| Softmax output | Turns raw scores into probabilities that add up to one | Last layer of a classifier or a language model |
How an error signal turns into adjustments
Training needs one number saying how wrong the output was. That is the loss. For a language model it usually measures how much probability the model put on the token that actually came next: high probability, low loss.
Backpropagation spreads the blame. It starts at the output, where the error is known, and works backwards layer by layer, using calculus to work out how much each weight contributed and in which direction. Out comes one number per weight saying which way to nudge it and how much it mattered.
The nudge is deliberately small. Its multiplier is the learning rate: too big and the weights oscillate forever, too small and training drags. Updates are averaged over batches rather than applied one example at a time.
Run that over a large enough corpus and the weights settle somewhere that predicts well on average. Prediction alone will not answer politely or decline a bad request, which is why the full pipeline from raw text to chatbot has several stages after this one.
Why nobody can point at the part that answered you
Ask which weights produced a particular sentence and there is no useful answer. Three things compound.
The representation is distributed. A sense of formal register, or the idea of a legal document, is not kept in one place. It is smeared across thousands of weights in many layers, and each of those weights also takes part in a hundred unrelated things. Zero out one and usually nothing measurable changes.
There are also more concepts than components to hold them, so networks pack unrelated features into overlapping sets of neurons and one unit fires for things with nothing in common. And there is scale: a mid sized model holds more weights than you could inspect in a lifetime, with every answer depending on all of them at once. What a parameter count actually tells you gives a sense of how fast those numbers grow.
Interpretability research has made real progress here, with probes that test whether a piece of information is present at a given depth and experiments that switch parts of the computation off to see what breaks. These find genuine structure. What they do not yet produce is a per answer receipt, which is exactly what an auditor or a rejected applicant wants. Asking the model to explain itself is no substitute: that generates more text, written the same way the answer was rather than read off the computation.
What to do when a decision has to be justified
If you cannot explain the machinery, test the behavior instead. That is more achievable than it sounds.
- Test outcomes, not internals. Run the system over cases where the answer is known, then split results by the groups you care about. A gap in error rates shows up in outputs even when its cause is invisible in the weights, and where AI bias comes from describes the patterns to look for.
- Test counterfactuals. Change one field, leave the rest identical, and see whether the decision flips. It is the cheapest way to catch a model leaning on something it should not be.
- Log inputs, outputs, model version and settings. You cannot rebuild a decision from the weights later, so that record is your only audit trail.
- Keep a person at the decision point wherever money, health, employment or legal consequences attach, and make overriding the model easy.
- Use a simpler model when an explanation is genuinely required. A regression or a decision tree gives up some accuracy and hands you reasons you can state in a sentence. Often that is the better trade.
Treat attribution tools that report which inputs mattered most as debugging aids rather than evidence. They approximate the model rather than report it, and different methods disagree about the same case.
Rules differ by region. In the EU, the AI Act attaches documentation, logging and human oversight duties to systems classed as high risk, which covers much of hiring, credit and education. The US has no single equivalent, so duties come from sector rules: a credit denial has long had to state the specific principal reasons behind it, network or no network. This is general information rather than advice about a particular situation.
What to do next
When someone calls a model a black box, hear a claim about structure rather than secrecy. Weights are often published in full and it changes very little, because a hundred billion unlabeled numbers tell a person nothing. So the useful questions about any AI system are what it learned from, how it behaves on cases you can check, and who is accountable when it is wrong. If a term here was new, the AI jargon glossary defines the rest in a sentence each, and the plain definition of artificial intelligence puts the network in its wider frame.
Common questions
Is a neural network like a human brain?
Only loosely, and the name oversells it. The design borrowed one idea from biology, that simple units connected in bulk can do complicated things, and stopped there. Real neurons fire in time, rewire themselves, and run on chemistry. An artificial one adds up numbers and applies a simple function. Treat it as an engineering metaphor rather than a model of thinking.
How many layers does a typical model have?
It varies enormously by task. Small image classifiers run to a few dozen layers, and large language models typically stack many dozens of transformer blocks, each containing several internal layers. More depth is not automatically better: past a point extra layers add cost and training difficulty without adding accuracy, which is why architecture choices are tested rather than guessed.
Can you see what a model learned by looking at the weights?
Not by reading them. Published weights are just long lists of decimals with no labels attached. Researchers extract partial insight using tools that look at how the network responds to inputs rather than at the raw numbers, and that work identifies some recognizable features. It falls a long way short of a full account of any specific output.
Does the model keep learning from the people who use it?
Not automatically. The weights are fixed once training ends, so a conversation changes nothing inside the model. Providers may collect conversations and use them later when training a future version, which is a separate process with its own settings you can usually turn off. Within a session, any apparent learning comes from earlier messages still being visible.
Do all AI systems use neural networks?
No. Plenty of production machine learning runs on simpler methods such as gradient boosted trees or regression, which often beat neural networks on ordinary table shaped data like transactions and customer records. Networks dominate where the input is raw and unstructured: pixels, waveforms and text. Picking the simpler method when it works is normal engineering, not a compromise.