AI AI images, video and voice: how they are made
How AI image generators turn noise into a picture
Diffusion explained in plain English: how a model starts from random noise and removes it step by step, guided by your words, to produce a finished image.
The short answer
- An image generator does not draw a picture, it starts with random static and removes the noise in stages until an image is left behind.
- The model was trained by adding noise to captioned images and learning to predict exactly what noise was added, so generation is that same skill run backwards.
- Your prompt does not describe the output, it steers every denoising step, which is why small wording changes can restructure the whole picture.
- Hands, text, reflections and jewelry go wrong because the model enforces local plausibility without any underlying model of anatomy, language or physics.
- The model file holds a few billion learned numbers, not a library of training pictures, though heavily duplicated images can still be reproduced closely.
An AI image generator does not draw. It starts with a square of random static, then removes the noise a little at a time over a few dozen passes, until a picture is all that remains. Your words steer which picture emerges. That one idea, generation as guided noise removal, explains nearly everything you notice about these tools: why the same prompt gives a different result each time, why hands and lettering come out wrong, why the tool takes several seconds rather than being instant, and why it can produce a convincing photograph of something that was never photographed.
Training is learning to undo damage
The technique is called diffusion, and it is taught backwards. Take a captioned photograph. Add a small amount of random noise, then a little more, then more again, over hundreds of stages, until the image is indistinguishable from television static. Every stage is recorded, so the system knows precisely how much noise was added and where.
Now train a neural network on one narrow task: given a noisy image and a note saying how far along the noising process it is, predict the noise. Not the picture, the noise. Do that across hundreds of millions of captioned images and the network gradually learns what has to be true of a photograph for a particular smudge to be noise rather than content. It learns that skin has certain textures, that shadows fall consistently, that the word "bicycle" in a caption goes with two circles and a frame.
The caption is fed in through a separate text encoder, a model that turns words into lists of numbers representing meaning. The denoising network sees those numbers alongside the noisy image, so its guess about what is noise is conditioned on what the picture is supposed to be. That conditioning is the entire mechanism by which your prompt has any effect.
Most modern systems do not work on raw pixels. A compression network first shrinks an image to a small numerical grid, so a 1024 by 1024 picture becomes something like a 128 by 128 grid of feature values. Denoising happens in that compressed space, which is dozens of times cheaper to compute, and a decoder expands the finished result back into pixels. This is why the fine grain of an AI image sometimes looks slightly soft or waxy: it was reconstructed, not recorded.
What actually happens when you press generate
- Your prompt goes through the text encoder and becomes a block of numbers. Most tools cap this at a few hundred words worth of input, after which extra text is ignored or diluted.
- A seed, which is just a starting random number, fills the working canvas with static. Change the seed and you change the image even if nothing else moves.
- The model predicts the noise in that canvas twice: once with your prompt and once ignoring it. The gap between the two predictions is the part of the picture your words are responsible for.
- That gap is amplified by a guidance setting, commonly somewhere between 3 and 10. Low values drift away from the prompt, high values produce oversaturated, brittle images that obey the words too literally.
- A scheduler subtracts part of the predicted noise and hands the slightly cleaner canvas back for another pass. Typical runs use 20 to 50 passes.
- The decoder expands the result to full resolution. On data center hardware the whole sequence takes a few seconds; on a home graphics card, usually tens of seconds.
Why the artifacts look the way they do
The model checks that every small region is plausible. Nothing checks that the whole thing is possible.
| What you see | Why it happens |
|---|---|
| Six fingers, fused knuckles | Hands occupy few pixels, adopt endless valid shapes, and every local patch of finger looks correct on its own |
| Garbled signs and labels | Letters are treated as texture, not language, so the model reproduces the look of writing rather than words |
| Reflections that do not match | There is no 3D scene and no light simulation, only learned correlations between surfaces and their surroundings |
| Chains and straps that merge or vanish | Thin objects pass behind other objects, and the two ends are denoised far apart on the canvas |
| Mismatched earrings or glasses arms | Same problem: left and right are distant regions with no shared record of what was decided |
| Glossy, evenly lit skin | Models are tuned on human preference ratings, and raters reward flattering images |
Knowing the mechanism is also the basis of spotting an AI generated image, because the failures cluster in predictable places rather than appearing at random.
What the model stores, and why that matters
A released image model is typically a file of 2 to 12 gigabytes holding a few billion learned numbers. It was trained on hundreds of millions, sometimes billions, of images. The arithmetic is worth doing: that leaves far less than one byte per training picture, so the file cannot be an archive of what went in. What it holds is statistical structure, the shapes and textures and co-occurrences that recur across the whole collection.
The important exception is duplication. When the same image appears thousands of times in a training set, a famous album cover or a stock photo reused across the web, models have been shown to reproduce it closely from a matching prompt. So "it never memorizes anything" is too strong. Where the training pictures came from, and whether collecting them was lawful, is a separate argument that is still unresolved in most countries. The background is in where AI training data comes from, and the practical consequences for your own output are covered in using AI images legally.
Image to image, inpainting and outpainting
These are the same engine with a different starting point. Instead of beginning from pure static, the tool takes your existing photo and adds noise to it, but stops partway. A strength setting decides how far: at 0.3 the original composition survives almost intact and only the surface changes, at 0.8 barely a trace of it remains. Denoising then proceeds as normal, guided by your prompt.
Inpainting masks a region and noises only that area, while the untouched pixels around it are fed in as context at every step. That is why a replaced sky usually matches the lighting of the buildings beneath it. Outpainting does the same beyond the frame edge, growing the canvas.
A further family of controls conditions generation on structure rather than words: a pose skeleton, a depth map, or a traced edge drawing. The model is then denoising toward an image that both matches your prompt and fits that skeleton, which is how people get consistent poses and camera angles across a set.
What to try first
Generate the same prompt four times and look at what stays constant. Whatever repeats is coming from your words; whatever varies is coming from the seed. Then change one thing at a time: the lighting phrase, the aspect ratio, the guidance value. Raising step counts above roughly 40 rarely improves anything and costs real time. When the picture is nearly right, fix the last part with inpainting instead of starting over, and put the effort into your wording using the structure in writing image prompts.
Common questions
Why do I get a different image every time I use the same prompt?
Each run begins from a different random starting pattern, called the seed. Fix the seed and keep every other setting identical and you will get the same image back, which is how people iterate without losing a composition they liked.
Does the generator search the internet for pictures while it works?
No. Generation happens entirely inside the model, using only the numbers learned during training. Some products add a separate web search step for reference or for facts, but the image itself is produced offline from noise.
Why is written text in AI images so often misspelled?
The model handles images as visual patterns rather than as language, so letters are just shapes that usually appear in certain arrangements. Short words on a large sign now come out right fairly often, while dense paragraphs and small labels still turn into convincing looking gibberish.
Is the image I generate a copy of someone's artwork?
Usually not in any direct sense, because the model stores statistical patterns rather than files. The legal question of whether training on copyrighted images was itself lawful is separate and still being argued in court.