How Diffusion Models Generate Images
You'll learn to
- -Explain the forward process as a blend of picture and noise, and compute one noising by hand
- -Show why predicting the noise is the same as predicting the clean picture
- -Describe one reverse denoising step and why many small steps are used
- -Explain classifier-free guidance and the CFG scale slider, with numbers
Every module before this one assumed the goal was generating text. Stable Diffusion, DALL-E and Midjourney generate images by a completely different mechanism, one that has nothing to do with predicting the next token. A diffusion model learns to reverse a process of gradually destroying a picture with noise. Understand that one idea and the whole family makes sense.
An Analogy: Static on an Old TV
Tune an old television between channels and you see static, random noise. Imagine a picture slowly being buried in that static, frame by frame, until nothing of it is left. Now imagine a skilled restorer who has watched this happen thousands of times, and who can look at any half-buried frame and say which part of it is static. Run that skill in reverse, again and again, and you can start from pure static and uncover a picture.
- -The clean picture is x0.
- -The random static is the noise.
- -A frame partly buried in static is the noisy image x_t, where t says how far along the burying we are.
- -The restorer's skill, looking at a noisy frame and saying which part is static, is the model's whole job.
- -Repeating that skill many times, removing a little static each time, is generation.
The Forward Process: Burying the Picture on a Schedule
Training needs pictures at every stage of burial. The forward process is a fixed recipe, with no learning involved. It blends the clean picture with fresh noise. A number called alpha-bar, between 0 and 1, says how much clean signal is left: 1 means none lost yet, 0 means everything is gone.
Take one pixel value x0 = 0.8 and noise = -0.5. With alpha-bar = 0.9 the result is 0.949 x 0.8 + 0.316 x (-0.5) = 0.60, mostly still the picture. At 0.5 it is 0.21, an even mix. At 0.01 it is -0.42: almost pure noise. The last column of the output below is the key insight: whatever the stage, if you know the noise you can recover the original exactly.
Here is the same idea on a whole picture. Slide the signal down to bury a small face in noise, see it come back when the exact noise is removed, and then see what happens when the model's guess is slightly off.
Burying a Picture in Noise, Then Digging It Out
Diffusion destroys a picture with noise on a fixed schedule, then trains a model to guess the noise so it can be taken away again. Slide the signal down to bury the picture, then make the model's noise guess less accurate.
1 means none lost yet, 0 means everything is gone.
One pixel: 0.8 plus noise -0.5 becomes
0.21
matches the chapter at alpha-bar 0.9, 0.5 and 0.01
A wrong guess is multiplied by
1.0x
sqrt(1 - alpha-bar) / sqrt(alpha-bar)
With 50% of the signal left, the picture is badly buried, yet subtracting the exact noise recovers it perfectly. The model's job is only to guess that noise.
The Reverse Process: Learning to Undo It
That recovery trick shows why the model's job is narrow and learnable. Given a noisy picture and the stage t, it only has to predict the noise inside it. Subtract the predicted noise and you have an estimate of the clean picture. In practice the prediction is imperfect, so the sampler removes only a small amount at a time and repeats. Real models take anywhere from a handful to thousands of steps, depending on the sampler.
Run one step with z set to zero. The noisy values [1.1179, 1.5821] come from a clean picture [1, 2]. After one reverse step they become [1.0575, 1.7886]: noticeably closer to [1, 2], but not there yet. Chain enough steps and static becomes an image.
Classifier-Free Guidance: Making the Model Follow the Prompt
A text-to-image model can predict the noise in two ways for the same noisy picture: reading the prompt, or ignoring it. Think of two advisers, one who has heard your request and one who has not. The difference between their advice is exactly what your request contributed. Classifier-free guidance (CFG) takes the prompt-aware answer and pushes further in that direction, by a multiple you choose.
Turning the scale up makes the image follow the prompt more literally, at the cost of variety and, at extremes, natural-looking results. That trade-off is why the slider exists.
The demo below uses the same two predictions as the code. Move the guidance scale and watch the guided prediction slide along the line from the no-prompt prediction, through the with-prompt one, and beyond it.
The CFG Scale Slider, With Numbers
A text-to-image model predicts the noise twice: once without the prompt and once with it. Guidance starts from the first prediction and moves away from it, in the direction the prompt pulled, by a multiple you choose.
No prompt
[0, 1]
With prompt
[1, 0]
Prompt's effect
[1, -1]
Guided
[3.0, -2.0]
At scale 3.0 the guided prediction is [3.0, -2.0]: the prompt's effect, [1, -1], multiplied by 3.0 and added to the no-prompt prediction. Above 1 the prompt is exaggerated. Everyday interfaces use roughly 7 to 15; higher scales follow the prompt more strongly but at the cost of variety.
AI Lab's Levels 68-70 (Forward Diffusion, Reverse Denoising Step and Classifier-Free Guidance) implement all three formulas in numpy, over full arrays instead of single points. The math is exactly the same, vectorized.
Interview Signal is part of Pro
See a real weak answer next to a real strong one for this exact topic.
Quiz is part of Pro
Test what you just read with a short quiz, and bank the XP.
Implement all three from scratch: Level 68 (Forward Diffusion), Level 69 (Reverse Denoising Step), and Level 70 (Classifier-Free Guidance) in AI Lab.
Discussion0
Join the Discussion
Sign in to leave comments, reply to others, or like insights.
No comments yet. Be the first to start the thread!