>_ Jorge Uc / projects ← All projects
~/projects/diffusion-models // GENERATIVE AI
◆ From-scratch implementation PyTorchDDPM + CFGVisual experiments

Learning to create from noise.

Diffusion Models from Scratch explores how a neural network can learn to reverse a gradual noising process and generate new images. I rebuilt the workflow in PyTorch using denoising diffusion probabilistic models (DDPM), a conditional U-Net, time embeddings and classifier-free guidance.

Inspired by practical exercises completed during an NVIDIA Deep Learning Institute course; the public repository is an independent reconstruction following the loss of my original working notebooks.

Diffusion Models project cover illustration
PROJECT FILE / DM-001PYTORCH · IMAGE GENERATION
GENERATIVE METHODDDPMDenoising diffusion
NEURAL NETWORKU-NetConditional noise prediction
GUIDANCECFGClass-conditioned sampling
DATA MODALITY2DGrayscale & RGB imagery
01 / PROJECT CONTEXT

Can a model learn to undo randomness?

Instead of predicting an existing label, a diffusion model learns a sequence of transformations that turns random noise into a structured image.

01THE PROBLEM

Model image distributions

Learn the visual structure of training images and generate plausible new samples, rather than simply classifying images.

GENERATIVE MODELING
02THE ENGINEERING

Predict noise at every step

Train a time-aware neural network to estimate the noise added to an image at different stages of diffusion.

DENOISING / U-NET
03THE EXPERIMENT

Observe image emergence

Export the intermediate reverse-diffusion states as GIFs and videos to inspect how the sampling trajectory evolves.

VISUALIZATION / GENERATION
02 / THE GENERATIVE PRINCIPLE

Two processes, opposite directions.

Training exposes the network to noisy versions of real images. Generation starts with random noise and repeatedly predicts what should be removed.

FORWARD DIFFUSION / qTRAINING DATA → NOISE
▦x₀Real image
▒xₜNoisy image
░xTGaussian noise

The forward process gradually adds Gaussian noise according to a predefined noise schedule. A random timestep t can be sampled directly during training.

xt = √ᾱt · x₀ + √(1 − ᾱt) · ε
REVERSE DIFFUSION / pθNOISE → GENERATED IMAGE
░xTInitial noise
▒xₜDenoising
▦x₀New image

The U-Net estimates the noise present at the current timestep. A sampling rule removes a portion of that noise and iterates toward the final image.

εθ(xt, t, y) ≈ ε

Conceptual diagrams for explanation; they do not represent measured model outputs. The equations show the usual DDPM noise-prediction parameterization, where ε is sampled Gaussian noise and y is optional class conditioning.

03 / MODEL ARCHITECTURE

A U-Net that knows the time step.

The network operates at several spatial resolutions, combines local and contextual information through skip connections, and predicts a noise tensor matching the image shape.

> model.conditional_unetCONCEPTUAL ARCHITECTURE
TIME EMBEDDING (t)OPTIONAL CLASS LABEL (y)RESIDUAL + SKIP CONNECTIONS

Structural overview; the exact layer widths and training settings are configurable in the repository.

01 / ENCODER

Downsampling

Extract increasingly abstract features while reducing spatial resolution.

02 / BOTTLENECK

Deep representation

Combine rich features with information about the diffusion timestep and available conditions.

03 / DECODER

Upsampling

Recover spatial resolution, reusing encoder features to preserve finer details.

04 / TECHNIQUES

The building blocks behind generation.

These mechanisms connect a conventional convolutional network to a probabilistic image-generation pipeline.

01 / DDPM

Gaussian noise schedule

A sequence of noise strengths controls how much clean-image information remains at each timestep.

02 / POSITIONAL ENCODING

Sinusoidal time embeddings

Convert timestep indices into vectors so the network can adapt its prediction to different noise levels.

03 / LEARNING OBJECTIVE

Noise prediction loss

Use mean squared error between sampled Gaussian noise and the network's predicted noise.

04 / CONDITIONING

Class information

Provide labels to steer the model toward specific visual classes rather than generating unconditionally.

05 / GUIDANCE

Classifier-Free Guidance

Train with occasional dropped conditions, then combine conditional and unconditional noise predictions while sampling.

06 / INFERENCE

Iterative reverse sampling

Run repeated denoising steps and save intermediate frames to visualize the model's evolving estimates.

OBJECTIVE / NOISE PREDICTION
L = E [ ‖ ε − εθ(xt, t, y) ‖² ]

Instead of directly drawing the final image, the model learns to estimate the added noise. During conditional sampling, classifier-free guidance adjusts the estimated noise using predictions with and without the class label.

εguided = εuncond + w · (εcond − εuncond)
05 / END-TO-END WORKFLOW

From a dataset to a generation video.

The workflow separates model training from the sampling and export stages so experiments can be reproduced with different datasets or trained checkpoints.

> workflow.train_and_sampleREPRODUCIBLE PIPELINE
01

Prepare data

Load images, resize and normalize; construct train batches.

02

Sample timestep

Draw t and Gaussian noise ε for each image.

03

Add noise

Compute noisy xₜ using the diffusion schedule.

04

Predict ε

Pass xₜ, time embedding and optional label through U-Net.

05

Optimize

Minimize noise prediction MSE and save a model checkpoint.

06

Generate

Start from noise; perform reverse diffusion with optional CFG.

07

Export media

Save final grids and denoising frames as PNG, GIF and MP4.

TRAINING LOOP

Learn the denoising function

  1. Select an image and, where applicable, a class label.
  2. Sample a random timestep and Gaussian noise.
  3. Create a noisy observation with the schedule.
  4. Predict the injected noise using the conditional U-Net.
  5. Backpropagate MSE loss and update parameters.
  6. Save the trained network for later inference.
SAMPLING LOOP

Turn noise into a visual sequence

  1. Sample a Gaussian noise tensor with the target image shape.
  2. Choose a class condition if using conditional generation.
  3. Predict noise at each reverse timestep, optionally using CFG.
  4. Apply the reverse-diffusion update to get a cleaner tensor.
  5. Collect selected intermediate frames.
  6. Assemble an image grid and export animations and videos.
06 / DATASETS & EXPERIMENTS

From grayscale structure to RGB images.

The project explores denoising in both single-channel and color images, with increasing visual complexity and conditioning requirements.

EXPERIMENT A / GRAYSCALE

Fashion-MNIST

Small grayscale images of clothing provide a controlled setting to understand how the model learns the overall structure of objects while removing noise.

1 image channelClothing categoriesDDPM sampling
EXPERIMENT B / RGB

Flowers

Color imagery introduces an additional degree of difficulty: learning patterns across three correlated channels and exploring class-conditioned generation.

3 image channelsColor patternsConditional generation
ADDITIONAL EXERCISE

MNIST & multimodal concepts

The learning material also covered digit generation and concepts associated with CLIP embeddings. The public reconstruction emphasizes the DDPM training and sampling pipeline.

MNISTTime embeddingsCLIP concepts
07 / VISUAL EXPERIMENTS

Watch the sampling process.

These are the original animation files preserved from my NVIDIA coursework. They illustrate generation trajectories; the repository also provides scripts to produce new ones from freshly trained checkpoints.

01 / FASHION-MNISTMP4 · ORIGINAL COURSE OUTPUT

Grayscale diffusion sequence

Frame-by-frame visualization of a denoising experiment with Fashion-MNIST.

Open fashion.mp4 ↗
02 / FLOWERSMP4 · ORIGINAL COURSE OUTPUT

Color diffusion sequence

Visual record of an RGB flower-generation experiment and its evolving samples.

Open flowers.mp4 ↗
ANIMATED FRAMES / GIF

Alternative view of the experiments

The same two experiments are also preserved as GIF files. Animations may take a moment to load on slower connections.

Animated grid of Fashion-MNIST diffusion samples
Fashion-MNIST / GIFGrayscale denoising frames
Animated grid of RGB flower diffusion samples
Flowers / GIFRGB denoising frames
How to interpret these results. The videos and GIFs are qualitative records of past coursework experiments, not a quantitative benchmark for the rebuilt repository. No FID, IS, training-loss comparison or convergence claim is presented without a documented evaluation.
08 / REPRODUCIBLE IMPLEMENTATION

A project you can run again.

The independent repository organizes training, sampling and video export into separate commands. The exact number of epochs and image quality depend on the hardware, dataset and chosen settings.

terminal — diffusion-models-from-scratch
# Install requirements
pip install -r requirements.txt

# Run automated tests
python -m pytest -q

# Train the Fashion-MNIST model
python scripts/train.py --dataset fashion --epochs 20

# Sample images and export PNG / GIF / MP4
python scripts/sample.py --checkpoint checkpoints/fashion.pt --samples 70 --output demo/generated_fashion
01 / REPRODUCIBILITY

Separate train & inference

Checkpoint-based sampling avoids training again every time a new generation or video is needed.

02 / MEDIA OUTPUT

Automated exports

Intermediate diffusion frames are collected and combined into visual grids, GIF animations and MP4 clips.

03 / VERIFICATION

Tests & limitations

Automated tests exercise core model and export functionality; full training quality must be evaluated after running a complete experiment.

09 / PERSONAL RETROSPECTIVE

What I learned building diffusion models.

The most valuable outcome was understanding how architectural design, probability and an iterative inference algorithm combine to make image generation possible.

BEYOND THE FINAL IMAGE

Generative AI is a process, not just an output. Building and rebuilding this workflow helped me understand how the noise schedule, training objective, neural architecture and sampling decisions influence the final result.

01 / PROBABILISTIC MODELING

Generation as iterative denoising

I learned how a model can approximate a reverse stochastic process by predicting noise at different timesteps, rather than directly producing an image in one pass.

DDPMGaussian noiseSampling
02 / NEURAL ARCHITECTURES

Why conditioning matters

Time embeddings help a single U-Net operate across multiple noise levels; class information and CFG offer additional control during generation.

U-NetTime embeddingsCFG
03 / EXPERIMENTAL DESIGN

Visualize intermediate states

Inspecting the full reverse trajectory makes debugging and interpretation more informative than examining only the final generated grid.

GIFMP4Qualitative analysis
04 / SOFTWARE ENGINEERING

Make experiments reproducible

Separating model architecture, training, sampling and exports, along with basic tests and documentation, makes the implementation easier to revisit and extend.

PyTorchTestingDocumentation
KEY TAKEAWAY

Understanding the mechanism matters more than producing a visually impressive demo.

This project connects the mathematics of probabilistic modeling to practical deep learning engineering, while preserving an honest distinction between historical course demonstrations and the newer independent reconstruction.

10 / REPOSITORY & ATTRIBUTION

Open implementation. Clear provenance.

The GitHub repository is an independent learning implementation. NVIDIA is credited as the educational context; this project is not presented as an official NVIDIA product or an endorsement.

SOURCE CODE / PUBLIC

Diffusion Models from Scratch

Explore the PyTorch code, model components, training scripts, sampling pipeline, tests and project documentation.

View GitHub repository ↗
PROJECT PROVENANCE

From coursework to independent code

The work originated as hands-on exercises in an NVIDIA Deep Learning Institute course. After losing the original code, I created a separate reconstruction and retained the original GIF/MP4 outputs as historical demonstrations.

Educational attribution · no claim of official affiliation