🎬Video•35 min

Backpropagation Explained

Master the algorithm that makes neural network learning possible - backpropagation.

Backpropagation: The Heart of Neural Network Learning

Backpropagation computes gradients for neural network training. It calculates how much each weight contributed to the error.

∂L/∂w = ∂L/∂y × ∂y/∂z × ∂z/∂w

We propagate gradients backward through the network.

w_new = w_old - learning_rate × ∂L/∂w

Variants:

Batch GD: Uses entire dataset (stable but slow)

SGD: Uses single example (noisy but fast)

Mini-Batch: Uses small batches (best of both)

Adam - Most popular, combines momentum with adaptive learning rates AdamW - Adam with weight decay, used in transformers