Xavier Initialization

Weight Initialization

Why It Matters

Poor initialization leads to:

Vanishing gradients (weights too small)

Exploding gradients (weights too large)

Slow convergence

Xavier/Glorot Initialization

For sigmoid/tanh activations: W ~ Normal(0, √(2/(nᵢₙ + nₒᵤₜ)))

He Initialization (Kaiming)

For ReLU activations: W ~ Normal(0, √(2/nᵢₙ))

Guidelines

|------------|---------------|