🎬Video•25 min

Perceptrons and Activation Functions

Understand the building blocks of neural networks - from perceptrons to activation functions.

The fundamental building block of neural networks:

output = activation(Σ(wᵢ × xᵢ) + b)

Without them, neural networks are just linear transformations. Activation functions introduce non-linearity.

1. Sigmoid - σ(x) = 1 / (1 + e⁻ˣ)

Output: (0, 1), Good for binary classification output

2. ReLU - ReLU(x) = max(0, x)

Most popular for hidden layers, Fast computation

3. GELU - Used in Transformers (BERT, GPT)

4. Softmax - Output: Probability distribution

Use for multi-class classification