Feedforward Neural Networks

Feedforward Neural Networks

Architecture Overview

A feedforward neural network (also called Multi-Layer Perceptron or MLP) consists of:

1. Input Layer

2. Hidden Layers

3. Output Layer

Forward Propagation

Data flows in one direction: Input → Hidden Layers → Output

For each layer: z = W × x + b (linear transformation) a = activation(z) (non-linear activation)

Network Design Considerations

Width vs Depth:

Common Configurations:

Universal Approximation Theorem

A neural network with a single hidden layer can approximate any continuous function, given enough neurons. However, deep networks can represent functions more efficiently.