Multilayer Perceptron (MLP): Architecture, Formula, Working, Examples, and Applications
A Multilayer Perceptron (MLP) is one of the fundamental architectures in artificial neural networks and deep learning. It uses multiple layers of interconnected neurons to learn patterns from data and make predictions. A basic perceptron can only solve linearly separable problems. A multilayer perceptron overcomes this limitation by adding hidden layers and non-linear activation functions. As a result, an MLP can learn much more complex relationships. MLPs form an important foundation for understanding neural networks. Concepts such as weights, biases, activation functions, forward propagation, loss functions, backpropagation, and gradient-based optimization all play a central role in how they work. This guide explains what a multilayer perceptron is, how its architecture works, the formulas behind it, why non-linearity matters, how an MLP solves the XOR problem, how to calculate its parameters, and how to build one in Python. What Is a Multilayer Perceptron? A Multilayer Perceptron (MLP) is a type of feedforward artificial neural network that contains an input layer, one or more hidden layers, and an output layer. The network receives input data, transforms that data through its hidden layers, and produces a prediction through its output layer. A typical MLP follows this structure: Input Layer → Hidden Layer(s) → Output Layer Each neuron receives information from the previous layer, applies weights and biases, performs a mathematical calculation, and passes the result through an activation function. During training, the MLP adjusts its weights and biases to reduce prediction errors. MLPs can perform both: Classification Regression For example, an MLP can classify whether an email is spam or predict the price of a house. Multilayer Perceptron Architecture A standard multilayer perceptron contains three main types of layers. Input Layer The input layer receives the original data. Each input neuron usually represents one feature. For example, a model that predicts student performance might receive: Hours studied Attendance percentage Previous exam score Number of completed assignments If the dataset contains four input features, the input layer usually receives four input values. The input layer does not perform the main learning process. Instead, it passes the feature values to the first hidden layer. Hidden Layers Hidden layers perform most of the transformations inside an MLP. Each neuron receives outputs from the previous layer, calculates a weighted sum, adds a bias, and applies an activation function. An MLP can contain: One hidden layer Several hidden layers Adding hidden layers allows the network to build increasingly complex representations of the input data. For example, an early hidden layer may learn simple relationships, while later layers can combine those relationships into more complex patterns. Output Layer The output layer produces the final prediction. The number of output neurons and the activation function depend on the problem. For example: A regression problem may use one output neuron. A binary classification problem often uses one output neuron with a sigmoid function. A multi-class classification problem often uses multiple output neurons with softmax. Are MLP Layers Fully Connected? Yes. A standard multilayer perceptron uses fully connected layers, also called dense layers. In a fully connected layer, every neuron in one layer connects to every neuron in the next layer. For example, if an input layer contains four neurons and the next hidden layer contains six neurons, every one of the four inputs connects to all six hidden neurons. These connections contain learnable weights. This fully connected structure gives an MLP its ability to combine information from multiple input features. However, it can also create a large number of parameters as the network grows. Multilayer Perceptron Formula The mathematical foundation of an MLP begins with the calculation performed by a single neuron. A neuron calculates a weighted sum: [ z = w_1x_1 + w_2x_2 + \dots + w_nx_n + b ] The neuron then applies an activation function: [ a = f(z) ] Where: (x_1, x_2, \dots, x_n) are the input values (w_1, w_2, \dots, w_n) are the weights (b) is the bias (z) is the weighted input (f) is the activation function (a) is the output of the neuron An MLP performs this calculation across many neurons and layers. General Multilayer Perceptron Formula For any layer (l), an MLP calculates the weighted input as: \mathbf{W}^{(l)} \mathbf{A}^{(l-1)} + \mathbf{b}^{(l)} ] The layer then applies an activation function: f\left( \mathbf{Z}^{(l)} \right) ] Combining both steps gives the general MLP formula: f\left( \mathbf{W}^{(l)} \mathbf{A}^{(l-1)} + \mathbf{b}^{(l)} \right) ] Where: (\mathbf{A}^{(l-1)}) is the output from the previous layer (\mathbf{W}^{(l)}) is the weight matrix (\mathbf{b}^{(l)}) is the bias vector (\mathbf{Z}^{(l)}) is the weighted input (f) is the activation function (\mathbf{A}^{(l)}) is the output of the current layer The output from one layer becomes the input to the next layer. MLP Formula for Multiple Layers Consider an MLP with an input layer, two hidden layers, and an output layer. The original input is: [ \mathbf{A}^{(0)} = \mathbf{X} ] First Hidden Layer The first hidden layer calculates: \mathbf{W}^{(1)} \mathbf{X} + \mathbf{b}^{(1)} ] It then applies an activation function: f_1 \left( \mathbf{W}^{(1)} \mathbf{X} + \mathbf{b}^{(1)} \right) ] Second Hidden Layer The second hidden layer receives the output from the first hidden layer: \mathbf{W}^{(2)} \mathbf{A}^{(1)} + \mathbf{b}^{(2)} ] Its output becomes: f_2 \left( \mathbf{W}^{(2)} \mathbf{A}^{(1)} + \mathbf{b}^{(2)} \right) ] Output Layer The output layer receives the final hidden representation: \mathbf{W}^{(L)} \mathbf{A}^{(L-1)} + \mathbf{b}^{(L)} ] The network then produces the prediction: f_L \left( \mathbf{W}^{(L)} \mathbf{A}^{(L-1)} + \mathbf{b}^{(L)} \right) ] Here, (\hat{\mathbf{Y}}) represents the predicted output. Understanding Matrix Dimensions in an MLP Understanding matrix dimensions helps explain how an MLP processes multiple inputs and neurons efficiently. Suppose: The input contains (n) features. The first hidden layer contains (h) neurons. The input vector can have the shape: [ \mathbf{X} \in \mathbb{R}^{n} ] The weight matrix can have the shape: [ \mathbf{W}^{(1)} \in \mathbb{R}^{h \times n} ] The bias vector can have the shape: [ \mathbf{b}^{(1)} \in \mathbb{R}^{h} ] The output of the hidden layer therefore has the shape: [ \mathbf{A}^{(1)} \in \mathbb{R}^{h} ] For a batch of (m) examples, implementations may organize the dimensions differently depending on the framework. However, the










