Forward propagation, or the forward pass, runs input through a neural network to an output. See the steps, a worked example, and forward vs backpropagation.
Updated August 2026
Every time a trained model answers, it runs forward propagation. The input enters at the first layer, moves through the network one layer at a time, and leaves as a prediction. No learning happens on the way through: the weights stay fixed, and the network only computes. Forward propagation is that computation, and it is the half of a neural network that actually produces answers.
Quick answer: Forward propagation is the process of passing input data through a neural network, layer by layer, to produce an output. At each layer the network computes a weighted sum of its inputs, adds a bias, and applies an activation function, then feeds the result to the next layer. It runs during both training and inference and produces predictions without changing the model's weights.
Forward propagation is how a neural network turns input into output. The data moves in one direction, forward, from the input layer through the hidden layers to the output layer, with no loops back. The name covers both the path the data takes and the math done along it.
What forward propagation does not do is learn. The weights it multiplies by are fixed during the pass, and adjusting them is a separate step called backpropagation. Keeping that line clear is the key to the concept: forward propagation computes a prediction, it does not improve the model.
Forward propagation works by repeating the same two-step calculation at every layer. First, each neuron computes a weighted sum of the values from the previous layer and adds a bias. Second, that sum passes through an activation function that adds non-linearity. The activated output becomes the input to the next layer, and the network repeats the pair until the output layer produces the result.
This content was generated with the assistance of AI. Our AI prompt chain workflow is carefully grounded and preferences .gov and .edu citations when available. All content is reviewed by a Telnyx employee to ensure accuracy, relevance, and a high standard of quality.
For a single neuron, the linear step is z = w·x + b, where w is the weight, x the input, and b the bias. The activation step applies a function such as ReLU or sigmoid, written a = f(z). Stacking those two steps across many layers is what lets a network represent complex, non-linear relationships instead of just straight lines.
The full pass runs in order:
w·x + b from the previous layer's values.
In practice a network does not compute one neuron at a time. It stacks a layer's weights into a matrix W and computes the whole layer in one operation: Z = W·X + b, then A = f(Z). X is the vector of inputs, Z is every neuron's weighted sum, and A is every neuron's activation.
Writing it as a matrix multiply is what makes forward propagation fast. A single W·X covers an entire layer, and the same operation can run a whole batch of inputs together, which is why graphics processors accelerate neural networks so well. The math is identical to the neuron-by-neuron version; it is just computed in bulk.
Activation functions matter because they are what let a deep network learn more than a straight line. Without them, every layer would apply only a linear transformation, and stacking linear transformations just produces one more linear transformation. A hundred layers would collapse into the power of one.
The non-linear activation between layers breaks that collapse. It lets each layer bend the data in a new way, so the network as a whole can represent curved, complex relationships. The choice of function shapes what the layer can express: ReLU is the common default in hidden layers, while sigmoid and softmax are typical at the output when the task needs a probability. This is why forward propagation always pairs the linear step with an activation, never runs the linear step alone.
The forward pass is another name for forward propagation. In a neural network the two terms are interchangeable, and both describe input data moving through the layers to produce an output. Engineers tend to say forward pass for a single run of the network and forward propagation for the mechanism, but nothing separates them technically.
The forward pass runs in both phases of a model's life. During training it produces the prediction that gets compared against the correct answer to measure error. During inference it produces the prediction you actually use. The computation is identical in both cases; only what happens after the output differs.
That symmetry has a practical consequence. A deployed model does nothing but forward propagation, so the forward pass is the entire cost of answering a request. Its speed sets the model's inference latency, which is why teams that serve models in real time spend so much effort making the forward pass cheaper, through smaller networks, quantization, and hardware built for matrix math.
Forward and backward propagation move in opposite directions and do opposite jobs. Forward propagation runs input to output and computes a prediction. Backward propagation runs output to input and computes how to adjust the weights so the next prediction is better. One produces the answer, the other corrects the model.
| Forward propagation | Backward propagation | |
|---|---|---|
| Direction | Input to output | Output to input |
| Computes | The prediction | Gradients for each weight |
| Changes weights | No | Yes (via the optimizer) |
| Runs during | Training and inference | Training only |
Training alternates the two. A forward pass makes a prediction, an objective function measures how wrong it is, and backpropagation sends that error backward to update the weights. Inference uses forward propagation alone, which is why a deployed model runs only the forward direction. For the mechanics of the backward step, see the backpropagation guide.
A simple example of forward propagation is a single neuron computing one output. Suppose it receives two inputs, x1 = 2 and x2 = 3, with weights w1 = 0.5 and w2 = -1, and a bias of 1. The linear step gives z = (0.5 × 2) + (-1 × 3) + 1 = -1. Apply a ReLU activation, which outputs max(0, z), and the neuron's output is 0.
Now add a second neuron to that layer and one output neuron on top. The second neuron takes the same inputs with weights w1 = 1 and w2 = 0.5 and a bias of 0: its linear step is z = (1 × 2) + (0.5 × 3) + 0 = 3.5, and ReLU leaves it at 3.5. The first layer's outputs are now [0, 3.5].
Those two values become the inputs to the output neuron. With weights [2, 1] and a bias of -1, its linear step is z = (2 × 0) + (1 × 3.5) - 1 = 2.5, and the network's output is 2.5. Repeat the same two steps, linear then activation, and a network of any depth computes its output the same way. That single trip from input to output is a complete forward pass.
Yes, forward propagation and the forward pass are the same process. Both describe input data moving through a neural network's layers to produce an output. The forward pass is the more common phrase for a single run of the network; forward propagation is the more common phrase for the underlying mechanism.
In a recurrent neural network, forward propagation moves through time as well as through layers. The network processes a sequence one step at a time, and each step passes its output forward to the next step as a hidden state. The two-step calculation is the same; it just repeats across the sequence.
Not quite, though the two are closely related. Feedforward describes a network architecture where data flows in one direction with no loops, while forward propagation is the process of running data through a network. A feedforward network uses forward propagation, but so does the forward pass of a recurrent network, so the terms overlap rather than match.
No, forward and backpropagation are opposite steps. Forward propagation computes a prediction by moving input to output. Backpropagation computes weight updates by moving the error from output back to input. Training uses both; inference uses only forward propagation.