Backpropagation is how a neural network learns, sending error backward to update weights. See how it works, a worked example, and backpropagation vs gradient descent.
Updated August 2026
A neural network starts out wrong. Its weights are random, its first predictions are guesses, and the only way it improves is by learning how wrong each weight made it. Backpropagation is the algorithm that works that out. It sends the error backward through the network and tells every weight how to change, and it is the reason deep learning works at all.
Quick answer: Backpropagation is the algorithm neural networks use to learn. After a forward pass produces a prediction, backpropagation compares it to the correct answer, then sends that error backward through the network to compute how much each weight contributed to it. An optimizer uses those values to update the weights, and repeating the cycle is how the network improves.
Backpropagation is short for backward propagation of error, and the name captures the whole idea: it runs the error backward through the network, from the output where the mistake is measured toward the input. That backward direction is the defining move. It is what lets a deep network with many layers assign the blame for a wrong answer all the way down to the weights that caused it, which is the hard part of training and the reason the method matters.
Before backpropagation, that blame-assignment was the unsolved problem holding neural networks back. The method was set out in a 1986 by David Rumelhart, Geoffrey Hinton, and Ronald Williams, and it turned neural networks from a theoretical idea into something you could actually train. Every modern deep learning model, from image classifiers to language models, learns this way.
This content was generated with the assistance of AI. Our AI prompt chain workflow is carefully grounded and preferences .gov and .edu citations when available. All content is reviewed by a Telnyx employee to ensure accuracy, relevance, and a high standard of quality.
Backpropagation works by running the error backward through the network one layer at a time, using the chain rule from calculus to compute a gradient for every weight. A gradient is just the slope of the loss with respect to one weight: it says which direction to move that weight, and how much, to reduce the error.
The full training step has four parts:
The chain rule is what makes the backward pass efficient. Each layer's gradient is the local derivative at that layer multiplied by the gradient flowing back from the layer after it. Because every layer reuses the gradient already computed downstream, the network finds all its gradients in a single backward sweep instead of recalculating from scratch for each weight.
Forward and backward propagation are the two halves of a training step, running in opposite directions. Forward propagation moves input to output and produces the prediction. Backward propagation moves the error from output back to input and produces the gradients that improve the model. A network cannot run backpropagation without a forward pass first, because the backward pass needs the prediction and the values each neuron computed on the way through.
The division of labor is clean: the forward pass asks "what does the model predict," and the backward pass asks "how should each weight change." For the mechanics of the forward direction, see the forward propagation guide.
Backpropagation and gradient descent are often confused, but they do different jobs. Backpropagation computes the gradients, the direction and size of the change each weight needs. Gradient descent uses those gradients to actually update the weights, stepping each one a small amount against its gradient. Backpropagation finds the slope; gradient descent walks down it.
They run together on every training step, but they are separable. Backpropagation is the only practical way to compute the gradients in a deep network, while the update step can use plain gradient descent or a more advanced optimizer such as Adam. Swapping the optimizer changes how the step is taken, not how the gradients are found.
The size of each step is set by the learning rate. Too large a learning rate can overshoot the minimum and make training diverge, which is the same instability behind exploding gradients. Too small a rate makes training crawl. Modern optimizers such as Adam adapt the effective step per weight to make that balance less fragile, but they still update using the gradients backpropagation computes.
A simple example of backpropagation is one weight learning from one prediction. Take a single neuron with input x = 2, weight w = 0.5, and no bias, so its prediction is ŷ = w·x = 1.0. Suppose the target is y = 2 and the loss is L = ½(ŷ - y)², which comes to 0.5.
The backward pass applies the chain rule to find how the loss changes with the weight: dL/dw = (ŷ - y) × x = (1.0 - 2) × 2 = -2. That gradient is negative, so increasing the weight will lower the loss. With a learning rate of 0.1, gradient descent updates the weight to w = 0.5 - 0.1 × (-2) = 0.7.
The new prediction is ŷ = 0.7 × 2 = 1.4, and the loss drops from 0.5 to 0.18. One step made the neuron less wrong. A real network runs this same chain-rule calculation for every weight at once, over many examples, and the repeated small corrections are what add up to a trained model.
The main challenge of backpropagation is unstable gradients in deep networks. When gradients pass back through many layers, they can shrink toward zero, called vanishing gradients, so early layers barely learn. They can also grow uncontrollably, called exploding gradients, so training diverges. Vanishing gradients are worst with saturating activations like sigmoid, which is part of why ReLU became the common default in hidden layers.
Backpropagation can also overfit, tuning the weights so closely to the training data that the model performs worse on new inputs. Teams manage these problems with careful weight initialization, gradient clipping for the exploding case, and regularization methods such as dropout for overfitting. None of these replace backpropagation; they keep its gradients well behaved.
Backpropagation uses the chain rule from differential calculus. The chain rule lets the algorithm break the derivative of the loss with respect to a deep weight into a product of simpler local derivatives, one per layer. Multiplying those derivatives from the output back toward the input gives each weight's gradient without recomputing the whole network for every parameter.
Yes, backpropagation is the standard training algorithm for neural networks and deep learning. Almost every deep model, including convolutional networks for images, recurrent networks for sequences, and transformers for language, is trained by backpropagation paired with an optimizer. Recurrent networks use a variant called backpropagation through time, which unrolls the sequence and runs the same backward pass across its steps. Frameworks such as PyTorch and TensorFlow compute the backward pass automatically through a feature called automatic differentiation.
In simple terms, backpropagation is how a network learns from its mistakes. It makes a prediction, checks how wrong it was, and then works backward to assign blame to each weight for that error. Each weight is then adjusted a little in the direction that would have made the prediction better, and doing this many times teaches the network.
