The bias-variance tradeoff balances two sources of model error. Learn what bias and variance are, how the tradeoff works, and how to manage it with examples.
Updated August 2026
A machine learning model can fail in two opposite ways. It can be too simple to capture the pattern, or so flexible that it memorizes the noise. The bias-variance tradeoff is the reason you usually cannot fix both at once, and it is why the best model is rarely the most powerful one.
Quick answer: The bias-variance tradeoff describes two sources of error in a model. Bias is error from oversimplifying the problem, which causes underfitting. Variance is error from being too sensitive to the training data, which causes overfitting. Lowering one often raises the other, so the goal is the balance that minimizes total error on new data.
The bias-variance tradeoff comes from a single constraint: a machine learning model has a fixed budget of flexibility, and where it spends that budget decides how it fails. Spend it fitting the training data closely, and the model absorbs that sample's noise along with its signal. Spend it staying simple, and the model misses the real pattern.
The skill spends that flexibility where it helps on data the model has not seen, not where it flatters the training set. A model tuned to ace its training data has usually spent its flexibility in the wrong place.
Bias and variance are two different ways a model can be wrong.
| Source | What it is | Symptom | Failure mode |
|---|---|---|---|
| Bias | Too-simple assumptions | Poor on training and test data | Underfitting |
| Variance | Too-sensitive to training data | Strong on training, poor on test | Overfitting |
A quick tell: high bias shows up as bad performance everywhere, while high variance shows up as a large gap between training and test performance.
The bias-variance tradeoff works by breaking a model's expected error into three parts, then trading two of them against each other: bias squared, variance, and irreducible error. The irreducible part is noise you cannot remove. The trade happens on the other two, bias and variance, and the lever is model complexity.
As complexity rises, bias falls and variance climbs. As complexity drops, bias climbs and variance falls. Total error follows a U-shape: high at both extremes, lowest at the complexity where the fall in bias and the rise in variance balance out. The bottom of that U is the target.
For a concrete read, imagine a model's expected error on new data is 0.30. If bias squared accounts for 0.20 of that and variance for 0.08, the model is bias-dominated, so the fix is more complexity or better features. Flip those numbers, with variance at 0.20, and the same total error calls for the opposite: less complexity, or more regularization.
Fitting a curve to noisy data is the clearest example. Fit the same points three ways, from too simple to too flexible, and all three cases appear.
Scikit-learn's underfitting-and-overfitting example shows exactly this with polynomial fits: the underfit model and the overfit model both score worse on new data than the one in between.
You manage it by moving model complexity toward the bottom of the U, then checking the result on held-out data.
Regularization is the most common lever, and it works straight through this tradeoff. It adds a penalty that discourages complexity, raising bias slightly to buy a larger drop in variance. That is the deal a good regularization setting makes.
Yes, but not by sliding along the same curve. The tradeoff describes a fixed dataset and model family: at that setting, lowering one raises the other. Change the setting, and both can fall together.
More training data is the clearest way. With more data, a flexible model can afford its flexibility without overfitting, so variance drops while bias stays low. Better features do the same, by making the real pattern easier to capture. This is why "get more or better data" often beats tuning: it moves the whole error curve down instead of sliding along it.
Compare training error with validation error.
That difference, the generalization gap, is the fastest read on which side of the tradeoff you are on. Google's machine learning crash course treats closing that gap as the core goal of generalization. Chase the wrong side and the model gets worse: adding complexity to a high-variance model only deepens the overfitting. Extra features can raise variance on their own, because high-dimensional data spreads thin, a problem covered under the curse of dimensionality.
The tradeoff is the framework for choosing a model, not just an abstract idea. It says a bigger, more flexible model is not automatically better, because past a point its variance costs more than its lower bias saves.
It also decides what to do when a model underperforms. A high-bias model and a high-variance model need opposite fixes, so diagnosing which one you have is the difference between a change that helps and one that makes things worse. That diagnosis, more than raw model power, is what usually separates a working model from a stuck one.
The bias-variance tradeoff is the balance between two sources of model error. Bias comes from a model that is too simple and underfits; variance comes from a model that is too sensitive and overfits. Reducing one often increases the other, so the goal is the balance with the lowest error on new data.
Bias is error from oversimplified assumptions, so the model is wrong in a consistent way and underfits. Variance is error from over-sensitivity to the training data, so the model overfits and changes a lot when the data changes. High bias hurts both training and test performance; high variance shows up as a gap between them.
Reduce variance by simplifying the model, adding regularization, collecting more training data, or averaging several models through ensembling. Each lowers the model's sensitivity to the specific training sample, usually at the cost of a small rise in bias.
Yes, regularization increases bias. It adds a penalty that discourages complexity, which raises bias slightly in exchange for a larger reduction in variance. When a model overfits, that trade lowers total error, which is the tradeoff working as intended.
This content was generated with the assistance of AI. Our AI prompt chain workflow is carefully grounded and preferences .gov and .edu citations when available. All content is reviewed by a Telnyx employee to ensure accuracy, relevance, and a high standard of quality.
