The expectation-maximization (EM) algorithm finds model parameters when data has hidden variables. Learn the E-step and M-step, a worked example, and its limits.
Updated August 2026
The expectation-maximization algorithm solves a chicken-and-egg problem. To fit a model you need to know which hidden group each data point belongs to, but to figure out the groups you need the model. EM breaks the deadlock by guessing one, using it to improve the other, and repeating until both settle. It is how you train a model when some of the data you need was never observed.
Quick answer: The expectation-maximization (EM) algorithm is an iterative method for finding the parameters of a statistical model when the data has hidden, or latent, variables. It alternates two steps: the E-step estimates the missing information from the current parameters, and the M-step updates the parameters to best fit that estimate. Each round increases the likelihood, and the loop repeats until the parameters stop changing. It is best known for fitting Gaussian mixture models, a form of clustering.
The expectation-maximization (EM) algorithm is a method for finding the most likely parameters of a model when some of the data is unobserved. That unobserved part is called a latent variable, and it is what makes the fit hard: you cannot directly maximize the likelihood because the equation depends on values you never saw.
A quick example makes the latent variable concrete. Imagine customer purchase histories where each customer belongs to an unlabeled segment, budget shopper or premium buyer, that you never recorded. The segment shapes the buying pattern but never appears in the data. EM infers those hidden segments and the spending model for each at the same time.
EM sidesteps this by filling in the missing values with their expected values, then optimizing as if they were real, and iterating. It was formalized by Dempster, Laird, and Rubin in their 1977 paper, and it remains a standard tool for clustering, density estimation, and any model with hidden structure. What it computes is a maximum likelihood estimate, the same target as ordinary maximum likelihood, reached by a route that works when latent variables block the direct one.
The EM algorithm works by alternating an expectation step and a maximization step until the parameters converge. It runs in four parts:
The reason it works is a guarantee: every full E-M round increases the likelihood, or leaves it unchanged, but never lowers it. So the loop climbs steadily toward a better fit. The E-step and M-step lean on each other, better assignments give better parameters, and better parameters give better assignments, which is how the chicken-and-egg problem unwinds. Because the likelihood only ever rises, the process is guaranteed to stop: once a round no longer changes the parameters, EM has reached a stable fit and halts.
The classic example of the EM algorithm is the Gaussian mixture model, a soft clustering method. Suppose points on a line came from two overlapping bell curves, but you do not know which curve produced each point or where the curves sit. EM finds both.
It starts with a rough guess for each curve's mean and spread. In the E-step, it assigns every point a probability of belonging to curve A versus curve B, so a point between the two might come out 70% A and 30% B. In the M-step, it recomputes each curve's mean and spread using all the points, weighted by those probabilities. After a few rounds, the curves settle onto the two clusters and the assignments stop shifting.
This is also why EM is described as the soft version of k-means. K-means assigns each point to exactly one cluster; EM lets a point belong partly to several, which handles overlapping groups that a hard split cannot.
A second classic example uses two coins with unknown biases. You see several runs of flips but not which coin produced each run, and that missing label is the latent variable. The E-step estimates, for each run, how likely each coin is to have produced it. The M-step re-estimates each coin's bias from the flips, weighting every run by those likelihoods. Repeat the two steps and both biases converge on their true values, even though you never learned which coin threw which run.
The main limitation of the EM algorithm is that it converges to a local optimum, not necessarily the global one. It climbs to the nearest peak of the likelihood surface, and if that peak is not the highest, EM will not find the better one on its own. This makes it sensitive to the starting guess, which is why practitioners run it several times from different initializations and keep the best result.
Two other costs matter in practice. EM is iterative, so it can be slow on large datasets, since every round touches every data point. And it needs a probabilistic model with a defined likelihood; it is not a general-purpose optimizer you can point at any problem, the way gradient descent and backpropagation can be. When those conditions hold, though, it is stable and reliable.
The EM algorithm is used wherever a model has hidden structure or missing values. Its most common jobs are:
The E-step estimates the hidden information; the M-step updates the model. In the E-step, the algorithm uses the current parameters to compute the expected values of the latent variables, such as the probability each point belongs to each group. In the M-step, it holds those estimates fixed and adjusts the parameters to maximize the expected likelihood. The two alternate.
The EM algorithm always converges in the sense that the likelihood never decreases from one round to the next, so it reaches a stationary point. It does not always reach the best possible solution, because that stationary point can be a local optimum rather than the global one. Running it from several starting guesses is the usual guard against a poor local result.
The EM algorithm is a way to perform maximum likelihood estimation when latent variables make the direct calculation intractable. Ordinary maximum likelihood maximizes the probability of the observed data in one shot; EM reaches the same kind of estimate iteratively, by filling in the missing values and re-optimizing, which is what makes latent-variable models solvable.
This content was generated with the assistance of AI. Our AI prompt chain workflow is carefully grounded and preferences .gov and .edu citations when available. All content is reviewed by a Telnyx employee to ensure accuracy, relevance, and a high standard of quality.
