Emergent behavior in AI is a capability that appears only past a threshold of scale, interaction, or training. Examples, causes, the mirage debate, and risks.
Updated October 2026
Emergent behavior in AI is a capability or pattern of behavior that appears in a larger or more complex AI system but is absent from smaller or simpler versions of it. It could not have been predicted by extrapolating from those versions. Large language models that start doing multi-digit arithmetic past a certain size, and game-playing agents that invent tool use against each other, are two of the most cited examples.
Nobody wrote either of those abilities into the code, which is true of every skill a neural network learns. What makes them emergent is the jump. In Wei and colleagues' 2022 survey of emergent abilities in large language models, GPT-3 models scored near random on 3-digit addition and subtraction until about 13 billion parameters. Past that size, their scores jumped sharply above random. Whether jumps like that are real or an artifact of how they are scored is the field's main open argument. The answer decides how well anyone can predict what the next, larger model will do.
Emergent behavior in AI is recognized by its curve: as a model grows, more agents interact, or training runs longer, the behavior stays near random, then jumps past a threshold that the earlier measurements gave no sign of. Wei and colleagues describe this as a qualitative change caused by a quantitative one, such as more parameters, more training, or more agents. Their 2022 paper, "Emergent Abilities of Large Language Models," traces the idea to physicist Philip Anderson's 1972 essay More is different, which argued that entirely new properties appear at each level of complexity, even when the laws of the level below are fully known.
For language models, Wei and colleagues turned that curve into a test: "An ability is emergent if it is not present in smaller models but is present in larger models." On a graph of a model family's score against model size, an ordinary learned ability climbs from the first point, and each model's score predicts the next. An emergent ability gives no such warning.
That test is why "not explicitly programmed" is not enough to call a behavior emergent. A spam filter is not programmed to flag a particular phrase either. The filter learns the phrase from labeled email, and it gets steadily better as it sees more. Emergence names two properties together: the change is sharp, and the smaller system gave no sign of it.
Emergent behavior is the broadest of the three terms, emergent abilities the narrowest, and emergent properties the oldest. In the AI literature they usually carry these meanings:
Outside AI, emergent behavior is behavior of a whole system that none of its parts shows on its own. The standard computer example is flocking. In his 1987 boids model, Craig Reynolds gave each simulated bird three local rules: avoid crowding nearby flockmates, steer toward their average heading, and move toward their average position. No rule mentions a flock, yet the birds move as one.
Flocking is what philosophers call weak emergence, the milder of two strengths. In David Chalmers' account of strong and weak emergence, a weakly emergent behavior is unexpected given the low-level rules, though it can still be deduced from them. A strongly emergent one could not be deduced from those rules even in principle.
Every emergent behavior documented in AI is weak emergence. The code and the weights fully specify the computation. The surprise is that the result could not be forecast without running the larger system. AI research borrows the idea with a twist. It compares smaller and larger versions of a system rather than parts and wholes, and calls a behavior emergent when the larger version shows it and the smaller ones give no sign of it.
Emergent behavior in AI arises along three axes: the size of a single model, interaction between agents over time, and the length of training. Each axis has a documented case where a behavior was absent for a long stretch, then appeared abruptly.
Model size is the axis most research covers. As a language model grows, its average error at predicting the next word falls smoothly. A scaling laws study found that this loss follows a power law in model size, data, and compute, across more than seven orders of magnitude. Specific abilities do not follow that smooth curve. Ganguli and colleagues called the combination predictability and surprise. Developers can forecast a model's average loss before training it, but not which tasks it will suddenly be able to do. The section on large language models below covers those abilities.
Interaction between agents produces emergent behavior when agents respond to each other, so each agent's actions change the situation the others face. In a hide-and-seek experiment, Baker and colleagues trained two teams with standard reinforcement learning. The only reward was visibility: hiders scored when every hider was out of the seekers' sight, and seekers scored otherwise. Over hundreds of millions of games, the teams went through six distinct strategies. Hiders barricaded themselves with boxes, seekers climbed into the shelter with ramps, and hiders then locked the ramps away first.
The researchers expected ramp defense to be the last stage. After 380 million games, seekers learned to push a box beside a locked ramp, climb onto it, and ride it into the shelter. The paper calls the move box surfing. It worked because the physics engine let an agent apply force to itself while standing on a box it held. Nobody had designed for that.
Language-model agents show the same effect with words instead of physics. Park and colleagues ran a simulation of 25 generative agents living in a small town and gave one agent a single idea: throw a Valentine's Day party. Over the next two simulated days, the other agents spread the invitations, asked each other to the party, and arrived together at the right time. No instruction told them to do any of it, and no agent learned anything new during the run: the party came from agents responding to each other's messages. That is emergence in the broad whole-versus-parts sense: no single agent planned the party, though no threshold in scale was crossed. The same dynamic applies to multi-agent language models built for real work.
Training time produces emergent behavior when a model's ability to generalize appears long after it has memorized its training data. Power and colleagues named this grokking. Their datasets were small algorithmic tables, such as modular arithmetic, the clock-style arithmetic where numbers wrap around past a fixed value. Small neural networks trained on them reached perfect training accuracy early. Their accuracy on unseen examples stayed at chance. Trained well past the point of overfitting, the same networks rose from chance to perfect accuracy on those unseen examples.
Grokking belongs to the same family of surprises as double descent, where a model's test error falls, rises, and falls again as the model grows or trains longer. Both break the textbook expectation that a model which has started to overfit only gets worse with more size or more training.
Emergent abilities of large language models are tasks that a model family performs near random at small sizes and well above random past a size threshold. Wei and colleagues' survey gathered them from published benchmark results. Each threshold below is where the ability appeared in the model families named on its row:
| Ability | Where it appeared | Model family |
|---|---|---|
| 3-digit addition and subtraction | About 13 billion parameters | GPT-3 |
| Massive Multitask Language Understanding (MMLU), 57 subjects | 70 to 280 billion parameters | GPT-3, Gopher, Chinchilla |
| Chain-of-thought gains on math word problems | 68 billion parameters | LaMDA |
| Word in Context, telling apart two senses of one word | 540 billion parameters | PaLM |
Most of those results were measured with in-context learning: the model performs a new task from a few examples in its prompt, without any retraining. The GPT-3 paper showed that this few-shot learning improves greatly with scale. Its 175-billion-parameter model solved tasks such as unscrambling words and 3-digit arithmetic from a handful of examples. In-context learning is itself emergent on the training-time axis. Olsson and colleagues found that it appears in a sudden jump during training, at the point where attention heads they call induction heads form.
The way a model is prompted can be emergent too. Chain-of-thought prompting asks a model to write out its intermediate reasoning steps before the answer. That beat asking for the answer directly only in the largest LaMDA models, from 68 billion parameters. Instruction tuning, which fine-tunes a model on many tasks phrased as instructions, followed the same pattern in Wei and colleagues' survey. It hurt models of 8 billion parameters or fewer and helped only the largest ones.
Each threshold belongs to its model family, not to large language models in general. Wei and colleagues say they do not claim that any specific scale is required. They note that the same abilities can appear in smaller models trained on higher-quality data. A 2024 study by Du and colleagues found that pre-training loss, the model's error on its training text, predicts when an ability appears better than parameter count does. In their experiments, models of different sizes with the same loss performed the same on downstream tasks.
Many reported emergent abilities of large language models turn into smooth curves when they are scored differently, but not all of them. In 2023, Schaeffer, Miranda, and Koyejo argued that much of the apparent jump comes from the choice of metric. An all-or-nothing score, such as exact match on a multi-token answer, stays near zero until nearly every token is right. A per-token score of the same outputs rises smoothly the whole way.
The metric effect takes one line of arithmetic to show. In this example, a model family's per-token accuracy rises smoothly with size, and each model is scored on a 10-token answer that counts only if every token is right. With tokens treated as independent, exact match equals per-token accuracy raised to the tenth power. The per-token curve below is an illustrative assumption, not a fit to any real model. Per-token values are rounded to three decimals, and exact match is computed before rounding:
| Model size (parameters) | Per-token accuracy | Exact match, 10-token answer |
|---|---|---|
| 10 million | 0.500 | 0.001 |
| 100 million | 0.719 | 0.037 |
| 1 billion | 0.842 | 0.179 |
| 10 billion | 0.911 | 0.394 |
| 100 billion | 0.950 | 0.599 |
| 1 trillion | 0.972 | 0.752 |
Both columns describe the same models. Each tenfold step adds less per-token accuracy than the one before it, from 22 points between the two smallest models down to 2 between the two largest. Exact match does the opposite at first: it gains under 4 points, then 14, then about 21 on each of the next two steps, before slowing to 15. On a log scale, the exact-match column looks like an ability switching on. The per-token column looks like steady learning.
Schaeffer and colleagues found the same pattern in published benchmark data. BIG-Bench is a suite of more than 200 language tasks. Of the 39 scoring metrics its tasks use, at most 5 showed emergence. Two of them, Multiple Choice Grade and Exact String Match, accounted for more than 92% of the claimed emergent abilities. On the tasks the authors reanalyzed, switching the LaMDA models from Multiple Choice Grade to a continuous score made the emergent abilities disappear.
The metric argument does not settle the question, and its authors say so. Their paper states that nothing in it claims language models cannot display emergent abilities. Du and colleagues found that performance on MMLU, the Chinese exam benchmark C-Eval, and the grade-school math benchmark GSM8K stayed at random guessing until pre-training loss fell below a threshold. On MMLU and C-Eval the same jump showed under two continuous scores, the probability given to the correct choice and the Brier score. The two studies plot different things: Schaeffer and colleagues put model size on the x-axis for the LaMDA family on BIG-Bench tasks, while Du and colleagues put pre-training loss on it for their own models on other benchmarks.
The working conclusion is narrower than either side's headline. Some jumps come from the scoring rule, some are real thresholds, and a team can tell which only by scoring the same outputs both ways.
Emergent abilities can be partly predicted before a model reaches them. Pre-training loss is a better yardstick than size, and loss itself can be forecast. But each task's loss threshold has to be measured before it can be used, and smaller models give no sign of where it sits. By definition, smaller models' scores on the same task and metric give no warning, so prediction needs a different measurement. A continuous metric turns some apparent jumps into trends that can be extrapolated, which is the practical lesson of the mirage argument.
Snell and colleagues found a second route to prediction in 2024. Fine-tuning a model on a task pulls the point where an ability emerges toward smaller models, and more fine-tuning data pulls it further. Fitting that relationship and extrapolating it back to the model without fine-tuning predicts where its threshold sits. Using only small models, they predicted in some cases whether models trained with up to four times more compute would show the ability.
Emergent behavior in AI is a risk mainly because it is unpredictable: a capability no one measured before release was never tested or guarded against. Most documented emergent abilities are useful, such as arithmetic, reasoning, and following instructions. The same sharp, unforecast jump can also produce a capability that does harm.
Shevlane and colleagues argued in 2023 that general-purpose training produces harmful capabilities alongside useful ones. Developers must therefore be able to find them, through dangerous capability evaluations that test for abilities such as offensive cyber operations or strong manipulation skills. Ganguli and colleagues made the deployment version of the point. A model's loss is predictable before training, but its specific capabilities, inputs, and outputs are not, which makes the consequences of releasing it hard to anticipate.
The best-documented harms so far get worse with size rather than switching on at a threshold. Perez and colleagues found that larger models more often repeat back a user's preferred answer, a behavior called sycophancy. A public contest, the Inverse Scaling Prize, collected 11 tasks where larger models did worse, and later results on some of those tasks found the trend reversing again at still larger sizes.
Interaction adds risks that no test of a single model reveals. The hide-and-seek seekers found box surfing in a gap in the physics engine that its designers had not seen. A deployed agent that calls tools, hands work to other agents, or talks to customers can find gaps of its own. One might be an instruction that two prompts contradict, another a tool that accepts an unexpected input. AI guardrails limit what such an agent may output. They do not reveal what it will try.
Programmed behavior is written by a developer, learned behavior is fitted from data and improves predictably, and emergent behavior is learned behavior that appears only past a threshold. The three differ most in whether a smaller version of the system predicts them.
Programmed behavior is stated in code or rules, so reading the code predicts it exactly, and it has no threshold to cross. A phone menu built on a programmable Voice API that routes "press 1" to billing is the classic case, and it describes most call automation built before language models.
Learned behavior comes from training data and an objective, and it improves along a steady curve that a smaller version of the system predicts. A spam filter gets better as it sees more labeled email, and evaluation on held-out data measures how much.
Emergent behavior is learned behavior that no one specifies directly and that smaller versions do not predict, because they sit near random on the task. GPT-3's 3-digit arithmetic past about 13 billion parameters is the standard example. It is found only by testing the larger system itself, often after training. A voice agent built on a language model inherits learned and emergent behavior from the model, so the agent has to be tested on what it does, not only on what its configuration says.
Emergent behavior in a production AI agent is managed the same way as any untested behavior: hostile tests before launch, and traces of live conversations after it. Most surprises in a deployed agent are not emergence in the research sense, where an ability appears past a threshold of scale. They are untested states, such as an input, a mix of instructions, or a handoff that the tests never covered. The two look alike in a transcript, and both surface only when the agent meets them.
A practical check separates them: the same conversation is replayed on a smaller model or an earlier version. If only the larger or newer model shows the behavior, it may be emergent in the research sense. If every version does it once the state is reproduced, it is an untested state.
Testing finds more of those states when its scenarios are hostile. On Telnyx's voice AI agents, an assistant test sets a scenario for a test conversation and scores the result against a rubric of named criteria. The documentation's example scenario is a frustrated customer who received a damaged product and escalates if the first answer does not satisfy. Scenarios like that one push an agent into combinations of requests that routine tests never reach.
Live traces show what the tests missed, and each surprising trace can become a new test scenario. With observability turned on, every model call and tool call in a Telnyx AI Assistant conversation goes to the team's Langfuse project, with its inputs and outputs, grouped by conversation.
A Telnyx-managed insight called Agent Instruction Following grades each conversation on how well the assistant followed its system prompt and tool-use instructions, from Excellent to Poor. The documentation lists debugging unexpected assistant behavior as one of its uses. The score catches an agent drifting from its instructions. It does not flag behavior that follows every instruction and still surprises the team, which only reading the traces reveals.
Multi-agent setups need the closest watch, because two assistants with different instructions end up sharing one conversation history. Telnyx's agent handoff passes a conversation, with its full history, from one assistant to a specialist assistant. Each agent in the chain can run a different model. After a handoff, the new assistant responds to messages the previous one wrote, under instructions it never saw. No study we found shows a two-assistant handoff producing coordination like Park's town, but the mechanism is the same: agents responding to each other's messages, so the conversation as a whole is the thing to review.
Any change to that chain, such as a new prompt, model, or voice, can reach a share of live traffic first. Traffic distribution splits live calls between assistant versions, so the change is measured on real conversations before it is promoted. Real callers interrupt, pause mid-thought, and restart sentences, which puts an agent in states its scripted tests never created. Evaluating a voice agent the way real conversations work exposes those states.
Emergent behavior means behavior of a whole system that none of its parts shows on its own, such as a flock's shape arising from birds that each follow a few local rules. In AI, it means a capability that appears only past a threshold of model size, agent interaction, or training time, with no sign of it in the smaller or simpler system.
An emergent ability is a task a large language model can perform that smaller models in the same family cannot, appearing sharply past a size threshold. Wei and colleagues defined it in 2022: "An ability is emergent if it is not present in smaller models but is present in larger models." Three-digit arithmetic in GPT-3, which appeared at about 13 billion parameters, is a standard example.
Large language models show emergent abilities on some tasks, by the standard definition that an ability is absent in smaller models of the family and present in larger ones. How many of those abilities are real thresholds is disputed. Some smooth out under a continuous score, while others stay abrupt until the model's pre-training loss falls below a threshold.
Some emergent abilities of large language models are partly a mirage created by all-or-nothing metrics, and others hold up. Schaeffer and colleagues showed that exact-match and multiple-choice scoring can turn steady improvement into an apparent jump. Their paper does not claim that language models cannot display emergent abilities, and later work found abilities that stay abrupt under continuous metrics.
Emergent AI is not unsafe by default, since most documented emergent abilities, such as arithmetic and reasoning, are useful. The risk is unpredictability. An ability that appears without warning has not been tested, so developers run dangerous capability evaluations on new models, and operators monitor deployed agents for behavior their tests did not cover.
The most common distinction is between weak and strong emergence. Weak emergence is behavior that is unexpected but follows in principle from the system's low-level rules, such as flocking or a language model's emergent abilities. Strong emergence is behavior that could not be deduced from the low-level rules even in principle, a category philosophers still debate.
In-context learning is an emergent property of large language models: the ability to learn a new task from a few examples in the prompt appears in a sudden jump during training. Olsson and colleagues traced the jump to the point where attention heads they call induction heads form, and the GPT-3 paper showed the ability improving greatly with model size.
An AI system shows unexpected behavior when an input, a combination of instructions, or an interaction with another system reaches a state its tests never covered. For a deployed agent, the fastest way to find the cause is a trace of the conversation where it happened: the exact prompt, each model response, and each tool call, in order.
This content was generated with the assistance of AI. Our AI prompt chain workflow is carefully grounded and preferences .gov and .edu citations when available. All content is reviewed by a Telnyx employee to ensure accuracy, relevance, and a high standard of quality.
