--- title: "Beam Search Algorithm: How It Works in NLP" canonical: https://telnyx.com/learn-ai/beam-search-algorithm --- # Beam Search Algorithm: How It Works in NLP [Start building](https://telnyx.com/sign-up) ### Ask AI * [![GPT](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/chatgpt_icon-ea55e09d.jpeg?width=92&format=webp)](https://chat.openai.com/?q=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.) * [![Claude](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/claude_icon-c7c8eef3.webp?width=92&format=webp)](https://claude.ai/new?q=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.) * [![Perplexity](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/perplexity_icon-cd5b74f1.webp?width=92&format=webp)](https://www.perplexity.ai/search?q=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.) * [![Gemini](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/gemini_icon-4ff1cbf4.webp?width=92&format=webp)](https://www.google.com/search?udm=50&aep=11&q=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.) * [![Grok](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/grok_icon-e588e5bd.webp?width=92&format=webp)](https://x.com/i/grok?text=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.) Beam Search Algorithm: How It Works in NLP [Back to Glossary](https://telnyx.com/learn-ai) # Beam Search Algorithm: How It Works in NLP Beam search keeps several promising sequences during decoding. Learn how beam width works, see an example, and compare beam search with greedy decoding. Editor: [Emily Bowen](https://www.linkedin.com/in/emilywynne/) _Updated August 2026_ ## What is beam search? Beam search is an approximate decoding algorithm that keeps a limited number of the most promising partial solutions at each step. In language generation, it expands several candidate token sequences instead of committing to only the highest-scoring next token. The number kept is the beam width. Beam width is a measurable control: a width of 1 is greedy decoding, while larger values retain more candidates and require more computation. [Hugging Face documents this relationship](https://huggingface.co/docs/transformers/generation%5Fstrategies) for transformer generation. This content was generated with the assistance of AI. Our AI [prompt chain](https://telnyx.com/learn-ai/promptchain-ai) workflow is carefully [grounded](https://telnyx.com/learn-ai/grounding-ai) and preferences .gov and .edu citations when available. **All content is reviewed by a Telnyx employee to ensure accuracy, relevance, and a high standard of quality.** ## Sign up and start building. [Sign Up](https://telnyx.com/sign-up)[Contact Us](https://telnyx.com/contact-us) ## How does beam search work? At each decoding step, beam search expands every sequence currently in the beam. It scores the resulting candidates, then retains only the top `k` sequences. The process repeats until a stop condition, such as an end token or maximum length. [Published beam-search research](https://arxiv.org/abs/1702.01806) describes this core procedure and the scoring choices used to rank candidates. With a beam width of 3, the algorithm keeps three partial sequences after every step. A wider beam explores more alternatives, but requires more computation and can still miss the globally best sequence because it prunes candidates along the way. ## Beam search example Imagine a model starts a translation with three likely first tokens: `The`, `A`, and `This`. It expands each one at the next step, scores all resulting two-token sequences, and retains the three highest-scoring sequences. The best final sentence can come from a second-ranked first token, which greedy decoding would have discarded. ### Beam search versus greedy decoding | Method | What it keeps | Main tradeoff | | --------------- | ----------------------------------------- | --------------------------------------------- | | Greedy decoding | One best next choice | Fast, but can miss better full sequences | | Beam search | Top k partial sequences | Better search coverage at higher compute cost | | Sampling | Randomized candidates from a distribution | More variety, less deterministic output | ## What is beam width? Beam width is the maximum number of candidate sequences retained per step. A width of 1 is greedy decoding. Increasing the width allows more alternatives to survive, but it increases memory and compute use. Do not assume a larger beam always produces a better output. In text generation, wider beams can favor generic or overly short sequences unless the scoring method includes length normalization or other constraints. ## Where is beam search used? Beam search is common in sequence-to-sequence tasks such as machine translation, speech recognition, summarization, and structured prediction. It is helpful when the model scores one step at a time but the system needs a strong full sequence. It is not the default answer for every generative task. Systems that need diverse outputs often use sampling or diversification methods, while systems with strict constraints may use constrained decoding. ## What are the limitations of beam search? Beam search is approximate, not exhaustive. It can prune the sequence that would eventually become best, and its output quality depends on the model’s score calibration. Its computational cost grows with beam width and vocabulary expansion. Evaluate decoding choices with representative inputs and task-specific metrics. A gain in likelihood is not necessarily a gain in usefulness, factuality, or diversity. ## How should you choose a beam width? Choose beam width by measuring the task, not by assuming a larger value is safer. In [one neural machine translation study](https://aclanthology.org/D18-1342/), beam sizes larger than 5 reduced translation quality; the authors' rescoring method improved results by 2.0 BLEU over the tested length-normalization heuristic. That is evidence for task-specific tuning, not a universal beam-width limit. Begin with a small set of widths, such as 1, 2, 4, and 8\. Compare output quality, latency, token cost, and failure modes on a fixed evaluation set. Beam width 1 gives greedy decoding. Higher values add candidate paths and increase the number of model scores that must be processed. At some point, the added compute may produce almost no improvement, or it may make the output more repetitive. Keep the smallest width that meets the task target. ## What is length normalization in beam search? Sequence probabilities are multiplied across tokens, so longer sequences can receive lower total scores even when they are sensible. Length normalization adjusts scoring so that the decoder does not favor a short sequence only because it contains fewer probability terms. The exact method depends on the model and library. Document the scoring rule with the beam width. Without that detail, two systems described as using beam search can produce very different output behavior. ## How does beam search differ from sampling? Beam search usually aims for high-scoring deterministic sequences. Sampling intentionally introduces randomness by choosing among candidates according to the model's probability distribution. It can produce more varied outputs, but the run-to-run result can differ. For generation systems, test both approaches against the job. A structured extraction task may need repeatable decoding. A creative writing interface may value controlled diversity. The [LLM API guide](https://telnyx.com/resources/llm-api) explains the implementation context for calling a language model, while the [hard versus soft token guide](https://telnyx.com/resources/hard-token-vs-soft-token) helps clarify why decoding happens at the token level. ## Related concepts Beam search belongs to a broader set of model-inference choices. Read [machine learning inference](https://telnyx.com/resources/machine-learning-inference) for the production side of running models, and [inference latency](https://telnyx.com/resources/inference-latency) before treating a decoding-quality improvement as free. The [training versus inference guide](https://telnyx.com/resources/ai-training-vs-inference) explains where decoding fits after model training is complete. ## Practical applications of beam search Neural machine translation is the clearest documented application in the sources used here. [Freitag and Al-Onaizan](https://arxiv.org/abs/1702.01806) evaluated beam-search strategies for translation and compared how candidate pruning and scoring affected decoded sentences. [Yang, Huang, and Ma](https://aclanthology.org/D18-1342/) later examined rescoring and stopping criteria on Chinese-to-English translation. Both studies treat beam width, candidate scoring, and stopping behavior as parts of one decoding design. That is the practical lesson: benchmark the full configuration on the target task instead of selecting a width in isolation. ## A practical decoding evaluation Build a small evaluation set that includes short prompts, long prompts, repeated phrases, rare terms, and examples where a wrong token has a high cost. Run the same inputs with each candidate beam width and scoring rule. Record output quality, generation time, and the number of tokens processed. Review the worst outputs by hand. An average metric can hide a pattern where the decoder produces fluent text but drops a required identifier, skips a negation, or ends too early. Those are the errors that should shape the decoding choice. ## Frequently asked questions ### Is beam search a greedy algorithm? Beam search is greedy in the sense that it prunes candidates at every step, but it is less myopic than standard greedy decoding. Greedy decoding keeps one candidate, while beam search keeps a fixed number of the highest-scoring partial candidates. ### What happens when beam width equals 1? Beam search with a width of 1 is equivalent to greedy decoding. The system chooses the highest-scoring next token at each step and never keeps an alternative path alive. ### Does beam search guarantee the best result? No. It explores only a limited subset of all possible sequences. It can produce a strong result efficiently, but an optimal search would need to consider many more paths and may be impractical for large vocabularies or long sequences. ## Sources 1. [Hugging Face generation strategies](https://huggingface.co/docs/transformers/generation%5Fstrategies) 2. [Beam search strategies](https://arxiv.org/abs/1702.01806) 3. [Beam search curse study](https://aclanthology.org/D18-1342/) Share on Social ## Jump to: [What is beam search?](#what-is-beam-search)[How does beam search work?](#how-does-beam-search-work)[Beam search example](#beam-search-example)[What is beam width?](#what-is-beam-width)[Where is beam search used?](#where-is-beam-search-used)[What are the limitations of beam search?](#what-are-the-limitations-of-beam-search)[How should you choose a beam width?](#how-should-you-choose-a-beam-width)[What is length normalization in beam search?](#what-is-length-normalization-in-beam-search)[How does beam search differ from sampling?](#how-does-beam-search-differ-from-sampling)[Related concepts](#related-concepts)[Practical applications of beam search](#practical-applications-of-beam-search)[A practical decoding evaluation](#a-practical-decoding-evaluation)[Frequently asked questions](#frequently-asked-questions)[Sources](#sources) ## Sign up for emails of our latest articles and news