AI Assistants Can Now Use More Than One Transcription Model

Telnyx AI Assistants can now run more than one transcription model on the same call. Two new modes, Fallback and Language Booster, keep calls going when a transcription model fails and pick the most accurate transcript for every turn.

What's new

• Fallback mode: Add up to 3 backup transcription models in priority order. If the model in use fails at the start of a call or mid-call, the next one takes over and the call keeps going. Set through transcription.fallback_models.
• Language booster: A second model transcribes alongside the main one, and a rule decides which transcript the assistant uses. best_turn (default) picks the better transcript each turn, best_engine locks onto the stronger model for the call, and merge_words merges Arabic and English spoken in the same sentence. Set through transcription.challenger.

An assistant runs one mode at a time, and each backup model or booster carries its own language and provider settings.

Why it matters

A voice agent is only as good as the transcript it reasons on. Until now you chose one transcription model for every call: if it failed, the call degraded with it, and if callers switched languages mid-sentence, accuracy dropped. With both modes on one control plane, you compose the right pair of models for each deployment instead of adding another vendor or another integration. In our internal tests, deepgram/flux with deepgram/nova-3 as the booster cut English word errors from 5.7% to 4.6%, and the Basira and Cohere merge_words pair cut errors from 27.7% to 22.4%.

Example use cases

• Support lines and outbound engagement: chain up to 3 fallback models so a transcription failure never ends a call.
• Contact centers serving Arabic and English callers: run merge_words so callers who switch between the two mid-sentence are transcribed accurately end to end.
• Accuracy tuning: use best_engine to let two models transcribe the first turns, then continue the call on whichever scores higher.

Getting started

1. In the Mission Control portal, open AI, then Assistants, select your assistant, and go to the Voice tab. Under Transcription, add models in Additional Transcription Models.
2. Or set the modes through the API when creating or updating an assistant:

"transcription": {
  "model": "deepgram/flux",
  "language": "en",
  "fallback_models": [
    { "model": "deepgram/nova-3" },
    { "model": "soniox/stt-rt-v5" }
  ]
}

Learn more in the transcription settings docs or on the Voice AI Agents page.