---
title: "Multi-turn LLM benchmark: 3 models, 4 providers"
canonical: https://telnyx.com/resources/multi-turn-llm-benchmark
---
# Multi-turn LLM benchmark: 3 models, 4 providers

[Back to blog](https://telnyx.com/resources)

**Inference**

# Multi-turn LLM benchmark: 3 models, 4 providers

See how three open-weight models performed across Telnyx, Baseten, Together AI, and Fireworks on multi-turn quality, reliability, and latency.

By [Sonam Gupta](https://telnyx.com/authors/sonam-gupta)

Single-request benchmarks can tell you how quickly an inference provider begins or finishes one response. They do not tell you what happens when an agent must carry requirements through a sequence of corrections, preserve state in the prompt history, and wait for each answer before it can take the next step. That gap matters because latency compounds in multi-turn systems. A fast first token can make an interface feel responsive, but the next agent step cannot begin until the current response is complete. Reliability compounds too: one failed request can interrupt the entire workflow.

We built a first-party multi-turn dataset to measure that behavior across three frontier open-weight model families and four hosted inference providers. The benchmark attempted **540 conversations spanning 2,160 expected model turns** across Telnyx, Baseten, Together AI, and Fireworks. The clearest result was conversation completion speed. **Telnyx recorded the lowest median and observed p95 total response time for GLM 5.3 Flash, GLM 5.3, and Kimi K3.** Against the next-fastest provider for each model, Telnyx's median conversation time was 15% to 34% lower, while observed p95 conversation time was 7% to 18% lower.

That is not the same as winning every metric. Together AI usually began visible answers fastest for the GLM models, quality scores were tightly grouped, and reliability varied by model-provider pair. The useful conclusion is narrower: in this dataset, Telnyx was strongest at completing the full sequence quickly while maintaining competitive quality and reliability.

**Benchmark takeaway:** For a sequential agent, measure the time to finish the conversation, not only the time to start the first response. 

## Multi-turn LLM benchmark results

Each cell contained 15 scenarios repeated three times, for 45 attempted conversations per provider-model pair. Conversation latency is the sum of model response time across the fixed multi-turn workflow. It excludes simulated user pauses.

| Model         | Telnyx p50 vs. next              | Telnyx p95 vs. next              |
| ------------- | -------------------------------- | -------------------------------- |
| GLM 5.3 Flash | 8.26s vs. 11.00s, **25% lower**  | 35.62s vs. 42.72s, **17% lower** |
| GLM 5.3       | 13.48s vs. 15.85s, **15% lower** | 41.33s vs. 50.65s, **18% lower** |
| Kimi K3       | 10.49s vs. 15.95s, **34% lower** | 46.40s vs. 49.66s, **7% lower**  |

“Next” means the next-fastest provider observed for that model and percentile. These results describe this workload and sample. They are not provider SLAs.

## Median conversation latency

Telnyx had the lowest median total response time for all three model families. The gap was smallest for GLM 5.3 and largest for Kimi K3.

This metric is especially useful for dependent agent steps. If each turn needs the previous answer, lower total response time lets the complete workflow advance sooner.

## Tail latency across a complete workflow

Median latency describes a typical successful conversation. Tail latency shows what users may encounter toward the slower end of the observed sample.

Telnyx also recorded the lowest observed p95 conversation time for each model family. The largest absolute gap appeared with GLM 5.3: 41.33 seconds on Telnyx, compared with 50.65 seconds on Baseten, 118.83 seconds on Fireworks, and 181.20 seconds on Together AI.

Observed p95 values were calculated only from successful conversations. Each provider-model cell had 40 to 45 successful conversations, so these percentiles should be read as sample observations rather than guarantees.

## Quality stayed close across providers

Speed only matters if the workflow still produces a useful result. We therefore evaluated final conversations with deterministic checks and one blind LLM judge. The judge scored task completion, correctness, state tracking, correction handling, and final-answer usability on a **1-to-5 scale**. Mean blind quality scores ranged from **4.79 to 5.00**, which is too narrow a spread to support sweeping quality claims from one judge.

The highest mean quality score depended on the model family:

* Together AI led GLM 5.3 Flash at 5.00/5.
* Telnyx led GLM 5.3 at 4.95/5.
* Baseten led Kimi K3 at 4.96/5.

Judge-completion results were similarly close. Telnyx received a completed-task decision for every judged conversation across all three models. Several competing provider-model pairs also reached 100%. The deterministic checks added a second layer of evidence. They tested requirements such as valid JSON, required fields, retained constraints, and removal of superseded instructions. Pass rates across provider-model pairs ranged from 76.2% to 81.5%.

## Reliability depends on the pairing

Across all three models, Telnyx and Baseten each completed 134 of 135 attempted conversations. Fireworks completed 133, while Together AI completed 129.

| Provider    | Completed | Success rate |
| ----------- | --------- | ------------ |
| Telnyx      | 134/135   | 99.3%        |
| Baseten     | 134/135   | 99.3%        |
| Fireworks   | 133/135   | 98.5%        |
| Together AI | 129/135   | 95.6%        |

The aggregate hides model-specific behavior. Baseten, Together AI, and Fireworks completed all 45 GLM 5.3 Flash conversations. Telnyx completed 44\. Telnyx and Baseten completed all 45 GLM 5.3 conversations. For Kimi K3, Telnyx and Fireworks completed all 45, while Together AI completed 40\. Observed failures included HTTP 500 and 503 responses, stream errors, and responses with no visible answer content. Failed conversations remain reliability failures and are excluded from latency percentiles.

## First answer speed tells a different story

We measured both first model activity and first visible answer text:

* **TTFT** measures when the model first produces something, including hidden reasoning.
* **TTFA** measures when visible answer text begins.
* **E2E** measures when the complete streamed response finishes.

Together AI had the lowest median TTFA for GLM 5.3 Flash at 0.42 seconds and GLM 5.3 at 0.46 seconds. Telnyx had the lowest median TTFA for Kimi K3 at 0.54 seconds. This is why provider selection should match the application. A chat interface may prioritize the first visible answer. An agent that cannot move forward until a response is complete should weigh end-to-end and conversation-level latency more heavily. For a deeper explanation, see [TTFT vs. E2E](https://telnyx.com/resources/inference-benchmark-ttft-vs-e2e).

## What we tested

The benchmark used 15 synthetic scenarios created for this study. They were not copied from an existing public leaderboard:

* **5 developer workflows:** cursor pagination, zero-downtime migration, webhook verification, JSON normalization, and distributed rate limiting.
* **5 business workflows:** support triage, launch planning, policy FAQs, account planning, and incident communications.
* **5 structured workflows:** workshop scheduling, executive editing, candidate evaluation, dependency planning, and invoice extraction.

Each scenario established an initial task, then introduced corrections, superseded requirements, additional constraints, or a strict final format. Every request explicitly included the visible conversation history. The runner did not rely on implicit provider memory.

The compared model families were [GLM 5.3](https://z.ai/blog/glm-5.3), [GLM 5.3 Flash](https://autoclaw.z.ai/blog/model/glm-5.3-flash/), and [Kimi K3](https://www.kimi.com/en/blog/kimi-k3). The [Telnyx model catalog](https://developers.telnyx.com/docs/inference/models) lists their current Telnyx identifiers and context limits.

## How the benchmark worked

| Benchmark element       | Configuration                           |
| ----------------------- | --------------------------------------- |
| Providers               | Telnyx, Baseten, Together AI, Fireworks |
| Scenarios               | 15 fixed multi-turn workflows           |
| Repeats                 | 3 per provider-model-scenario cell      |
| Attempted conversations | 540                                     |
| Expected model turns    | 2,160                                   |
| Reasoning effort        | low requested for every candidate       |
| Temperature             | 0                                       |
| Output limit            | 4,096 tokens per turn                   |
| Blind judge             | DeepSeek V4.1 Flash via Telnyx          |

A Telnyx [stateful actor](https://developers.telnyx.com/docs/edge-compute/stateful-actors) orchestrated the benchmark and used the same execution path to call every provider's public API. The actor persisted progress, while the inference request itself carried the explicit conversation history. Developers can use [Telnyx Functions](https://telnyx.com/products/functions) for similar event-driven orchestration patterns.

The blind judge evaluated the first repeat when it completed successfully, producing 14 or 15 judged conversations per provider-model pair. Provider identity was excluded from the judge input. Deterministic checks and latency metrics used all successful repeats. Failed requests were counted in reliability but not latency percentiles.

## What this benchmark does not prove

This benchmark is a workload-specific comparison, not a universal provider or model ranking.

* The scenarios are synthetic and conversation-only. They do not invoke live tools, retrieval systems, or external APIs.
* Quality used one automated judge. Small score differences are directional, not decisive.
* Hosted providers may use different serving stacks, precision, quantization, or routing that their public APIs do not expose.
* The same `reasoning_effort="low"` setting was requested everywhere, but provider implementations may interpret it differently.
* Observed p95 values come from 40 to 45 successful conversations per cell and are not SLAs.
* Network conditions and provider load can change over time.

These limits are why the article reports quality, completion, reliability, and latency separately instead of producing a weighted overall winner.

## Choosing an inference provider for multi-turn agents

The benchmark suggests three practical selection rules:

1. **Measure the complete loop.** First-answer latency and completed-conversation latency can identify different leaders.
2. **Test the provider-model pair.** Reliability and latency varied by model on the same provider.
3. **Bring your own workflows.** Public benchmarks are useful filters, but corrections, formats, and tool patterns from your application are more representative.

In this dataset, Telnyx was the strongest starting point when the priority was completing a sequential workflow quickly. It led median and observed p95 conversation latency for every tested model family, while maintaining a 99.3% aggregate conversation success rate and competitive blind quality.

For model selection within Telnyx, GLM 5.3 Flash offered the lowest median conversation time, GLM 5.3 produced Telnyx's highest blind quality score, and Kimi K3 combined 100% request reliability with the fastest first visible answer in its provider comparison. Our [model selection guide](https://telnyx.com/resources/efficient-frontier-model-selection) provides more context for matching models to workloads.

## Build and measure your own workflow

[Telnyx Inference](https://telnyx.com/products/inference) exposes frontier open-weight models through an OpenAI-compatible API. Preserve the explicit message history, measure both the beginning and end of every streamed response, and evaluate the final task rather than only individual turns. Start with the [Inference quickstart](https://developers.telnyx.com/docs/inference/getting-started), select a model from the [model catalog](https://developers.telnyx.com/docs/inference/models), and test the scenarios that define success for your application. For a production setting where latency and conversational continuity matter together, explore [AI contact centers](https://telnyx.com/solutions/contact-center).

## Resources

* [Inference quickstart](https://developers.telnyx.com/docs/inference/getting-started)
* [Available models](https://developers.telnyx.com/docs/inference/models)
* [Stateful actors](https://developers.telnyx.com/docs/edge-compute/stateful-actors)
* [MT-Bench overview](https://telnyx.com/resources/what-is-mt-bench)
* [GLM 5.3 benchmarks](https://telnyx.com/resources/glm-5-3-latency-benchmarks-provider)
* [GLM Flash benchmarks](https://telnyx.com/resources/glm-5-3-flash-benchmarks)
* [Latency metric guide](https://telnyx.com/resources/inference-benchmark-ttft-vs-e2e)
* [Model selection guide](https://telnyx.com/resources/efficient-frontier-model-selection)

Share on Social

Sonam Gupta

Developer Advocate

Sonam is a San Francisco-based developer advocate, originally from India. She has completed 2 Master's Degrees and her PhD in Data Science from the Harrisburg University of Science & Technology. Previously, Sonam worked for the startups Ozmosi and aiXplain. In her free time, you 

will find Sonam dancing, writing, and exploring local coffee shops.

\[...\]

## Jump to:

[Multi-turn LLM benchmark results](#multi-turn-llm-benchmark-results)[Median conversation latency](#median-conversation-latency)[Tail latency across a complete workflow](#tail-latency-across-a-complete-workflow)[Quality stayed close across providers](#quality-stayed-close-across-providers)[Reliability depends on the pairing](#reliability-depends-on-the-pairing)[First answer speed tells a different story](#first-answer-speed-tells-a-different-story)[What we tested](#what-we-tested)[How the benchmark worked](#how-the-benchmark-worked)[What this benchmark does not prove](#what-this-benchmark-does-not-prove)[Choosing an inference provider for multi-turn agents](#choosing-an-inference-provider-for-multi-turn-agents)[Build and measure your own workflow](#build-and-measure-your-own-workflow)[Resources](#resources)

## Sign up for emails of our latest articles and news

[Start building](https://telnyx.com/sign-up)

Related articles

* [**Inference**AI inference hardware: How to choose chips and placement![Eli Mogul](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/Eli_Mogul_1de9939dcb.png?width=96&format=webp)By Eli Mogul20, Oct 2026](https://telnyx.com/resources/ai-inference-hardware)
* [**Inference**Inference optimization: cut LLM latency and cost![Eli Mogul](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/Eli_Mogul_1de9939dcb.png?width=96&format=webp)By Eli Mogul16, Oct 2026](https://telnyx.com/resources/inference-optimization)
* [**Inference**GLM 5.3 Flash benchmark: Telnyx vs Modal![Sonam Gupta, PhD](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/sonam_gupta_phd_2ae3a78101.png?width=96&format=webp)By Sonam Gupta, PhD15, Oct 2026](https://telnyx.com/resources/glm53-flash-benchmark)
* [**Verify API**Caller ID name (CNAM): how your business name reaches the phone![Deniz Yakışıklı](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/Deniz_Yakisikli_6b7b420ed4.png?width=96&format=webp)By Deniz Yakışıklı19, Oct 2026](https://telnyx.com/resources/caller-id-name-cnam)
* [**Voice**What is call control? Direct, programmable control over every call![Serhii](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/_DSC8799-6.jpg?width=96&format=webp)By Serhii Omelchenko16, Oct 2026](https://telnyx.com/resources/what-is-call-control)
* [**Voice**The best Vapi alternatives for low latency voice AI in 2026![Osman Husain Telnyx](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/Osman%20Husain%20headshot?width=96&format=webp)By Osman Husain15, Oct 2026](https://telnyx.com/resources/vapi-alternative)

### Ask AI

* [![GPT](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/chatgpt_icon-ea55e09d.jpeg?width=92&format=webp)](https://chat.openai.com/?q=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.)
* [![Claude](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/claude_icon-c7c8eef3.webp?width=92&format=webp)](https://claude.ai/new?q=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.)
* [![Perplexity](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/perplexity_icon-cd5b74f1.webp?width=92&format=webp)](https://www.perplexity.ai/search?q=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.)
* [![Gemini](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/gemini_icon-4ff1cbf4.webp?width=92&format=webp)](https://www.google.com/search?udm=50&aep=11&q=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.)
* [![Grok](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/grok_icon-e588e5bd.webp?width=92&format=webp)](https://x.com/i/grok?text=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.)