Multi-turn LLM benchmark: 3 models, 4 providers

Single-request benchmarks can tell you how quickly an inference provider begins or finishes one response. They do not tell you what happens when an agent must carry requirements through a sequence of corrections, preserve state in the prompt history, and wait for each answer before it can take the next step. That gap matters because latency compounds in multi-turn systems. A fast first token can make an interface feel responsive, but the next agent step cannot begin until the current response is complete. Reliability compounds too: one failed request can interrupt the entire workflow.

We built a first-party multi-turn dataset to measure that behavior across three frontier open-weight model families and four hosted inference providers. The benchmark attempted 540 conversations spanning 2,160 expected model turns across Telnyx, Baseten, Together AI, and Fireworks. The clearest result was conversation completion speed. Telnyx recorded the lowest median and observed p95 total response time for GLM 5.3 Flash, GLM 5.3, and Kimi K3. Against the next-fastest provider for each model, Telnyx's median conversation time was 15% to 34% lower, while observed p95 conversation time was 7% to 18% lower.

That is not the same as winning every metric. Together AI usually began visible answers fastest for the GLM models, quality scores were tightly grouped, and reliability varied by model-provider pair. The useful conclusion is narrower: in this dataset, Telnyx was strongest at completing the full sequence quickly while maintaining competitive quality and reliability.

Multi-turn LLM benchmark results

Each cell contained 15 scenarios repeated three times, for 45 attempted conversations per provider-model pair. Conversation latency is the sum of model response time across the fixed multi-turn workflow. It excludes simulated user pauses.

“Next” means the next-fastest provider observed for that model and percentile. These results describe this workload and sample. They are not provider SLAs.

Median conversation latency

Telnyx had the lowest median total response time for all three model families. The gap was smallest for GLM 5.3 and largest for Kimi K3.

This metric is especially useful for dependent agent steps. If each turn needs the previous answer, lower total response time lets the complete workflow advance sooner.

Tail latency across a complete workflow

Median latency describes a typical successful conversation. Tail latency shows what users may encounter toward the slower end of the observed sample.

Telnyx also recorded the lowest observed p95 conversation time for each model family. The largest absolute gap appeared with GLM 5.3: 41.33 seconds on Telnyx, compared with 50.65 seconds on Baseten, 118.83 seconds on Fireworks, and 181.20 seconds on Together AI.

Observed p95 values were calculated only from successful conversations. Each provider-model cell had 40 to 45 successful conversations, so these percentiles should be read as sample observations rather than guarantees.

Quality stayed close across providers

Speed only matters if the workflow still produces a useful result. We therefore evaluated final conversations with deterministic checks and one blind LLM judge. The judge scored task completion, correctness, state tracking, correction handling, and final-answer usability on a 1-to-5 scale. Mean blind quality scores ranged from 4.79 to 5.00, which is too narrow a spread to support sweeping quality claims from one judge.

The highest mean quality score depended on the model family:

• Together AI led GLM 5.3 Flash at 5.00/5.
• Telnyx led GLM 5.3 at 4.95/5.
• Baseten led Kimi K3 at 4.96/5.

Judge-completion results were similarly close. Telnyx received a completed-task decision for every judged conversation across all three models. Several competing provider-model pairs also reached 100%. The deterministic checks added a second layer of evidence. They tested requirements such as valid JSON, required fields, retained constraints, and removal of superseded instructions. Pass rates across provider-model pairs ranged from 76.2% to 81.5%.

Reliability depends on the pairing

Across all three models, Telnyx and Baseten each completed 134 of 135 attempted conversations. Fireworks completed 133, while Together AI completed 129.

The aggregate hides model-specific behavior. Baseten, Together AI, and Fireworks completed all 45 GLM 5.3 Flash conversations. Telnyx completed 44. Telnyx and Baseten completed all 45 GLM 5.3 conversations. For Kimi K3, Telnyx and Fireworks completed all 45, while Together AI completed 40. Observed failures included HTTP 500 and 503 responses, stream errors, and responses with no visible answer content. Failed conversations remain reliability failures and are excluded from latency percentiles.

First answer speed tells a different story

We measured both first model activity and first visible answer text:

• TTFT measures when the model first produces something, including hidden reasoning.
• TTFA measures when visible answer text begins.
• E2E measures when the complete streamed response finishes.

Together AI had the lowest median TTFA for GLM 5.3 Flash at 0.42 seconds and GLM 5.3 at 0.46 seconds. Telnyx had the lowest median TTFA for Kimi K3 at 0.54 seconds. This is why provider selection should match the application. A chat interface may prioritize the first visible answer. An agent that cannot move forward until a response is complete should weigh end-to-end and conversation-level latency more heavily. For a deeper explanation, see TTFT vs. E2E.

What we tested

The benchmark used 15 synthetic scenarios created for this study. They were not copied from an existing public leaderboard:

• 5 developer workflows: cursor pagination, zero-downtime migration, webhook verification, JSON normalization, and distributed rate limiting.
• 5 business workflows: support triage, launch planning, policy FAQs, account planning, and incident communications.
• 5 structured workflows: workshop scheduling, executive editing, candidate evaluation, dependency planning, and invoice extraction.

Each scenario established an initial task, then introduced corrections, superseded requirements, additional constraints, or a strict final format. Every request explicitly included the visible conversation history. The runner did not rely on implicit provider memory.

The compared model families were GLM 5.3, GLM 5.3 Flash, and Kimi K3. The Telnyx model catalog lists their current Telnyx identifiers and context limits.

How the benchmark worked

A Telnyx stateful actor orchestrated the benchmark and used the same execution path to call every provider's public API. The actor persisted progress, while the inference request itself carried the explicit conversation history. Developers can use Telnyx Functions for similar event-driven orchestration patterns.

The blind judge evaluated the first repeat when it completed successfully, producing 14 or 15 judged conversations per provider-model pair. Provider identity was excluded from the judge input. Deterministic checks and latency metrics used all successful repeats. Failed requests were counted in reliability but not latency percentiles.

What this benchmark does not prove

This benchmark is a workload-specific comparison, not a universal provider or model ranking.

• The scenarios are synthetic and conversation-only. They do not invoke live tools, retrieval systems, or external APIs.
• Quality used one automated judge. Small score differences are directional, not decisive.
• Hosted providers may use different serving stacks, precision, quantization, or routing that their public APIs do not expose.
• The same reasoning_effort="low" setting was requested everywhere, but provider implementations may interpret it differently.
• Observed p95 values come from 40 to 45 successful conversations per cell and are not SLAs.
• Network conditions and provider load can change over time.

These limits are why the article reports quality, completion, reliability, and latency separately instead of producing a weighted overall winner.

Choosing an inference provider for multi-turn agents

The benchmark suggests three practical selection rules:

1. Measure the complete loop. First-answer latency and completed-conversation latency can identify different leaders.
2. Test the provider-model pair. Reliability and latency varied by model on the same provider.
3. Bring your own workflows. Public benchmarks are useful filters, but corrections, formats, and tool patterns from your application are more representative.

In this dataset, Telnyx was the strongest starting point when the priority was completing a sequential workflow quickly. It led median and observed p95 conversation latency for every tested model family, while maintaining a 99.3% aggregate conversation success rate and competitive blind quality.

For model selection within Telnyx, GLM 5.3 Flash offered the lowest median conversation time, GLM 5.3 produced Telnyx's highest blind quality score, and Kimi K3 combined 100% request reliability with the fastest first visible answer in its provider comparison. Our model selection guide provides more context for matching models to workloads.

Build and measure your own workflow

Telnyx Inference exposes frontier open-weight models through an OpenAI-compatible API. Preserve the explicit message history, measure both the beginning and end of every streamed response, and evaluate the final task rather than only individual turns. Start with the Inference quickstart, select a model from the model catalog, and test the scenarios that define success for your application. For a production setting where latency and conversational continuity matter together, explore AI contact centers.

Resources

• Inference quickstart
• Available models
• Stateful actors
• MT-Bench overview
• GLM 5.3 benchmarks
• GLM Flash benchmarks
• Latency metric guide
• Model selection guide