GLM-5.3 latency benchmarks across four inference providers

An agent can receive its first streamed token and still be waiting on the answer it needs for its next step. A document review may need the complete extraction before it can write a record. A coding agent may need the full response before deciding which tool to call. For those workflows, time to the completed response matters alongside time to the first token.

We tested GLM-5.3 on Telnyx Inference, Baseten, Together AI, and Fireworks with 60 distinct prompts shared across providers. Telnyx had the lowest median end-to-end (E2E) latency in five of six workloads and the lowest observed p95 E2E in all six. Each provider had only 6-10 answered requests per workload, so the p95 values describe only this sample. They are not an SLA. Together AI started producing answer text sooner at the median in four of six workloads. Those results matter differently depending on whether your application can act on partial output or must wait for a completed response.

Completed-response latency at p95

E2E latency runs from sending a request to receiving its final streamed response. For an agent with dependent steps, the slower completed responses can govern how long the workflow waits, even when the first token arrives quickly. Here is the observed p95 by prompt and output length, with the number of answered requests shown in each cell.

On the longest-input, longer-output profile, Telnyx's observed p95 E2E was 12.80 seconds, compared with 23.18 seconds on Baseten, 26.84 seconds on Fireworks, and 69.64 seconds on Together AI. The measured input was about 88.7k tokens, despite the profile's "100k" shorthand, and median output was 1,200 tokens. Together's p95 is particularly sensitive to a slow request in this small sample and should not be generalized into a provider-wide tail-latency claim.

What a typical completed response looked like

Telnyx had the lowest p50 E2E in five of six profiles. At about 88.7k input and 1,200 output tokens, its p50 E2E was 9.87 seconds, versus 14.67 seconds on Together AI, 20.61 seconds on Fireworks, and 21.81 seconds on Baseten. Together AI led the shorter 10k-input/~100-output profile at 0.81 seconds, versus 1.02 seconds on Telnyx.

These are completed-response times rather than model-quality scores. The providers used the same prompt variants and approximate output budgets, but provider serving configurations and reasoning behavior may still differ.

Generation speed and response completion

If a workflow needs a longer answer, the rate at which output arrives can contribute substantially to E2E latency. The following chart focuses on the three profiles targeting about 1,200 output tokens. It shows the median of per-request estimated output tokens per second, measured after the first non-empty stream delta.

Telnyx's observed medians were 191.7, 230.5, and 232.7 output tokens/second across the 1k-, 10k-, and ~88.7k-input profiles, respectively. This is useful directional context for the E2E result rather than a precise cross-provider throughput win. Telnyx did not provide usable streamed token counts, so its counts came from a separate non-streaming usage probe on the same prompt. The other providers' counts came from streamed usage. A probe can produce a different output from the timed stream, and the calculation does not isolate network time, reasoning time, or decoding speed.

Fast first answer versus fast final answer

For a chat UI, time to the first visible answer can be the most important responsiveness metric. For an agent waiting to act, E2E may matter more. We tracked both, and kept reasoning separate:

• TTFT: time to the first non-empty streamed delta, which may be reasoning rather than answer text.
• Time to first answer (TTFA): time to the first non-whitespace answer-content delta.
• E2E: time until the streamed response finishes.

Together AI had the lowest p50 TTFA in four of six profiles, while Telnyx had the lowest p50 E2E in five. GLM-5.3 can stream reasoning before answer text, so the first reasoning delta and first visible answer are separate events.

How we ran the benchmark

We ran six approximate input/output profiles: 1k, 10k, and "100k" input, each with ~100 and ~1k output targets. Every profile had ten deterministic but distinct prompt variants. Each provider received the same variants. The longest prompts were approximately 88.7k provider-reported input tokens, and the longer-output profiles had a median of 1,200 provider-reported output tokens. Calls streamed with temperature set to 0, and the harness requested reasoning_effort=high from each provider. That request does not establish identical reasoning work or serving configuration across providers.

The same benchmark runner called each provider's public API, keeping the request origin consistent. These measurements include the API path from that runner. They do not isolate model compute time or establish which data center served a request.

There were 240 attempts (60 variants × four providers), of which 230 produced answer text and usable answer timing. We excluded six HTTP failures from Together AI (four 402, one 500, one 503), plus four HTTP-success responses without answer text (two Telnyx, one Together AI, one Fireworks), from answer-latency percentiles. The per-profile charts show successful answered requests only, with counts in the p95 and throughput charts. p95 was computed by linear interpolation over each profile's answered requests. With at most ten per cell, one slow request can move it substantially.

Measure your own workflow

The benchmark is a starting point. Your own prompts and output lengths are the real test. With Telnyx's OpenAI-compatible Inference API, you can time both first answer and completed response in a streamed call:

import os
import time
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["TELNYX_API_KEY"],
    base_url="https://api.telnyx.com/v2/ai/openai",
)

start = time.perf_counter()
first_answer = None
stream = client.chat.completions.create(
    model="zai-org/GLM-5.3",
    messages=[{"role": "user", "content": "Summarize this document for my workflow..."}],
    reasoning_effort="high",
    stream=True,
)

for chunk in stream:
    if not chunk.choices:
        continue
    content = chunk.choices[0].delta.content or ""
    if content.strip() and first_answer is None:
        first_answer = time.perf_counter() - start

e2e = time.perf_counter() - start
print(f"First answer: {first_answer:.2f}s" if first_answer is not None else "No answer text")
print(f"Completed response: {e2e:.2f}s")

Use your real prompts, vary repeated inputs, and record failed or empty responses alongside latency. For agentic applications, measure the step your next action truly waits for: the first useful answer, the final response, or a parsed tool result.

For developers choosing where to run GLM-5.3, the measured distinction is clear: Telnyx had the lowest median completed-response time in five of six workloads and the lowest observed p95 E2E in all six. Together AI had the lowest median first-answer time in four. Inference at Telnyx runs on GPU infrastructure we own and a network we operate, so the path from API to hardware is under our control end to end. Choose the metric your application actually waits for, then try Telnyx Inference with your own prompts and output lengths.