---
title: Run Telnyx Voice AI Assistants with Any OpenAI-Compatible LLM
canonical: https://telnyx.com/resources/run-telnyx-voice-ai-assistants-with-any-open-ai-compatible-llm
---
# Run Telnyx Voice AI Assistants with Any OpenAI-Compatible LLM

[Back to blog](https://telnyx.com/resources)

**Conversational AI**

# Run Telnyx Voice AI Assistants with Any OpenAI-Compatible LLM

Use your own OpenAI-compatible model with Telnyx Voice AI. Bedrock, Azure, or self-hosted LLMs now supported.

By [Abhishek Sharma](https://telnyx.com/authors/abhishek-sharma)

Most voice AI platforms lock you into their inference stack. If you already run models on Bedrock or your own GPUs, integrating them usually means rebuilding the whole telephony layer from scratch.

We removed that constraint. You can now run [Telnyx Voice AI Assistants](https://telnyx.com/products/voice-ai-agents?utm%5Fsource=telnyx%5Fblog&utm%5Fmedium=internal%5Flink&utm%5Fcampaign=202510%5FTOFU%5FAI%5FAgent%5FTL%5FGLOBAL&utm%5Fcontent=run-telnyx-voice-ai-assistants-with-any-open-ai-compatible-llm) on any OpenAI-compatible model endpoint without losing carrier-grade voice performance.

## What’s now supported

You can point your AI Assistant at:

* **AWS Bedrock** \- Use Anthropic, Meta, Cohere, or Mistral models under your existing AWS contract
* **Azure OpenAI Service** \- Route to Microsoft-hosted GPT models where regional or compliance rules apply
* **Self-hosted inference servers** \- Run vLLM, sglang, or TGI on your own GPUs
* **Custom fine-tuned models** \- Use models trained on proprietary data such as support logs, product catalogs, or internal documentation
* **Any OpenAI-compatible endpoint** \- Platforms like Baseten, Replicate, or together.ai work out of the box

The only requirement: your endpoint must implement the OpenAI Chat Completions API. If it accepts `/v1/chat/completions` requests, it will work.

## How it works

In the [Mission Control Portal](https://portal.telnyx.com), create or edit an AI Assistant. Under the **Agent** tab, enable **Use Custom LLM** and provide:

1. **Base URL** \- The public endpoint for your inference server
2. **API Key** \- Stored securely as an Integration Secret
3. **Model Name** \- Auto-populated if your endpoint supports `/models`, otherwise entered manually

Telnyx validates the connection before saving. Once configured, your assistant routes LLM calls to your endpoint instead of Telnyx’s GPU clusters. Voice synthesis, speech recognition, and call control still run on Telnyx infrastructure.

You can test immediately in the portal or deploy to production numbers.

## The latency trade-off

Our default architecture delivers sub-second round-trip time by running inference on GPUs co-located with our telephony core. Using an external LLM adds network hops.

If your inference server is near a Telnyx PoP (for example, us-east-1), expect an additional 20-50 ms. Remote or variable endpoints may add 100-300 ms per turn. That’s still faster than most stitched-together voice stacks but no longer guaranteed sub-200 ms.

When response speed is critical, use Telnyx’s built-in inference.

When model control, compliance, or cost take priority, bring your own endpoint.

## When this matters

This option fits when:

* You have existing Bedrock, Azure, or GCP credits and want to use them for inference
* Inference must stay within a specific region for compliance
* Your models are fine-tuned on proprietary data that can’t leave your environment
* You’re running high-volume workloads on self-managed GPU infrastructure

If none of those apply, Telnyx’s built-in LLM library (Llama, Mistral, Gemini, GPT-4) remains the fastest path.

## Availability

Custom LLM support is available now for all Telnyx AI Assistant users.

Documentation: [developers.telnyx.com/docs/inference/ai-assistants/custom-llm](https://developers.telnyx.com/docs/inference/ai-assistants/custom-llm)

Whether you’re optimizing for latency, cost, or compliance, you can now choose exactly where your model runs and keep everything else running on Telnyx.

_Building Voice AI assistants with Telnyx? Join our [subreddit](https://www.reddit.com/r/Telnyx/)._

Share on Social

Abhishek Sharma

Sr Technical Product Marketing Manager

Senior Technical Product Marketing Manager

## Jump to:

[What’s now supported](#whats-now-supported)[How it works](#how-it-works)[The latency trade-off](#the-latency-trade-off)[When this matters](#when-this-matters)[Availability](#availability)

## Sign up for emails of our latest articles and news

[Start building](https://telnyx.com/sign-up)

Related articles

* [**Conversational AI**AI Agents in Financial Services: Use Cases in India![megha-profile](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/Untitled%20(400%20x%20400%20px)%20(1).png?width=96&format=webp)By Megha Sujanani12, Oct 2026](https://telnyx.com/resources/ai-agents-indian-financial-services)
* [**Conversational AI**How to build an omnichannel AI agent![Deniz Yakışıklı](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/Deniz_Yakisikli_6b7b420ed4.png?width=96&format=webp)By Deniz Yakisikli12, Oct 2026](https://telnyx.com/resources/how-to-build-an-omnichannel-ai-agent)
* [**Conversational AI**Why your voice AI agent is slow on phone calls, and how to fix it![Osman Husain Telnyx](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/Osman%20Husain%20headshot?width=96&format=webp)By Osman Husain8, Oct 2026](https://telnyx.com/resources/voice-ai-delay-causes)
* [**SIP Trunking**SIP response codes: a quick reference to every SIP code and error![Serhii](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/_DSC8799-6.jpg?width=96&format=webp)By Serhii Omelchenko14, Oct 2026](https://telnyx.com/resources/sip-response-codes)
* [**STT API**Telnyx STT against ElevenLabs Scribe on real phone audio![Abhishek Sharma](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/Abishek_Sharma_8313622a2b.png?width=96&format=webp)By Abhishek Sharma13, Oct 2026](https://telnyx.com/resources/telnyx-stt-vs-elevenlabs-scribe-benchmark)
* [**Insights and Resources**B300 GPU rental price per hour in 2026: what you actually pay![Eli Mogul](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/Eli_Mogul_1de9939dcb.png?width=96&format=webp)By Eli Mogul12, Oct 2026](https://telnyx.com/resources/b300-gpu-rental)

### Ask AI

* [![GPT](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/chatgpt_icon-ea55e09d.jpeg?width=92&format=webp)](https://chat.openai.com/?q=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.)
* [![Claude](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/claude_icon-c7c8eef3.webp?width=92&format=webp)](https://claude.ai/new?q=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.)
* [![Perplexity](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/perplexity_icon-cd5b74f1.webp?width=92&format=webp)](https://www.perplexity.ai/search?q=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.)
* [![Gemini](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/gemini_icon-4ff1cbf4.webp?width=92&format=webp)](https://www.google.com/search?udm=50&aep=11&q=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.)
* [![Grok](https://d2efw2akbx8gaf.cloudfront.net/https://dotcom-payload-assets-prod.s3.us-east-1.amazonaws.com/cms-media/grok_icon-e588e5bd.webp?width=92&format=webp)](https://x.com/i/grok?text=What makes Telnyx's global network different, breakdown call quality for normal phone calls and voice AI calls.)