Voice AI Conversations Over WebSocket Now Available for Telnyx AI Assistants

You can now talk to Telnyx AI Assistants over a direct WebSocket connection, with no telephony required. Stream PCM16 audio from any web or mobile client and receive real-time transcription, assistant audio, and tool-call events over the same connection.

What's new

• WebSocket conversation endpoint: Connect to wss://api.telnyx.com/v2/ai/assistants/{assistant_id}/conversation and stream audio directly to your assistant, with no phone number or PSTN connection required.
• Real-time bidirectional audio: The client sends base64-encoded PCM16 audio frames and Telnyx returns assistant audio, transcription, and lifecycle events in real time.
• Server-side voice activity detection: Telnyx handles VAD, turn detection, and turn-taking. The assistant manages conversation flow, so you do not build silence detection or turn logic on the client.
• Text turn injection: Send a user message as text via conversation.item.create and the assistant responds as if the user spoke it. Useful for hybrid voice and text interfaces.
• Client-side tool support: Telnyx can request tool calls that execute on your client, with results returned as function_call_output. Build tools that access local device state or browser APIs.
• Flexible sample rates: Input audio supports 8, 16, 24, 44.1, and 48 kHz. Output rate is determined by the assistant voice and reported in the session.created frame.
• Response cancellation: Cancel an in-progress assistant response with response.cancel for client-initiated barge-in without audio.

Why it matters

• Every previous Voice AI interaction on Telnyx required a phone call through Call Control. You needed a phone number, a PSTN connection, and per-minute telephony costs to talk to your assistant.
• WebSocket conversations remove that requirement. Build voice AI into web apps, mobile apps, kiosks, and any platform that speaks WebSocket, using the same assistant configuration with no changes.
• No telephony costs for app-native voice. You pay for AI processing, not the phone call.
• The assistant definition is identical whether the conversation arrives over WebSocket or a phone call. One configuration, two transport options.

Example use cases

• Web-based voice agents for support portals where users talk through their browser microphone instead of dialing a number.
• Mobile app voice assistants that use the device microphone for in-app voice interactions, with no SIP stack required.
• Kiosk and IoT deployments where a microphone and WebSocket connection are available but no phone line exists.
• Hybrid voice and text chat interfaces where users can type or speak, with the same assistant handling both.
• Developers testing Voice AI assistants locally without provisioning phone numbers or managing call flows.

Getting started

1. Create or select an AI Assistant in Mission Control under AI, then Assistants.
2. Configure your assistant voice, transcription, LLM, and tools as usual.
3. In your client application, open a WebSocket connection to wss://api.telnyx.com/v2/ai/assistants/{assistant_id}/conversation with your Telnyx API v2 key as the Bearer token.
4. Set query parameters for input_sample_rate and output_sample_rate to match your audio capture and playback configuration.
5. Stream base64-encoded PCM16 audio frames using input_audio_buffer.append and listen for session.created, transcription, and assistant-audio events.

// Open a WebSocket connection to your Telnyx AI Assistant
const ws = new WebSocket(
  'wss://api.telnyx.com/v2/ai/assistants/ASSISTANT_ID/conversation'
  + '?input_sample_rate=16000&output_sample_rate=24000'
);

ws.onmessage = (event) => {
  const frame = JSON.parse(event.data);
  console.log(frame.type);
  // session.created: confirms session and negotiated audio formats
  // input_audio_buffer.speech_started: user started speaking (VAD)
  // conversation.item.input_audio_transcription.completed: user transcript
  // response.created: assistant turn starts
  // response.audio.delta: assistant audio chunk (base64 PCM16)
};

// Stream microphone audio as base64-encoded PCM16 frames
function sendAudioChunk(pcm16Buffer) {
  ws.send(JSON.stringify({
    type: 'input_audio_buffer.append',
    audio: arrayBufferToBase64(pcm16Buffer)
  }));
}

// Inject a text turn (assistant responds as if the user spoke it)
ws.send(JSON.stringify({
  type: 'conversation.item.create',
  item: {
    type: 'message',
    role: 'user',
    content: [{ type: 'input_text', text: 'What are your business hours?' }]
  }
}));

// Cancel the in-progress assistant response (barge-in)
ws.send(JSON.stringify({ type: 'response.cancel' }));

Learn more in the Telnyx WebSocket API reference or the Voice AI Assistants docs.