# Telnyx AI: Assistants — Full Documentation > Complete page content for Assistants (AI section) of the Telnyx developer docs (https://developers.telnyx.com). > Root index: https://developers.telnyx.com/llms.txt · Lightweight index for this subsection: https://telnyx.com/llms/ai/assistants.txt ## Assistants ### Voice Assistant > Source: https://developers.telnyx.com/docs/inference/ai-assistants/no-code-voice-assistant.md In this tutorial, you'll learn how to configure a voice assistant with Telnyx. You won't have to write a single line of code or create an account with anyone besides Telnyx. You'll be able to talk to your assistant over the phone in under five minutes. After this tutorial covers the basics, we will explore some optional enhancements, such as changing your voice or language model providers and empowering your assistant with built-in tools. Try out the [public demos](https://telnyx.com/) for a real example of the finished product. ## Video Tutorial Watch this step-by-step demonstration of creating a voice assistant: ## Requirements There are 2 required steps for this tutorial 1. Configure your AI Assistant 2. Configure the voice settings ### Configure your AI Assistant First, navigate to the [AI Assistants tab](https://portal.telnyx.com/#/ai/assistants) in the portal. You will create a new assistant to configure what context your assistant has and how it behaves. For this tutorial, we will use a blank template. ![AI Assistant Portal Templates](/img/ai-assistant-templates.png) You can use the following instructions and greeting, or use your own. Instructions: ``` You are an intelligent and concise voice assistant. This is a {{telnyx_conversation_channel}} happening on {{telnyx_current_time}}. The agent is at {{telnyx_agent_target}} and the user is at {{telnyx_end_user_target}}. ``` Greeting: ``` Hi {{first_name}}, this is Nyx, your friendly Telnyx Assistant! How can I help you today? ``` ![AI Assistant Portal Config](/img/create_assistant.png) Telnyx provides these system variables to inject details about a specific call into the instructions and greeting: | Variable | Description | Example | |----------|-------------|---------| | `{{telnyx_current_time}}` | The current date and time in UTC | `Monday, February 24 2025 04:04:15 PM UTC` | | `{{telnyx_conversation_channel}}` | This can be `phone_call` , `web_call`, or `sms_chat` | `phone_call` | | `{{telnyx_agent_target}}` | The phone number, SIP URI, or other identifier associated with the agent. | `+13128675309` | | `{{telnyx_end_user_target}}` | The phone number, SIP URI, or other identifier associated with the end user | `+13128675309` | | `{{telnyx_sip_header_user_to_user}}` | The User to User SIP header for the call, if applicable | `cmlkPTM0Nzg1O3A9dQ==;encoding=base64;purpose=call` | | `{{telnyx_sip_header_diversion}}` | The Diversion SIP header for the call, if applicable | `;reason=user-busy` | | `{{call_control_id}}` | The call control ID for the call, if applicable | `v3:u5OAKGEPT3Dx8SZSSDRWEMdNH2OripQhO` | Telnyx also supports timezone-aware date/time variants (e.g. `{{telnyx_current_time_America/New_York}}`), shorthands (`{{telnyx_current_date}}`, `{{telnyx_current_weekday}}`), and a custom `date` format filter. See [dynamic variables](https://developers.telnyx.com/docs/inference/ai-assistants/dynamic-variables) for the full list. You can also define your own [custom dynamic variables](https://developers.telnyx.com/docs/inference/ai-assistants/dynamic-variables) and set them via webhook, custom SIP headers, or an outbound API call. In this example, you've given the assistant access to the most commonly used system variables and a custom `{{first_name}}` variable in the greeting. Notice that by default, the Hangup tool is configured. This enables your assistant to end the call at an appropriate time. ### Configure the voice settings In this step, you can use the default voice settings, click `Create`, and enable the agent for calls when prompted. Or feel free to explore the wide range of voices in the playground. - TTS (Text-to-Speech): Telnyx, AWS, Azure, Elevenlabs, Inworld - STT (Speech-to-Text): Telnyx (whisper), Deepgram, Azure — see [Transcription Settings](/docs/inference/ai-assistants/transcription-settings) for model details and configuration You can browse the list of available voices and configuration options in the [Text-to-Speech Available Voices](/docs/tts-stt/tts-available-voices/index#text-to-speech-available-voices) page. [Ultra](/docs/voice/tts/providers/telnyx/ultra) and [xAI Grok](/docs/voice/tts/providers/telnyx/grok) voices support Expressive Mode with inline SSML emotion tags and nonverbal cues like `[laughter]`. **Background Audio** You can also configure background audio to play during the call. This provides a more realistic noise environment for voice calls, making longer pauses from tool calls feel more natural. Customers can select from our list of predefined options or share their own custom public URL. **Speaking Plan** Customers are in full control of when an agent starts talking and can distinguish between 4 types of pauses: 1. **Wait seconds** sets your baseline. A customer service agent might use 0.3 seconds for snappy responses. An agent calling into an IVR system needs 1.5 seconds to account for slower robotic speech. 2. **On punctuation seconds** handles high-confidence endpoints. When the transcription ends with a period or question mark, the user likely finished their thought. Set this to 0.1 seconds for minimal delay. 3. **On no punctuation seconds** handles uncertainty. The user said "my order number is" and paused. They're probably looking at their screen. Set this to 1.5 seconds so the agent doesn't interrupt with "I didn't catch that" while they're reading the digits. 4. **On number seconds** handles digit sequences specifically. People read numbers slowly: "4... 7... 2... 9." Each pause could trigger a response. Extending this to 1.0 seconds prevents the agent from cutting them off at "4... 7..." **Noise Suppression** Enable noise suppression to reduce background noise during calls. This is especially valuable for AI assistants — cleaner audio improves STT accuracy and leads to better AI responses. Two engines are available: | Engine | Description | |--------|-------------| | **Krisp** | Industry-leading noise suppression, effective across diverse environments (home offices, contact centers, outdoor) | | **DeepFilterNet** | Configurable attenuation for fine-grained control | To enable via the [Mission Control Portal](https://portal.telnyx.com/#/ai/assistants), select the noise suppression engine under the voice settings when creating or editing your assistant. ![AI Assistant Noise Suppression](/img/ai-assistant-noise-suppression.png) To enable via the API, set `noise_suppression` in `telephony_settings`: ```bash curl -X POST https://api.telnyx.com/v2/ai/assistants/{assistant_id} \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "telephony_settings": { "noise_suppression": "krisp" } }' ``` Set to `"disabled"` to turn off noise suppression. See the [Noise Suppression guide](/docs/voice/programmable-voice/noise-suppression) for more details on engines and direction options. ![AI Assistant Voice Config](/img/full-voice-config.png) ### (Optional) Assign a phone number If you have already purchased a Telnyx number with voice features, you can immediately assign it to your assistant. You can also click `Next`, as completing this step is not needed for testing your assistant. ![AI Assistant Number Config](/img/assistant_assign_numbers.png) ### (Optional) Enable messaging Telnyx assistants work with messaging as well. This is not covered in this tutorial. ### Test it out! You should now be able to interact with your voice assistant. When you are ready, you can ask the assistant to hang up to test out the Hangup tool. ![Call Agent](/img/test_assistant.png) ![Talking to Agent](/img/talking_to_assistant.png) If you assigned a number to your assistant, you can also have your assistant call you: - from the portal - via API/CLI - in an automated workflow like Zapier using our [TeXML Outbound Call](https://zapier.com/apps/telnyx/integrations#triggers-and-actions) action. ![Outbound Selection](/img/assistant_test_outbound.png) ![Outbound Calling Agent](/img/outbound_agent_test.png) *If you are using the curl provided, make sure to set the `TELNYX_API_KEY` environment variable with your API Key* ### Outbound calls via API To initiate an outbound call with your AI assistant via API, use the `/v2/texml/ai_calls/` endpoint: ```bash curl --request POST \ --url https://api.telnyx.com/v2/texml/ai_calls/ \ --header "Authorization: Bearer $TELNYX_API_KEY" \ --header 'Content-Type: application/json' \ --data '{ "From": "+13128675309", "To": "+15551234567", "AIAssistantId": "assistant-6207ab25-b185-478f-b2ef-85159e226727" }' ``` If your assistant has [voicemail detection options](https://developers.telnyx.com/api-reference/assistants/create-an-assistant#body-telephony-settings-voicemail-detection) configured in the telephony settings, include these AMD parameters: ```bash curl --request POST \ --url https://api.telnyx.com/v2/texml/ai_calls/ \ --header "Authorization: Bearer $TELNYX_API_KEY" \ --header 'Content-Type: application/json' \ --data '{ "From": "+13128675309", "To": "+15551234567", "AIAssistantId": "assistant-6207ab25-b185-478f-b2ef-85159e226727", "MachineDetection": "Enable", "AsyncAmd": true, "DetectionMode": "Premium" }' ``` ### Review the conversation You can see all historical conversations in the **Conversation History** tab ![AI Assistant Transfer](/img/handoff_conversation.png) This example shows a conversation where the Handoff tool was used. ## MMS integration during voice calls Your AI Assistant can now receive and process MMS messages during live voice calls, enabling visual context and real-time image analysis. This powerful feature allows your assistant to handle complex scenarios where visual information is crucial. ### How it works When a user sends an MMS message during an ongoing voice call with your AI Assistant, the agent can: - Automatically detect incoming MMS messages. - Access and analyze attached images using vision-capable Language Models (VLMs). - Provide real-time responses based on visual content. - Continue the voice conversation with enhanced context. ### Use cases | Category | Use Case | Description | |----------|----------|-------------| | **Visual Support** | Customer Service | Analyze product photos sent by customers | | | Technical Support | Review error screenshots or equipment photos | | | Healthcare | Examine medical documents or symptoms | | **Document Verification** | Insurance Claims | Process claims with photo evidence | | | Identity Verification | Verify identity with document images | | | Compliance | Conduct compliance checks with visual documentation | | **Real-time Analysis** | Quality Control | Perform inspections with photo submissions | | | Inventory Management | Manage inventory with visual confirmation | | | Damage Assessment | Assess damage with real-time photo analysis | ### Configuration MMS integration requires the following setup for your AI Assistant: 1. **Messaging must be enabled** - Ensure messaging is enabled for your Voice AI Agent to receive MMS during calls. ![AI Assistant Messaging Configuration](/img/ai-assistant-messaging-enabled.png) 2. **Vision-capable models required** - Use one of the two supported vision models: `Groq/llama-4-maverick-17b-128e-instruct` or `OpenAI/gpt-4o` for image processing. 3. **Image processing** - The assistant can handle common image formats (JPEG, PNG, etc.). ### Best practices - **Model selection**: Choose one of the two supported vision models when configuring your assistant. ![AI Assistant Model Selection](/img/ai-assistant-agent-model-selection.png) - **Response timing**: The assistant will process images and respond within the voice call flow. - **Image quality**: Higher resolution images provide better analysis results. - **Context integration**: The assistant seamlessly combines visual and conversational context. For detailed information about vision Language Models and how to use them with our API, see our [Models page](/docs/inference/models). ## Supported language models Telnyx AI Assistants support multiple language models. You can select a model in the **Agent** tab when creating or editing an assistant. | Model | Provider | Description | | --- | --- | --- | | `anthropic/claude-haiku-4-5` | Anthropic (native) | Fast, lightweight model — no API key required | | `openai/gpt-4o` | OpenAI | Requires OpenAI API key (see [OpenAI integration](#openai-integration)) | | `moonshotai/Kimi-K2.5` | Telnyx (native) | Recommended balance of intelligence and cost | For a complete list of available models, see [Available Models](/docs/inference/models). Native models run on Telnyx infrastructure with no external API key required. For models from external providers, see [Third-party integrations](#third-party-integrations) or [Custom LLMs for Assistants](/docs/inference/ai-assistants/custom-llm). ## Optional enhancements ### Integrations You can integrate your assistant with enterprise platforms to access customer data, create tickets, update records, and automate workflows directly during conversations. Telnyx AI assistants support integrations with: - **Salesforce** - CRM and customer service - **ServiceNow** - IT service management - **Jira** - Project and issue tracking - **HubSpot** - Marketing and sales CRM - **Zendesk** - Customer support - **Intercom** - Customer messaging - **Github** - Code hosting and version control - **Greenhouse** - Applicant tracking Select the integration from the dropdown ![AI Integration Dropdown](/img/ai-integration-dropdown.png) Enter your credentials ![AI Integration Creds](/img/ai-integration-creds.png) Connect the integration to the assistant and choose the enabled tools ![AI Integration Dropdown](/img/ai-integration-tools.png) **Learn more:** For comprehensive setup guides, required credentials, available tools, and best practices for each integration, see the [AI Assistant Integrations](/docs/inference/ai-assistants/integrations) documentation. ### Additional built-in tools Besides `Hangup`, we offer several additional built-in tools to empower your agent to take real-world actions. **Webhook** With the webhook tool, your agent can make API requests. You can configure headers (with integration secrets), along with path, query, and body parameters. You can also reference dynamic variables in the webhook path or in the parameter descriptions. ![AI Assistant Webhook](/img/agent_webhook_tool.png) After you've saved a webhook tool on your assistant, you can test it out with sample data by clicking the play button icon on the tool. ![AI Assistant Webhook Test](/img/agent_webhook_test.png) **Handoff** With the handoff tool, you can enable multiple assistants to support a user in a single conversation. By default, the handoff is transparent to the user: assistants share the same context and voice, allowing for a unified experience where a variety of tasks can be handled by a team of specialists working behind the scenes. You can also toggle to the distinct voice mode, where all assistants retain their voice configuration, providing the experience of a conference call with a team of assistants ![AI Assistant Transfer](/img/handoff_config.png) Learn more about agent handoff, including best practices, industry-specific templates, and advanced configuration in the [Agent Handoff guide](/docs/inference/ai-assistants/agent-handoff). **Transfer** and **SIP Refer** With the transfer and SIP Refer tools, your agent can transfer or refer a call to a list of named targets. ![AI Assistant Transfer](/img/agent_transfer.png) **Send DTMF** With the Send DTMF, your agent can interact with legacy IVR systems. ### Model Context Protocol (MCP) Servers You can [configure an MCP Server with Telnyx](https://portal.telnyx.com/#/ai/mcp-servers) and then add it to an assistant. If the URL for the server must be kept secret (because the server is not otherwise authenticated), you may store it securely as an integration secret with Telnyx. ![AI Assistant MCP](/img/mcp_server.png) ![AI Assistant MCP Config](/img/mcp_assistant.png) When setting up your MCP servers with Telnyx AI Assistants, Telnyx automatically includes a `telnyx_conversation_id` with each MCP tool call. If you are managing your own MCP Server, the `telnyx_conversation_id` can be used for tracking and controlling the flow of conversations within your applications. This is sent on _meta field of MCP. To receive the conversation ID at the start of a voice conversation, you have two options: - For call control applications, the conversation ID is returned by the [Start AI Assistant command](/api-reference/call-commands/start-ai-assistant#start-ai-assistant) - If you have configured a [dynamic variables webhook URL](https://developers.telnyx.com/docs/inference/ai-assistants/dynamic-variables), the conversation ID will be sent in this request payload at the start of a voice conversation. *The `telnyx_conversation_id` is set by the Telnyx platform, not by the AI agent, and as such is not susceptible to prompt injection attacks.* ```json { "jsonrpc": "2.0", "id": 1, "method": "some_mcp_method", "params": { "_meta": { "progressToken": null, "telnyx_conversation_id": "123" } } } ``` ### Knowledge Bases You can use the **Knowledge Bases** tab to enable your assistant to retrieve your custom context. First, provide a name ![AI Assistant KB](/img/create-knowledge-base.png) Then upload files or provide a URL ![AI Assistant KB URL](/img/upload_for_knowledge_base.png) ### Insights You can automatically run structured and unstructured analysis on every assistant conversation using the **Insights** tab. ![AI Assistant Insights](/img/insights-config.png) You can assign an **Insight Group** to your assistant, which can have one or more Insights. Insights can be reused across Groups, and Groups can be reused across Assistants. You can also configure a webhook URL to receive conversation insights after they are generated. ![AI Assistant Insights Webhook](/img/insights-webhook.png) **Learn more:** For comprehensive guides on creating insights, using structured data schemas, organizing insight groups, configuring webhooks, and industry-specific use cases, see the [AI Insights documentation](https://developers.telnyx.com/docs/inference/ai-insights). ### Embeddable Widget You can easily embed a customizable voice and chat widget on your frontend in the **Widget** tab. ## Programmatic Voice You can also start your assistant as part of a programmatic voice application using the [Start Assistant](/api-reference/call-commands/start-ai-assistant#start-ai-assistant) command. ## Third-party integrations By default, every component of a Telnyx AI Assistant runs on Telnyx infrastructure. You can, however, BYO LLM or TTS using third-party providers. ### Vapi integration If you have voice assistants configured in Vapi, you can [import them](https://developers.telnyx.com/docs/inference/ai-assistants/importing) as Telnyx AI Assistants in a single click. If you want to use a voice from Vapi in your existing Telnyx assistant, you will 1. Create a Vapi API Key. 2. Reference the key in your Assistant voice configuration. ##### Create a Vapi API Key First, check out their guide on [creating an API Key](https://docs.vapi.ai/chat/web-widget#1-get-your-public-api-key) ##### Reference the key in your Assistant voice configuration In the voice tab for your assistant, you can select Vapi as the provider. A new dropdown will then appear to reference your API key. You can give the secret a friendly identifier and securely store your API key as the secret value. To enable a multilingual agent, set the transcription model to `deepgram/nova-3`. *You will not be able to access the value of a secret after it is stored.* You can also manage all your secrets in the [Integration Secrets](https://portal.telnyx.com/#/integration-secrets) tab in the portal. Choose a memorable identifier to refer to the API key and store it in the `Secret Value` field. ![Integration Secret Portal Config](/img/integration_secrets.png) ### ElevenLabs integration If you have Conversational AI agents configured in Elevenlabs, you can [import them](https://developers.telnyx.com/docs/inference/ai-assistants/importing) as Telnyx AI Assistants in a single click. If you want to use a voice from ElevenLabs in your existing Telnyx assistant, you will 1. Create an ElevenLabs API Key 2. Reference the key in your Assistant voice configuration *Requests from a free plan are rejected. You will likely have to use a paid plan to set up this integration successfully.* ##### Create an ElevenLabs API Key First, check out their guide on [creating an API Key](https://help.elevenlabs.io/hc/en-us/articles/14599447207697-How-to-authorize-yourself-using-your-xi-api-key) ##### Reference the key in your Assistant voice configuration In the voice tab for your assistant, you can select ElevenLabs as the provider. A new dropdown will then appear to reference your API key. You can give the secret a friendly identifier and securely store your API key as the secret value. To enable a multilingual agent, set the transcription model to `deepgram/nova-3`. *You will not be able to access the value of a secret after it is stored.* ![ElevenLabs Voice Config](/img/eleven_labs_voice_secret.png) You can also manage all your secrets in the [Integration Secrets](https://portal.telnyx.com/#/integration-secrets) tab in the portal. Choose a memorable identifier to refer to the API key and store it in the `Secret Value` field. ![Integration Secret Portal Config](/img/integration_secrets.png) ### OpenAI integration To use an LLM from OpenAI in your assistant, you will 1. Create an OpenAI API Key 2. Configure the language model in your AI Assistant *Requests from a free plan are rejected. You will likely have to use a paid plan to set up this integration successfully.* ##### Create an OpenAI API Key First, check out their guide on [creating an API Key](https://help.openai.com/en/articles/4936850-where-do-i-find-my-openai-api-key) ##### Configure the language model in your AI Assistant Back at the [AI Assistants tab](https://portal.telnyx.com/#/ai/assistants) in the portal, edit your assistant. First, change the model to an OpenAI model like `openai/gpt-4o`. Then follow the same API Key steps as described in the ElevenLabs section above. --- ### Multi-Participant Calls > Source: https://developers.telnyx.com/docs/inference/ai-assistants/multi-participant-calls.md Multi-participant Voice AI calls let an assistant bring another person into an active call, follow who is speaking, and continue using the same tools and instructions it would use in a one-to-one voice conversation. Use this pattern when your assistant needs to coordinate between people in real time, such as scheduling a meeting, connecting a customer with a specialist, or letting multiple callers complete a task together. Listen to this example call to hear how an assistant can invite a participant, stay silent while people talk to each other, and resume when asked to take action. In this guide, you will learn how to: - Add an **Invite** tool so your assistant can invite another participant to the current call. - Design assistant instructions for multi-participant conversations. - Use the **Skip Turn** tool so the assistant can stay silent while people talk to each other. - Review a multi-participant call in Conversation History. - Configure the same behavior through the Portal or the [Assistants API](/api-reference/assistants/create-an-assistant). --- ## How multi-participant calls work A multi-participant Voice AI call starts like any other voice assistant call. The assistant speaks with the main caller, then uses an Invite tool when it needs to bring another participant into the conversation. Once the new participant joins, the assistant can: - Tell whether the main user or an invited participant is speaking. - Respond to either participant when appropriate. - Use its configured tools, memory, dynamic variables, and integrations. - Stay silent when the participants are talking to each other. - Resume when someone addresses the assistant again. By default, an assistant will respond to every turn it receives. For natural multi-participant calls, add a Skip Turn tool and describe when the assistant should remain silent. --- ## Requirements Before you start, create a voice assistant by following the [Voice Assistant Quickstart](/docs/inference/ai-assistants/no-code-voice-assistant). You can configure assistants and tools in the Portal or through the [Assistants API](/api-reference/assistants/create-an-assistant). You also need: - A phone number or SIP URI for the assistant. - A phone number or SIP URI for each participant the assistant may invite. - Any tools the assistant should use after the participant joins, such as a calendar, CRM, or booking integration. --- ## Step 1: Add an Invite tool The Invite tool lets your assistant invite another participant into the current call. 1. In the [Mission Control Portal](https://portal.telnyx.com), open your assistant. 2. Go to the assistant's **Tools** section. 3. Add an **Invite** tool. 4. Configure the invite target with the participant's phone number or SIP URI. 5. Save the assistant. You can also configure the assistant, its model, voice settings, and tools through the [Assistants API](/api-reference/assistants/create-an-assistant). When the assistant decides that another person should join, it calls the Invite tool. After the participant joins, the assistant receives the updated conversation context and can continue the call with both participants. In the example below, the user asks the assistant to invite Enzo. The assistant calls the Invite tool, receives a `Participant joined` response, then confirms that Enzo has joined. ![Invite tool call shown in Conversation History](/img/ai-assistants-multi-participant/conversation-invite-tool-call.png) --- ## Step 2: Give the assistant multi-participant instructions Your assistant should know that the conversation may include multiple people. Add instructions that tell the assistant how to behave when participants are speaking to each other. Example instructions: ```text You are Amber, a voice assistant helping coordinate a multi-participant call. The call may include the main user and one or more invited participants. Pay close attention to who is speaking and who they are addressing. When the main user asks you to invite someone, use the Invite tool. After the participant joins, briefly confirm that they joined, then let the participants talk. If the participants are talking to each other and are not addressing you, use the Skip Turn tool and remain silent. If someone addresses you directly, respond normally. You can use any of your configured tools to help complete the task. ``` You can adapt this prompt for your use case. For example, a scheduling assistant might be instructed to listen while participants compare availability, then resume when asked to book a time. --- ## Step 3: Add a Skip Turn tool In a one-to-one call, an assistant usually responds after every user turn. In a multi-participant call, that can feel unnatural because the assistant may interrupt people who are talking to each other. The Skip Turn tool lets the assistant intentionally remain silent for a turn. Add a Skip Turn tool when you want the assistant to: - Stay silent while invited participants talk to the main user. - Avoid responding to side conversations. - Wait until someone directly addresses the assistant. - Let humans confirm details with each other before the assistant takes action. In this example, James asks Enzo how he is doing. The assistant recognizes that James is speaking to Enzo, not to the assistant, and uses `skip_turn` with the reason `James and Enzo are talking to each other, not addressing me`. ![Skip Turn tool call shown in Conversation History](/img/ai-assistants-multi-participant/skip-turn-tool-call.png) Skip Turn does not end the call or disable the assistant. It only tells the assistant not to speak for that turn. The assistant can respond again when a participant addresses it. If your assistant should only respond when a specific name is spoken, or you know the participant names ahead of time, add those names to **Keyterm Boost** on the assistant's **Voice** tab. Boosting the names improves transcription accuracy on those exact words, which makes name-based Skip Turn rules far more reliable. Keyterm Boost accepts a comma-separated list, for example `Telnyx,Amber,Enzo`. It also supports [dynamic variables](/docs/inference/ai-assistants/dynamic-variables), so you can pass participant names or other caller-specific terms at conversation start, for example `Telnyx,{{participant_names}}`. It is supported by `deepgram/flux` and `deepgram/nova-3`. See the [Transcription Settings guide](/docs/inference/ai-assistants/transcription-settings) for details. ![Keyterm Boost field on the Voice tab of the AI Assistant settings](/img/ai-assistants-multi-participant/keyterm-boost-voice-tab.png) --- ## Step 4: Use other assistant tools during the call After a participant joins, the assistant can do everything it can do in a one-to-one voice call. It can call APIs, look up information, schedule meetings, update records, send messages, and more. In this example, James asks the assistant to schedule time with Enzo. The assistant checks calendar availability, proposes a time, waits while James confirms with Enzo, then books the meeting. ![Calendar availability tool call in a multi-participant call](/img/ai-assistants-multi-participant/calendar-tool-call.png) The assistant can continue to combine tools with turn-taking logic. For example: 1. The assistant checks availability. 2. The assistant proposes a time to both participants. 3. The main user asks the invited participant whether the time works. 4. The assistant stays silent while the invited participant answers. 5. The assistant books the meeting after the participants confirm. ![Meeting booking completed in a multi-participant call](/img/ai-assistants-multi-participant/booking-complete.png) --- ## Best practices ### Be explicit about when to speak Tell the assistant when it should respond and when it should stay silent. Multi-participant calls work best when the assistant has clear turn-taking rules. Good instruction: ```text If James and Enzo are speaking directly to each other, use Skip Turn and stay silent. Only respond when someone addresses you by name or asks you to take an action. ``` ### Review calls in Conversation History Use Conversation History to inspect the transcript, audio, tool calls, and tool responses. This is especially useful when tuning Skip Turn instructions because you can see why the assistant decided to speak or remain silent. --- ## Next steps - Build your first assistant with the [Voice Assistant Quickstart](/docs/inference/ai-assistants/no-code-voice-assistant). - Add reusable tools from the [Tools Library](/docs/inference/ai-assistants/tools-library). - Use [dynamic variables](/docs/inference/ai-assistants/dynamic-variables) to personalize calls. - Review production calls with [Agent Observability](/docs/inference/ai-assistants/agent-observability). --- ### Memory > Source: https://developers.telnyx.com/docs/inference/ai-assistants/memory.md Memory enables your AI assistant to recall essential details from past conversations. Instead of starting each phone call or text exchange from scratch, your AI assistant naturally continues previous discussions. In this tutorial, you will learn how to: - Specify which conversations your AI Assistant has memory access to - Configure this dynamically at the start of every conversation *Telnyx Assistants natively support our Voice and Messaging APIs, meaning the same assistant can seamlessly remember conversations across channels.* --- ## Identifying the conversations to include There is no one-size-fits-all answer for which previous conversations an AI Assistant should remember during a specific conversation. You may want an AI Assistant to have memory access to: - Every conversation it had with any user - Every conversation it had with **this specific user** - Every conversation it had with **users in a specific group** in **the past 10 days** - Or something else entirely... To support this, we have exposed a flexible query language to give customers full control over their assistant's memory. Any query you can build with our [List Conversations endpoint](/api-reference/conversations/list-conversations), you can use to configure memory access. ## Configuring memory with the Dynamic Variables Webhook If the `dynamic_variables_webhook_url` is set for the assistant, we will send the following payload at the start of the conversation. ``` { "data" :{ "record_type": "event", "id": "event_id", "event_type": "assistant.initialization", "occurred_at": "2025-04-07T10:00:00Z", "payload": { "telnyx_conversation_channel": "phone_call", "telnyx_agent_target": "+1234567890", "telnyx_end_user_target": "+1234567890", "telnyx_end_user_target_verified": false } } } ``` *For inbound phone calls to an assistant, the `telnyx_end_user_target_verified` field will be set to `true` if the call has Full (A) [STIR/SHAKEN attestation](https://support.telnyx.com/en/articles/5402969-stir-shaken-with-telnyx) and Telnyx was able to verify the authenticity of the PASSporT token.* We expect a JSON response with the following structure. If we do not receive this response within a 1-second timeout, the call will proceed "best effort". ``` { "dynamic_variables": { "full_name": "Rachel Thomas", "facility_name": "UCHealth", "facility_department": "Cardiology" }, "memory": { "conversation_query": "metadata->telnyx_end_user_target=eq.+13128675309&limit=5&order=last_message_at.desc" } } ``` In this example, the optional `memory` field provides your AI assistant with memory access to the last 5 conversations with the current user's phone number. You can read more about the optional `dynamic_variables` field in our [tutorial on Dynamic Variables](https://developers.telnyx.com/docs/inference/ai-assistants/dynamic-variables). ![AI Assistant Variable Config](/img/configure-dynamic.png) In addition to controlling which conversations are remembered, you can customize which information from a conversation is remembered. To do this, specify a comma-delimited list of insight IDs in the memory field. Insight IDs can be retrieved in the Insights tab for your assistant, as shown below. Only the results from the insights you specify will be included in the assistant's memory. ``` { "dynamic_variables": { "full_name": "Rachel Thomas", "facility_name": "UCHealth", "facility_department": "Cardiology" }, "memory": { "conversation_query": "metadata->telnyx_end_user_target=eq.+13128675309&limit=5&order=last_message_at.desc", "insight_query": "insight_ids=123,456"" } } ``` ![AI Assistant Insight Copy](/img/copy-insight-id.png) ## Custom Metadata You may want to create your own memory access system based on custom metadata for conversations. To do this, you can add metadata to conversations in the dynamic variable webhook response: ``` { "dynamic_variables": { "full_name": "Rachel Thomas", "facility_name": "UCHealth", "facility_department": "Cardiology" }, "memory": { "conversation_query": "metadata->telnyx_end_user_target=eq.+13128675309&limit=5&order=last_message_at.desc" }, "conversation": { "metadata": { "your_custom_metadata": "your_custom_value" } } } ``` In future conversations, you can filter that metadata in the `memory` field using the following syntax `metadata->your_custom_metadata=eq.your_custom_value`. --- ### Dynamic Variables > Source: https://developers.telnyx.com/docs/inference/ai-assistants/dynamic-variables.md Dynamic variables let you configure a template for your agent's behavior. You can re-use the same general instructions while dynamically personalizing every conversation your agent has. In this tutorial, you will learn how to: - Template your AI assistant configuration with dynamic variables. - Supply the values via an outbound API request or the dynamic variable webhook. --- ## Overview Dynamic variables enable you to create a single AI assistant configuration that can handle personalized conversations for different users and contexts. Instead of creating multiple assistants for different scenarios, you can use placeholders that get filled with specific values at runtime. ### How dynamic variables work The dynamic variable lifecycle follows these steps: 1. **Define**: Create placeholders in your assistant's instructions, greeting, or tools using `{{variable_name}}` syntax. 2. **Inject**: Provide values through API calls, webhooks, SIP headers, or default configurations. 3. **Resolve**: Telnyx replaces placeholders with actual values when the conversation starts. 4. **Use**: Your assistant uses the personalized content throughout the conversation. ### Key benefits - **Scalability**: One assistant configuration serves multiple use cases. - **Personalization**: Each conversation can be tailored to specific users or contexts. - **Efficiency**: Reduce configuration overhead and maintenance complexity. - **Flexibility**: Update values dynamically without modifying the assistant. ### Common use cases - **Customer Service**: Personalize greetings with customer names and account details. - **Appointment Scheduling**: Include facility names, departments, and contact information. - **Account Management**: Reference specific account numbers, balances, or service details. - **Healthcare**: Customize interactions with patient names, appointment types, and provider information. --- ## Best practices Following these best practices will help you implement dynamic variables effectively and avoid common pitfalls. ### Variable naming and organization - **Use descriptive names**: Choose clear, meaningful variable names like `customer_name` instead of generic names like `name`. - **Follow consistent conventions**: Use snake_case for variable names (`facility_name`, `account_number`). - **Group related variables**: Organize logically related variables together (`facility_name`, `facility_department`, `facility_contact`). - **Avoid reserved namespaces**: Don't use the `telnyx_` prefix, which is reserved for system variables. ```json // Good variable naming { "customer_name": "Sarah Johnson", "account_number": "ACC-12345", "facility_name": "Memorial Hospital", "facility_department": "Cardiology" } // Poor variable naming { "name": "Sarah", "num": "12345", "place": "Hospital", "dept": "Card" } ``` ### Implementation strategy - **Start with defaults**: Set default values in the Assistant builder for testing and as fallback values. - **Use API injection for dynamic data**: Pass call-specific values through the `AIAssistantDynamicVariables` parameter in API calls. - **Leverage webhooks for complex lookups**: Use the dynamic variables webhook for real-time data retrieval or database lookups. - **Implement graceful degradation**: Ensure your assistant can handle scenarios where variables fail to resolve. ### Security and data handling - **Protect sensitive information**: Be cautious about including sensitive data in variables that may appear in logs. - **Validate webhook data**: Implement proper validation for data returned from webhook endpoints. - **Consider data retention**: Review your data retention and privacy requirements for variable content. - **Handle timeouts gracefully**: Webhook responses must return within 1 second or the call proceeds with fallback values. ### Performance and reliability - **Optimize webhook response time**: Keep webhook responses fast (well under the 1-second timeout). - **Implement error handling**: Plan for scenarios where webhooks fail or return invalid data. - **Use caching strategies**: Cache frequently accessed data to improve response times. - **Test across channels**: Verify variable resolution works consistently across phone calls, web calls, and SMS. ### Testing and debugging - **Test with Portal defaults**: Use the Assistant builder's default values to test variable behavior during development. - **Monitor webhook logs**: Use the conversation transcript and webhook logs in the Portal to debug variable resolution issues. - **Validate fallback behavior**: Ensure unresolved variables (displayed as `{{variable_name}}`) don't break conversation flow. - **Test transfer scenarios**: Verify that variables pass correctly when using transfer tools with SIP headers. ### Cross-channel considerations - **Ensure universal compatibility**: Test that your variables work across phone calls, web calls, and SMS channels. - **Handle channel-specific data**: Consider how variables might behave differently across conversation channels. - **Plan for transfer scenarios**: When transferring calls, use SIP headers to pass variable data to the receiving assistant. --- ## Dynamic variables syntax Dynamic variables are placeholders surrounded by double curly braces. For example: ```json Hello, this is ABC Ambulance. Am I speaking with {{full_name}} with {{facility_name}} - {{facility_department}}?" ``` You can template them in the following fields: - Instructions - Greeting - Tools ## Telnyx system variables Telnyx also provides these system variables: | Variable | Description | Example | |----------|-------------|---------| | `{{telnyx_current_time}}` | The current date and time in UTC | `Monday, February 24 2025 04:04:15 PM UTC` | | `{{telnyx_conversation_channel}}` | This can be `phone_call` , `web_call`, or `sms_chat` | `phone_call` | | `{{telnyx_agent_target}}` | The phone number, SIP URI, or other identifier associated with the agent. | `+13128675309` | | `{{telnyx_end_user_target}}` | The phone number, SIP URI, or other identifier associated with the end user | `+15551234567` | | `{{telnyx_shaken_stir_attestation}}` | The SHAKEN/STIR attestation level for inbound phone calls, if available. This can be `a`, `b`, or `c`. | `a` | | `{{call_control_id}}` | The call control ID for the call, if applicable | `v3:u5OAKGEPT3Dx8SZSSDRWEMdNH2OripQhO` | ### Date and time variables In addition to the default `{{telnyx_current_time}}`, Telnyx provides a family of reserved date/time variables. Use these to give the assistant timezone-aware context, convenient shorthands, or a fully custom format. The timezone variants and shorthands use the same default formatting as `{{telnyx_current_time}}` — a different format is only used when you explicitly specify one with the `date` filter. | Variable | Description | Example output | |----------|-------------|----------------| | `{{telnyx_current_time_}}` | Current date and time, in the given timezone | `Monday, February 24 2025 11:04:15 AM EST` | | `{{telnyx_current_date}}` | Current date (UTC) | `February 24 2025` | | `{{telnyx_current_time_of_day}}` | Current time of day (UTC) | `04:04:15 PM UTC` | | `{{telnyx_current_time_of_day_}}` | Current time of day, in the given timezone | `11:04:15 AM EST` | | `{{telnyx_current_weekday}}` | Current day of the week (UTC) | `Monday` | | `{{telnyx_current_month}}` | Current month name (UTC) | `February` | | `{{telnyx_current_day}}` | Current day of the month (UTC) | `24` | | `{{telnyx_current_year}}` | Current year (UTC) | `2025` | The shorthand variables (`telnyx_current_date`, `telnyx_current_time_of_day`, `telnyx_current_weekday`, `telnyx_current_month`, `telnyx_current_day`, `telnyx_current_year`) are evaluated in UTC. To render them for a specific timezone, use the `_` variants or the `date` filter described below. #### Timezone variants Append an [IANA timezone name](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones) to `telnyx_current_time_` or `telnyx_current_time_of_day_` to render the value in that zone. These use the same default formatting as `{{telnyx_current_time}}`: - `{{telnyx_current_time_America/New_York}}` → `Monday, February 24 2025 11:04:15 AM EST` - `{{telnyx_current_time_Australia/Sydney}}` → `Tuesday, February 25 2025 03:04:15 AM AEDT` - `{{telnyx_current_time_of_day_Europe/London}}` → `04:04:15 PM GMT` #### Custom formatting with the `date` filter The default format is used everywhere unless you explicitly request a different one. For full control over formatting, pipe `telnyx_current_time` through the `date` filter. The filter takes a standard [`strftime`](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes) format string and an optional IANA timezone. These are the same `%`-style codes used by `strftime` across most languages (C, Python, Ruby, and others): | Template | Example output | |----------|----------------| | `{{ telnyx_current_time \| date: "%A, %B %d, %Y" }}` | `Monday, February 24, 2025` (UTC) | | `{{ telnyx_current_time \| date: "%I:%M %p", "America/New_York" }}` | `11:04 AM` (in the given zone) | | `{{ telnyx_current_time \| date: "%A, %B %d, %Y at %I:%M %p %Z", "America/Los_Angeles" }}` | `Monday, February 24, 2025 at 08:04 AM PST` | When the timezone argument is omitted, the value is formatted in UTC. ##### Common format codes These are the most useful `strftime` codes for the `date` filter. For the complete list, see any [standard `strftime` reference](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes): | Code | Meaning | Example | |------|---------|---------| | `%A` | Weekday name | `Monday` | | `%a` | Abbreviated weekday | `Mon` | | `%B` | Month name | `February` | | `%b` | Abbreviated month | `Feb` | | `%d` | Day of month (zero-padded) | `24` | | `%Y` | Four-digit year | `2025` | | `%I:%M %p` | 12-hour time | `11:04 AM` | | `%H:%M` | 24-hour time | `16:04` | | `%Z` | Timezone abbreviation | `PST` | | `%z` | UTC offset | `-0800` | Combine codes with any literal text, e.g. `%A, %B %d at %I:%M %p %Z`. If a timezone is unknown, a format string is invalid, or a variable is unrecognized, the template placeholder is left untouched rather than rendered to an empty or error value. ## Customer-defined dynamic variables You can also define your own variables. There are several ways to resolve a customer-defined dynamic variable before a conversation. They follow this order of precedence. **1. Pass the variables via the outbound API call** ```bash curl --request POST \ --url https://api.telnyx.com/v2/texml/ai_calls/ \ --header "Authorization: Bearer $TELNYX_API_KEY" \ --header 'Content-Type: application/json' \ --data '{ "From": "+13128675309", "To": "+15551234567", "AIAssistantId": "assistant-6207ab25-b185-478f-b2ef-85159e226727", "AIAssistantDynamicVariables": { "full_name": "James Smith", "facility_name": "Cleveland Clinic Main Campus", "facility_department": "Emergency Department" } }' ``` ![AI Assistant Outbound Call Variables](/img/call-dynamic-vars.png) **2. Custom SIP Headers** Custom SIP headers using the `X-` prefix will be mapped to dynamic variables. Header names are case-insensitive, and `-` in the header names will be replaced by `_` in the variable names. For instance, `X-Full-Name` will be resolved as `{{full_name}}`. *Telnyx reserves the `{{telnyx_}}` namespace for system variables, meaning SIP headers beginning `X-Telnyx` will not be resolved to dynamic variables.* Here you can see an example of passing a dynamic variable to another AI Assistant using custom SIP headers in the Transfer tool. ![AI Assistant SIP Header Config](/img/ai-assistant-custom-sip-variable.png) **3. Configure Dynamic Variables Webhook** If the `dynamic_variables_webhook_url` is set for the assistant, we will `POST` the following payload at the start of the conversation. Telnyx will [sign this webhook](https://developers.telnyx.com/development/api-fundamentals/webhooks/receiving-webhooks#webhook-signing) so that the authenticity of the request can be verified. ![AI Assistant Variable Config](/img/configure-dynamic.png) ``` { "data": { "record_type": "event", "id": "event_12345678-90ab-cdef-1234-567890abcdef", "event_type": "assistant.initialization", "occurred_at": "2025-04-07T10:00:00Z", "payload": { "telnyx_conversation_channel": "phone_call", "telnyx_agent_target": "+13128675309", "telnyx_end_user_target": "+15551234567", "telnyx_end_user_target_verified": false, "call_control_id": "v3:u5OAKGEPT3Dx8SZSSDRWEMdNH2OripQhO", "assistant_id": "assistant_12345678-90ab-cdef-1234-567890abcdef" } } } ``` *For inbound phone calls to an assistant, the `telnyx_end_user_target_verified` field will be set to `true` if the call has Full (A) [STIR/SHAKEN attestation](https://support.telnyx.com/en/articles/5402969-stir-shaken-with-telnyx) and Telnyx was able to verify the authenticity of the PASSporT token.* We expect a JSON response with the following structure. If we do not receive this response within a 1-second timeout, the call will proceed "best effort" with the alternatives listed below. All three fields (`dynamic_variables`, `memory`, and `conversation`) are optional. The `dynamic_variables` field sets the values for the specified dynamic variables. You can read more about the `memory` and `conversation` fields in our [tutorial on Memory](https://developers.telnyx.com/docs/inference/ai-assistants/memory). ```json { "dynamic_variables": { "full_name": "Rachel Thomas", "facility_name": "UCHealth", "facility_department": "Cardiology", "account_number": "ACC-12345", "appointment_time": "2:30 PM" }, "memory": { "conversation_query": "metadata->telnyx_end_user_target=eq.+13128675309&limit=5&order=last_message_at.desc" }, "conversation": { "metadata": { "customer_tier": "premium", "preferred_language": "en", "timezone": "America/Denver" } } } ``` **4. Configure default dynamic variables in the Assistant builder** In the above view, you can also set default values for dynamic variables at the agent level. These will serve as a fallback if any of the above mechanisms do not properly resolve a variable. **5. Unset variables** Variables not set by any of the above methods will remain in their raw form (`{{full_name}}`). ## Troubleshooting Dynamic variables can sometimes fail to resolve or behave unexpectedly. This section provides comprehensive guidance for identifying and resolving common issues. ### Common issues #### Variables not resolving When variables appear as raw text (`{{variable_name}}`) in conversations, check these potential causes: - **Webhook timeout**: Your webhook endpoint took longer than 1 second to respond, causing fallback to defaults. - **Variable name mismatch**: Typos or case sensitivity differences between variable definition and usage. - **Incorrect precedence**: Understanding the resolution order (API > SIP headers > webhook > defaults). - **Missing configuration**: Variable not defined in any resolution method. #### Incorrect formatting Variables may not work correctly due to formatting issues: - **Invalid JSON**: Webhook responses contain malformed JSON syntax. - **Wrong data types**: Sending numbers instead of strings or vice versa. - **Missing required fields**: Webhook response missing `dynamic_variables` object. - **Template syntax errors**: Incorrect mustache template usage (`{variable}` instead of `{{variable}}`). #### Timeout issues Webhook timeouts are a common source of variable resolution failures: - **Slow database queries**: Optimize database lookups in webhook endpoints. - **Network connectivity**: Check firewall rules and network connectivity to webhook URLs. - **External service delays**: Implement proper timeout handling for third-party API calls. - **Processing overhead**: Minimize computation time in webhook response generation. ### Debugging checklist Follow this step-by-step process to diagnose variable issues: #### 1. Verify variable definition - Check variable name spelling and case sensitivity. - Confirm variables are used in supported fields (`instructions`, `greeting`, `tools`). - Validate mustache syntax: `{{variable_name}}`. - Ensure variable names don't use reserved `telnyx_` prefix. #### 2. Check resolution priority Test variables using the resolution hierarchy: ```bash # Test API injection (highest priority) curl --request POST \ --url https://api.telnyx.com/v2/texml/ai_calls/ \ --header "Authorization: Bearer $TELNYX_API_KEY" \ --header 'Content-Type: application/json' \ --data '{ "From": "+13128675309", "To": "+15551234567", "AIAssistantId": "assistant-6207ab25-b185-478f-b2ef-85159e226727", "AIAssistantDynamicVariables": { "test_variable": "API_VALUE" } }' ``` - Verify SIP header format for transfer scenarios: `X-Variable-Name` becomes `{{variable_name}}`. - Confirm webhook endpoint is reachable and responding correctly. - Set default values in Assistant builder as fallback. #### 3. Validate webhook implementation - Test webhook endpoint independently using tools like Postman. - Verify JSON response format matches expected structure. - Check response time (must be under 1 second). - Validate all required payload fields are present. #### 4. Test in Portal - Use Assistant builder default values for initial testing. - Review conversation logs for variable resolution behavior. - Check webhook logs for errors, timeouts, or unexpected responses. ### Webhook debugging #### Using webhook logs effectively To facilitate troubleshooting webhook issues, logs are viewable per conversation in the portal alongside the conversation transcript and insights. ![AI Assistant Webhook Logs](/img/dynamic-webhook-log.png) **Understanding webhook logs:** - **Request logs**: Show the exact payload Telnyx sent to your webhook - **Response logs**: Display your webhook's response and any errors - **Timing information**: Reveals if responses exceeded the 1-second timeout - **Error details**: Provide specific error messages for failed requests #### Common webhook issues **Authentication failures:** ```json // Webhook response indicating auth failure { "error": "Unauthorized: Invalid API key", "dynamic_variables": {} } ``` **Malformed JSON responses:** ```json // Incorrect - missing quotes around keys { dynamic_variables: { customer_name: "John Doe" } } // Correct JSON format { "dynamic_variables": { "customer_name": "John Doe" } } ``` **Timeout handling:** ```json // Webhook response for timeout scenarios { "dynamic_variables": { "customer_name": "Default Name" }, "message": "External service timeout, using cached data" } ``` #### Testing webhook endpoints **Independent webhook testing:** ```bash # Test your webhook with the exact payload Telnyx sends curl --request POST \ --url https://your-webhook-url.com/dynamic-variables \ --header 'Content-Type: application/json' \ --data '{ "data": { "record_type": "event", "id": "event_12345678-90ab-cdef-1234-567890abcdef", "event_type": "assistant.initialization", "occurred_at": "2025-04-07T10:00:00Z", "payload": { "telnyx_conversation_channel": "phone_call", "telnyx_agent_target": "+13128675309", "telnyx_end_user_target": "+15551234567", "call_control_id": "v3:test-call-control-id" } } }' ``` ### Variable validation #### Input validation **Sanitize variable values:** ```json // Good - Clean, safe variable values { "dynamic_variables": { "customer_name": "John Smith", "account_number": "ACC-12345", "appointment_time": "2:30 PM" } } // Avoid - Potentially problematic values { "dynamic_variables": { "customer_name": "alert('xss')", "account_number": "'; DROP TABLE users; --", "appointment_time": null } } ``` **Handle special characters properly:** - Escape quotes and backslashes in variable content. - Validate data types match expected formats. - Provide meaningful defaults for empty or null values. #### Response format validation **Ensure proper JSON structure:** ```json // Complete webhook response format { "dynamic_variables": { "customer_name": "Sarah Johnson", "facility_name": "Memorial Hospital" }, "memory": { "conversation_query": "metadata->customer_id=eq.12345" }, "conversation": { "metadata": { "customer_tier": "premium" } } } ``` #### Testing scenarios **Cross-channel testing:** - Verify variables work consistently across phone calls, web calls, and SMS. - Test variable behavior in transfer scenarios using SIP headers. - Validate fallback behavior when webhook endpoints are unavailable. **Load testing:** - Test webhook endpoint performance under expected call volumes. - Verify response times remain under 1 second during peak usage. - Monitor for memory leaks or performance degradation over time. --- ### Conversation Workflows > Source: https://developers.telnyx.com/docs/inference/ai-assistants/workflows.md Conversation workflows let you turn a single AI Assistant into a guided, multi-step experience. Instead of asking one prompt to handle every part of a call or chat, you can design a graph of conversation steps, define when the assistant should move between them, and tune the assistant's behavior at each step. Use workflows when your assistant needs to guide people through a process, such as intake, qualification, booking, verification, escalation, or support triage. ## Why use workflows? A single prompt works well for open-ended conversations. Workflows are better when the customer journey has structure. With workflows, you can: - **Break complex conversations into focused steps**: Give each stage its own name and instructions, such as `Intake`, `Billing`, `Schedule appointment`, or `Escalate to specialist`. - **Route with natural language or deterministic rules**: Move between steps when the LLM decides a condition is met, or when a dynamic variable matches a configured value. - **Tune behavior per step**: Append or replace the assistant's main instructions, and optionally override the model or voice for a specific node. - **Route to another assistant**: Route from one workflow to another assistant when the conversation should be handled by a different specialist. - **Debug what happened later**: Conversation transcripts can show the workflow node associated with assistant messages, so you can connect real conversations back to the workflow design. ## How workflows work A workflow is a directed graph stored on the AI Assistant as `conversation_flow`. The graph has two main building blocks: - **Nodes**: Conversation steps. A node is either a **prompt node** (an LLM-driven step with its own label, instructions, instruction mode, model override, and voice override) or a **speak node** (a deterministic step that plays a fixed scripted message with no LLM turn). - **Edges**: Transitions between nodes, or from a node to another assistant. Each edge has a condition that decides when that path should be taken. Conditions can be natural-language (**LLM**), deterministic (**variable comparison**), or a **default** fallback. When a conversation starts, Telnyx begins at the workflow's start node. After the assistant responds, Telnyx evaluates the outgoing edges from the active node. If a condition matches, the conversation moves to the target node or assistant. If no condition matches, the conversation stays on the current node — except for speak nodes, which always advance along their required default edge after delivering the message. Workflows are optional. Assistants without a workflow continue to use their standard assistant-level instructions, model, voice, tools, and settings. ## What you can build with workflows Conversation workflows support the building blocks you need for guided, adaptive customer journeys: - **Start from a defined entry point**: Choose which node begins the workflow. - **Create focused conversation stages**: Add prompt nodes for each step and give each one its own instructions. - **Play scripted lines verbatim**: Add speak nodes for greetings, disclosures, or compliance statements that must be delivered word-for-word, with no model turn. - **Control how prompts combine**: Append node instructions to the assistant's base instructions, or replace the base instructions for a specific step. - **Tune individual steps**: Override the model, voice, tools, or transcription behavior for a node when that step needs different capabilities. - **Connect steps with conditional routing**: Use LLM conditions for natural-language decisions and variable comparisons for deterministic decisions. - **React to tool outcomes**: Route based on whether a configured tool succeeds or fails. - **Move across assistants**: Send a conversation from one workflow into another assistant when a specialist configuration should take over. - **Review workflow context in transcripts**: Use conversation history to understand which workflow step produced assistant messages. ## Create a workflow in the Portal 1. Open **AI Assistants** in the Telnyx Portal. 2. Select an assistant. 3. Open the **Workflow** tab. 4. Enable workflows if the assistant does not already have one. 5. Add nodes for each major stage of the conversation. 6. Connect nodes with edges and configure the condition for each edge. 7. Save the assistant. The workflow canvas lets you pan, zoom, drag nodes, and inspect nodes or edges from the side panel. ![Workflow editor for a Front Desk Receptionist assistant showing a start node, transfer node, FAQ node, conditional edges, and a node inspector panel](/img/ai-assistant-workflow-front-desk-editor.png) For example, a front desk assistant can start at **Greeting & Identify Intent**, route office-hours questions to **Answer FAQ**, and route callers who need a person or department to **Transfer Call**. If an assistant is still using the legacy handoff tool flow, the Portal shows the legacy handoff graph instead of the new workflow editor. Remove or migrate the legacy handoff setup before building a new conversation workflow for that assistant. ## Configure workflow nodes A workflow node represents one active stage of the conversation. ### Node types Workflows support two kinds of nodes: - **Prompt node**: An LLM-driven step. The assistant generates its response from the node's instructions (combined with the assistant's base instructions), using the node's model, voice, and tool settings. Most workflow steps are prompt nodes. This is the default node type. - **Speak node**: A deterministic step that plays a fixed, scripted message and then advances. The assistant does **not** call the model on a speak node, so the wording is exactly what you configure. Use speak nodes for greetings, disclosures, hold messages, compliance statements, or any line that must be delivered verbatim. Add a node from the **Add Node** menu and choose **Prompt node** or **Speak node**. You can also drag an edge from an existing node to an empty area of the canvas and pick the node type from the menu. #### Speak node message A speak node centers on a single **Message** field that defines the line to deliver. - The message is delivered verbatim — there is no model turn, so the caller hears exactly what you type. - The message supports `{{variable}}` placeholders. Both system variables and custom dynamic variables defined on the assistant are interpolated at runtime, with inline highlighting and autocomplete in the editor. - A speak node must always have **exactly one outgoing [default edge](#default-conditions)**. Because a speak node does not make a routing decision itself, it advances along this default edge after delivering the message. ```text Thanks for calling Acme, {{first_name}}. This call may be recorded for quality and training. ``` The message uses the assistant's (or flow's) configured voice to deliver the scripted line. The instructions, tool availability, model, and voice settings described below apply to **prompt nodes**, which generate their response with the model. ### Node name Use short, descriptive names. Node names are visible in the workflow canvas and can appear in conversation transcript context. Good node names: - `Greeting & Identify Intent` - `Answer FAQ` - `Transfer Call` - `Collect appointment details` ### Node instructions Each node has instructions that control the assistant while that node is active. You can choose how the node instructions combine with the assistant's main instructions: - **Append to assistant instructions**: Keep the assistant's base behavior and add step-specific guidance. - **Replace assistant instructions**: Use only the node's instructions for this step. Append mode is usually the safest default because it preserves global policies, tone, and business rules. Replace mode is useful for tightly-scoped steps where the assistant should follow a very different prompt. ### Tool availability Each node controls which of the assistant's tools the model can call while that node is active. This lets you configure every tool once on the assistant, then expose only the relevant subset at each step. A node's **Tools** tab shows two groups: - **Inherited tools**: Every tool configured on the assistant. Each tool has a toggle so you can enable or disable it for this specific node. Tools are enabled by default, so a new node starts with the full assistant toolset until you turn tools off. - **Added for this node**: Tools attached to this node only. Use the **Select a new tool** dropdown to add a node-specific tool that should not be available elsewhere in the workflow. For example, a `Take Message` node might disable the **Transfer** tool while keeping a `take-message` webhook and a **Hang Up** tool enabled, so the assistant can only capture and close out the message during that step. ![Workflow node inspector open to the Tools tab for a Take Message node, showing inherited tools with per-node toggles (Transfer off, take-message webhook and Hang Up on) and a Select a new tool dropdown](/img/ai-assistant-workflow-node-tools.png) Scoping tools per node is one of the most effective ways to make each step reliable. When a node only exposes the tools that fit its job, the model has fewer choices to weigh, calls the right tool more consistently, and is far less likely to fire an unrelated action (such as transferring a caller during an FAQ answer). ### Model override A node can use the assistant's default LLM, or override it with another Telnyx-supported model. Use model overrides when one step needs different reasoning or latency characteristics. For example: - Use a faster model for intake. - Use a stronger model for complex qualification or policy-heavy decisions. - Keep the assistant default everywhere except one high-value decision point. ### Voice override A node can inherit the assistant's default voice, or use a node-specific voice configuration. Voice overrides are useful when different conversation stages should feel different. For example: - Use the standard brand voice during greeting and intent detection. - Switch to a calmer voice for sensitive support flows. - Use a different voice when routing to a specialist assistant persona. ## Configure workflow edges An edge defines where the conversation can go next. Each edge has: - **Source node**: The node the conversation is leaving. - **Target type**: Another workflow node, or another assistant. - **Condition type**: The logic that decides whether the edge should be followed — an **LLM** condition, a **variable comparison**, or a **default** fallback. ### LLM conditions Use an LLM condition when the routing decision depends on conversation meaning. Example conditions: ```text The caller has a general office-hours, location, or FAQ question. ``` ```text The caller wants to speak with a specific person or department. ``` ```text The user has asked to speak with a human agent. ``` LLM conditions are best for intent, sentiment, completeness, and other natural language judgments. ### Variable comparison conditions Use a variable comparison when the routing decision should be deterministic. Variable comparison conditions can use system variables and custom dynamic variables defined on the assistant. Examples: - `telnyx_conversation_channel == "phone_call"` - `customer_tier == "enterprise"` - `telnyx_conversation_duration_secs >= 30` - `telnyx_shaken_stir_attestation != "a"` Variable comparisons are best for account state, channel-specific behavior, elapsed conversation time, authentication flags, or data returned by a dynamic variables webhook. ### Default conditions A **default** condition is a fallback edge that is followed whenever no other outgoing edge's condition matches. It has no prompt or expression to configure — it simply defines where the conversation goes by default. Default conditions are required for nodes that do not make their own routing decision: - A **speak node** delivers a scripted message and then advances. It must have **exactly one** outgoing default edge so the conversation always has a defined next step. - Default conditions are only valid on edges that **leave a speak node**. They are not used on edges leaving a prompt node, which route based on LLM or variable comparison conditions. When you draw the first edge out of a speak node in the Portal, it is automatically created as a default edge. If a speak node already has its default edge, any additional edges you draw fall back to an LLM condition that you can configure. A speak node with zero default edges, or more than one, is invalid and cannot be saved. Make sure every speak node has exactly one outgoing default edge. ## Configure workflows with the API You can also configure workflows programmatically through the [Assistants API](/api-reference/assistants/create-an-assistant). Workflows are stored on the assistant as `conversation_flow`. The API accepts the full workflow graph when you create or update an assistant. To change one node or edge, send the updated `conversation_flow` object with the assistant update request. Assistant updates treat `conversation_flow` atomically. If you omit the field, the existing workflow is unchanged. If you send `conversation_flow: null`, the workflow is cleared. A simplified workflow payload looks like this: ```json { "conversation_flow": { "start_node_id": "n_greeting", "nodes": [ { "id": "n_greeting", "name": "Greeting & Identify Intent", "instructions": "Quickly determine whether the caller needs a person, a department, or a general FAQ answer.", "instructions_mode": "append" }, { "id": "n_faq", "name": "Answer FAQ", "instructions": "Answer hours, location, and general information questions. If they need more detail, offer to connect them to the right person.", "instructions_mode": "append" }, { "id": "n_transfer", "name": "Transfer Call", "instructions": "Confirm the destination department or person, then verbally confirm the transfer before connecting the caller.", "instructions_mode": "append" } ], "edges": [ { "id": "e_greeting_to_faq", "start_node_id": "n_greeting", "target": { "type": "node", "node_id": "n_faq" }, "condition": { "type": "llm", "prompt": "The caller has a general office-hours, location, or FAQ question." } }, { "id": "e_greeting_to_transfer", "start_node_id": "n_greeting", "target": { "type": "node", "node_id": "n_transfer" }, "condition": { "type": "llm", "prompt": "The caller wants to speak with a specific person or department." } } ] } } ``` ### Speak nodes and default edges in the API A speak node uses `"type": "speak"` and carries its scripted text in the `message` field instead of `instructions`. Its single outgoing edge uses a `default` condition. ```json { "conversation_flow": { "start_node_id": "n_disclosure", "nodes": [ { "type": "speak", "id": "n_disclosure", "name": "Recording Disclosure", "message": "Thanks for calling Acme, {{first_name}}. This call may be recorded for quality and training." }, { "type": "prompt", "id": "n_intake", "name": "Identify Intent", "instructions": "Find out what the caller needs and route them accordingly.", "instructions_mode": "append" } ], "edges": [ { "id": "e_disclosure_to_intake", "start_node_id": "n_disclosure", "target": { "type": "node", "node_id": "n_intake" }, "condition": { "type": "default" } } ] } } ``` The prompt node's `"type"` field is optional and defaults to `"prompt"`. A `default` condition takes no `prompt` or `expression`, and is only valid on an edge that leaves a speak node. ## Route to another assistant An edge can target another assistant instead of another node in the same workflow. Use assistant routing when a conversation should move to a different specialist configuration, such as: - A sales assistant routing qualified technical questions to a solutions assistant. - A front-desk assistant routing billing questions to a billing assistant. - A general support assistant routing high-risk cases to a stricter compliance assistant. This keeps each assistant focused while still letting customers move through a connected experience. ## Example workflow patterns ### Front desk receptionist Use one assistant to greet callers, identify intent, and route to the right next step. ```text Greeting & Identify Intent ├── Answer FAQ when the caller asks about hours, location, or general information └── Transfer Call when the caller wants a person or department ``` ### Appointment booking Guide the customer through a sequence of required information before confirmation. ```text Collect request → Collect availability → Confirm details → Final confirmation ``` Use variable comparisons for deterministic gates, such as whether a required dynamic variable exists, and LLM conditions for softer gates, such as whether the customer has verbally confirmed the details. ### Escalation after timeout Use the conversation duration system variable to escalate when the assistant has spent too long in a step. ```text Troubleshooting ├── Continue troubleshooting when the issue is not resolved └── Route to specialist when telnyx_conversation_duration_secs >= 300 ``` ### Multi-assistant specialization Use a workflow edge to route from a general assistant into another assistant. ```text Main support assistant ├── Billing assistant ├── Technical support assistant └── Sales assistant ``` This pattern is useful when each destination needs its own model, voice, instructions, tools, or operational ownership. ## Test and debug workflows After saving a workflow, test the assistant with realistic conversations that exercise each route. When reviewing conversation history: - Check whether the assistant followed the expected path. - Look for assistant messages labeled with workflow node context. - Open the workflow from transcript context when you need to inspect the node that produced a response. - Review dynamic variable values and webhook behavior if a variable comparison did not route as expected. In this example, the assistant starts in **Greeting & Identify Intent**, the caller asks about hours, and the workflow routes the response through **Answer FAQ**. ![Conversation transcript showing a Front Desk Receptionist workflow moving from Greeting & Identify Intent to Answer FAQ after the user asks about hours](/img/ai-assistant-workflow-front-desk-transcript.png) ## Best practices ### Keep nodes focused Each node should represent one clear job. If a node's instructions cover multiple unrelated tasks, split it into separate nodes. ### Limit each node to the tools it needs Configure all of your tools on the assistant, then disable the ones that do not apply on each node. A focused, node-specific toolset gives the model a clear purpose, reduces the chance it calls the wrong tool, and keeps each step predictable. As a rule of thumb, only leave a tool enabled on a node if that step is supposed to be able to use it. ### Prefer append mode for global rules Use append mode when the node should keep the assistant's normal safety rules, brand voice, and business constraints. Use replace mode only when the node truly needs a standalone prompt. ### Write edge conditions as clear decisions For LLM conditions, describe the moment when the edge should fire. Avoid vague conditions like `billing`. Prefer explicit criteria, such as `The caller has a general office-hours, location, or FAQ question.` ### Use variable comparisons for hard rules If the condition depends on structured data, use a variable comparison instead of an LLM condition. This makes routing predictable and easier to debug. ### Avoid too many paths from one node A node with many outgoing edges is harder to reason about and test. If the routing logic gets complex, add an intermediate triage node. ### Test every path before production Run through the happy path, fallback path, escalation path, and at least one negative case for every important node. ## Related resources - [Dynamic Variables](/docs/inference/ai-assistants/dynamic-variables): Personalize prompts and route with runtime data. - [Version Testing and Traffic Distribution](/docs/inference/ai-assistants/version-testing-traffic-distribution): Test assistant changes before sending all traffic to a new version. - [Integrations](/docs/inference/ai-assistants/integrations): Connect assistants to external systems and data sources. --- ### Async Tools > Source: https://developers.telnyx.com/docs/inference/ai-assistants/async-tools.md Async tools allow your AI assistant to trigger long-running operations without blocking the conversation. Combined with the [Add Messages API](https://developers.telnyx.com/api-reference/call-commands/add-messages-to-ai-assistant), you can inject results back into the conversation whenever they're ready—whether that's 5 seconds or 5 minutes later. In this guide, you will learn: - How to configure webhook tools to run asynchronously - How to use the Add Messages API to inject context mid-conversation - How to combine both features for powerful async workflows --- ## Overview Traditional webhook tools block the conversation until they complete. This works fine for fast operations, but creates awkward pauses for slow backend queries. Async tools solve this by letting the assistant continue the conversation while operations run in the background. ### The two building blocks These features are **orthogonal**—each is useful on its own, but they become especially powerful when combined. | Feature | What it does | Use alone | |---------|--------------|-----------| | **Async webhook flag** | Lets the assistant continue talking while the webhook executes | Fire-and-forget operations (logging, notifications) | | **[Add Messages API](https://developers.telnyx.com/api-reference/call-commands/add-messages-to-ai-assistant)** | Injects new context into an active conversation | External triggers, scheduled reminders, supervisor interventions | ### Combined workflow When used together, these features enable a new pattern: 1. Assistant triggers an async webhook (e.g., order lookup) 2. Assistant continues chatting with the customer 3. Backend processes the request (5, 10, 30 seconds later) 4. Backend calls Add Messages API to inject the results 5. Assistant naturally incorporates the new information This creates a seamless experience where the assistant stays engaged while slow operations complete in the background. --- ## Async webhooks The `async` flag on webhook tools tells the assistant not to wait for the response. The webhook fires, and the assistant immediately continues the conversation. ### Configuring an async webhook Set `async: true` in your webhook tool configuration: ```json { "type": "webhook", "webhook": { "name": "lookup_order_status", "description": "Triggers an async order status lookup. Results will be delivered automatically when ready.", "url": "https://your-backend.com/order-lookup", "method": "POST", "async": true, "headers": [ {"name": "Content-Type", "value": "application/json"} ], "body_parameters": { "type": "object", "properties": { "order_id": { "type": "string", "description": "The customer's order ID" } }, "required": ["order_id"] } } } ``` ![Async webhook configuration in Portal](/img/async-webhook-config.png) ### Key configuration options | Field | Description | |-------|-------------| | `async` | When `true`, the assistant continues without waiting for a response | | `url` | Your backend endpoint that will process the request | | `method` | HTTP method (typically `POST`) | | `body_parameters` | JSON schema defining the parameters the assistant should provide | For the complete webhook tool schema, see the [Create Assistant API reference](https://developers.telnyx.com/api-reference/assistants/create-an-assistant). ### What your backend receives When the assistant triggers an async webhook, your endpoint receives: - The configured body parameters (e.g., `order_id`) - The `x-telnyx-call-control-id` header identifying the active call ``` POST /order-lookup HTTP/1.1 Content-Type: application/json x-telnyx-call-control-id: v3:abc123def456... { "order_id": "ORD-12345" } ``` The `x-telnyx-call-control-id` header is critical—you'll need it to inject results back into the conversation using the [Add Messages API](https://developers.telnyx.com/api-reference/call-commands/add-messages-to-ai-assistant). --- ## Add Messages API The [Add Messages API](https://developers.telnyx.com/api-reference/call-commands/add-messages-to-ai-assistant) lets you inject new messages into an active conversation from outside the call flow. This is useful for delivering async results, supervisor interventions, or external triggers. ### API endpoint ``` POST /v2/calls/{call_control_id}/actions/ai_assistant_add_messages ``` ### Request format ```bash curl -X POST "https://api.telnyx.com/v2/calls/{call_control_id}/actions/ai_assistant_add_messages" \ -H "Authorization: Bearer $TELNYX_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "messages": [ { "role": "system", "content": "Order ORD-12345 status: SHIPPED. Tracking: 1Z999AA10123456784. Estimated delivery: Tomorrow. Share this with the customer now." } ] }' ``` For the complete API specification, see the [Add Messages API reference](https://developers.telnyx.com/api-reference/call-commands/add-messages-to-ai-assistant). ### Message roles | Role | Use case | |------|----------| | `system` | Instructions or context for the assistant (recommended for async results) | | `user` | Simulate user input | | `assistant` | Inject assistant responses | ### Standalone use cases The Add Messages API is valuable even without async webhooks: - **Supervisor intervention**: A human supervisor injects guidance during a difficult call - **Scheduled reminders**: External system reminds the assistant about time-sensitive information - **Cross-system triggers**: CRM or ticketing system pushes updates to an active call - **Escalation prompts**: Monitoring system detects frustration and injects de-escalation guidance --- ## Combining async webhooks with Add Messages The real power comes from combining these features. Here's a complete example of an async order lookup system. ### Architecture ``` ┌─────────────┐ 1. Trigger async webhook ┌─────────────────┐ │ │ ──────────────────────────────────▶│ │ │ Assistant │ │ Your Backend │ │ │◀────────────────────────────────── │ │ └─────────────┘ 4. Add Messages API └─────────────────┘ │ ▲ │ │ │ │ ▼ │ ▼ 2. Continue 3. Process request Query databases, conversation (5-30 seconds) external APIs, etc. ``` ### Step 1: Configure the assistant Create an assistant with async webhook tools. Notice how the instructions tell the assistant to continue engaging while waiting: ```json { "name": "Customer Service Agent", "instructions": "You are a helpful customer service agent for Acme Electronics.\n\nWhen a customer asks about an order, trigger the lookup_order_status tool. This runs asynchronously—results will arrive automatically in 10-20 seconds.\n\nAfter triggering the lookup, keep the customer engaged:\n- Mention current promotions\n- Ask about their experience\n- Offer to help with anything else\n\nWhen results arrive, naturally incorporate them: \"Great news, I have your order info now!\"", "tools": [ { "type": "webhook", "webhook": { "name": "lookup_order_status", "description": "Async order lookup. Results delivered automatically when ready.", "url": "https://your-backend.com/order-lookup", "method": "POST", "async": true, "body_parameters": { "type": "object", "properties": { "order_id": {"type": "string", "description": "Order ID to look up"} }, "required": ["order_id"] } } } ] } ``` ### Step 2: Build the backend service Your backend receives the webhook, processes the request, and calls the [Add Messages API](https://developers.telnyx.com/api-reference/call-commands/add-messages-to-ai-assistant) when done: ```python import os import time import requests from flask import Flask, request, jsonify app = Flask(__name__) TELNYX_API_KEY = os.environ.get("TELNYX_API_KEY") @app.route("/order-lookup", methods=["POST"]) def order_lookup(): data = request.get_json(silent=True) or {} # Get the call control ID from headers call_control_id = request.headers.get("x-telnyx-call-control-id") order_id = data.get("order_id") if not call_control_id: return jsonify({"error": "Missing call control ID"}), 400 # Simulate slow backend query (replace with real logic) time.sleep(15) # Build the result message result = { "status": "SHIPPED", "tracking": "1Z999AA10123456784", "delivery": "Tomorrow" } system_message = f"""[ORDER LOOKUP COMPLETE] Order {order_id}: {result['status']} Tracking: {result['tracking']} Estimated delivery: {result['delivery']} Share these details with the customer now.""" # Inject results back into the conversation inject_message(call_control_id, system_message) return jsonify({"status": "sent"}) def inject_message(call_control_id: str, message: str): """Send a message to an active conversation via Add Messages API.""" url = f"https://api.telnyx.com/v2/calls/{call_control_id}/actions/ai_assistant_add_messages" response = requests.post( url, headers={ "Authorization": f"Bearer {TELNYX_API_KEY}", "Content-Type": "application/json" }, json={ "messages": [{"role": "system", "content": message}] } ) return response.json() if __name__ == "__main__": app.run(host="0.0.0.0", port=8000) ``` ### Step 3: Test the flow 1. Call your assistant and ask about an order 2. The assistant triggers the async lookup and continues chatting 3. After 15 seconds, your backend injects the results 4. The assistant seamlessly shares the order details ![Conversation transcript showing async message injection](/img/async-transcript-injection.png) --- ## Multiple parallel lookups You can trigger multiple async webhooks simultaneously. Each completes independently and injects results as they become available. ### Example: Staggered results Configure multiple tools with different backend processing times: | Tool | Processing time | Information returned | |------|----------------|---------------------| | `check_loyalty_points` | ~10 seconds | Points balance, membership tier | | `lookup_order_status` | ~20 seconds | Order status, tracking, delivery estimate | The assistant triggers both at once. Results drip into the conversation naturally: ``` Customer: "Where's my order 12345?" Agent: [Triggers both lookups] "Let me pull that up for you! By the way, we're running 20% off all accessories this week." [10 seconds pass - loyalty results arrive] Agent: "Oh nice, I see you have 2,500 reward points - that's Gold status! You've got $25 to use on your next purchase." [20 seconds pass - order results arrive] Agent: "And here's your order info - order 12345 is out for delivery! Tracking number is 1Z999AA10123456784, should arrive tomorrow." ``` ### Instructing the assistant For parallel lookups to work well, your assistant instructions should emphasize calling tools together: ``` When a customer asks about an order, trigger BOTH lookup tools at the same time: 1. check_loyalty_points 2. lookup_order_status Do not wait for one to complete before calling another. Call both immediately. Results will arrive automatically as each lookup completes. ``` --- ## Best practices ### Crafting system messages When injecting results via the [Add Messages API](https://developers.telnyx.com/api-reference/call-commands/add-messages-to-ai-assistant), format them clearly: ```python # Good - Clear, actionable system_message = """[ORDER LOOKUP COMPLETE] Order ORD-12345: SHIPPED Tracking: 1Z999AA10123456784 Estimated delivery: Tomorrow Share these details with the customer now.""" # Avoid - Ambiguous system_message = "The order was found in the system." ``` ### Handling edge cases **Call ended before results arrive:** ```python response = inject_message(call_control_id, message) if response.get("status_code") == 404: # Call has ended, log and move on print(f"Call {call_control_id} already ended") ``` **Multiple results for same lookup:** - Include identifiers in messages so the assistant knows which query the results belong to - Use timestamps or request IDs if needed ### Backend considerations - Your backend should return a 200 response quickly to acknowledge receipt - Process the actual work asynchronously (use background workers, Celery, AWS Lambda, etc.) - There's no timeout constraint on async webhooks—your backend can take as long as needed before calling the Add Messages API ### Testing tips - Use tools like ngrok to expose local backends during development - Log all headers to verify `x-telnyx-call-control-id` is received - Test with various delay lengths to ensure natural conversation flow - Monitor the conversation transcript in the Portal to see messages being injected --- ## Use cases ### Customer service - **Order lookups**: Query multiple systems (warehouse, shipping, payments) in parallel - **Account reviews**: Pull account history, loyalty status, and recent tickets simultaneously - **Product availability**: Check inventory across multiple warehouses ### Healthcare - **Patient record retrieval**: Fetch records from multiple systems while confirming appointment details - **Insurance verification**: Run eligibility checks while gathering patient information - **Lab results**: Query lab systems and deliver results when ready ### Financial services - **Loan pre-qualification**: Run credit checks and affordability calculations in background - **Account aggregation**: Pull balances from multiple accounts simultaneously - **Fraud alerts**: Inject real-time fraud warnings from monitoring systems ### Scheduling - **Multi-calendar availability**: Check availability across multiple calendars/resources - **Booking confirmations**: Process reservations and inject confirmation details - **Waitlist updates**: Notify assistant when spots become available --- ## Related resources - **[Add Messages API Reference](https://developers.telnyx.com/api-reference/call-commands/add-messages-to-ai-assistant)** - Complete API specification for injecting messages - **[Create Assistant API Reference](https://developers.telnyx.com/api-reference/assistants/create-an-assistant)** - Full webhook tool configuration options - **[Webhooks & Workflows](/docs/inference/ai-assistants/workflows)** - Learn more about configuring webhook tools - **[Dynamic Variables](/docs/inference/ai-assistants/dynamic-variables)** - Pass context into conversations at start time - **[Memory](/docs/inference/ai-assistants/memory)** - Persist information across conversations --- ### Tools Library > Source: https://developers.telnyx.com/docs/inference/ai-assistants/tools-library.md The Tools Library lets you create tools in a shared library and assign them to any assistant. Previously, tools were tied to individual assistants — if multiple assistants needed the same tool, you had to recreate it each time. --- ## Overview With the Tools Library, you define a tool once and reuse it across all your assistants. Updating a shared tool keeps behavior consistent across every assistant that uses it. --- ## Capabilities - **Shared tools** — Create tools in a central library and assign them to any assistant. - **Reduced duplication** — Eliminates recreating identical tools across assistants with similar workflows. - **Centralized configuration** — Update a tool in one place and maintain consistency across assistants. - **Faster deployment** — Speed up assistant creation using prebuilt tools from the library. --- ## Getting started 1. Log in to the [Mission Control Portal](https://portal.telnyx.com). 2. Navigate to **AI, Storage and Compute** > **[AI Tools](https://portal.telnyx.com/#/ai/tools)**. 3. Click **Create New Tool** to add a new tool to the library. 4. When building or editing an assistant, assign tools from the library instead of creating them inline. ![Tools Library overview](/img/tools-library-overview.png) You can also assign library tools directly from the assistant builder. --- ## Migrating existing tools Existing assistants with legacy (inline) tools continue to work. You can migrate legacy tools to the library at your own pace. Migration is optional. Legacy tools remain functional. You can migrate tools one at a time as needed. --- ### Agent Handoff > Source: https://developers.telnyx.com/docs/inference/ai-assistants/agent-handoff.md Agent handoff enables your AI assistant to seamlessly transfer conversations to other specialized AI assistants while preserving full context. This allows you to build a team of expert agents, each focused on specific domains or tasks, working together to provide comprehensive support in a single conversation. In this guide, you will learn how to: - Configure agent handoff for your AI assistants. - Choose between Unified and Distinct handoff modes. - Design effective multi-agent architectures. - Implement agent handoff via API and Portal. --- ## Overview Agent handoff represents a powerful approach to building sophisticated AI systems. Instead of creating one complex agent that handles everything, you can build a team of specialized agents that collaborate seamlessly, each bringing domain expertise to the conversation. Agent handoff is model-agnostic and works with any AI model supported by Telnyx, including OpenAI GPT models, Meta Llama models, Anthropic Claude, Qwen, and others. Each agent in the handoff chain can use a different model based on its specialized needs. ### How agent handoff works The agent handoff lifecycle follows these steps: 1. **Detection**: The current agent identifies that another agent would be better suited to handle the user's request based on intent, domain, or task requirements. 2. **Agent selection**: The system determines which specialist agent should receive the handoff. 3. **Context transfer**: Full conversation history, user data, and relevant context is transferred to the target agent. 4. **Transition**: The handoff occurs, either seamlessly (Unified mode) or explicitly (Distinct mode). 5. **Continuation**: The target agent continues the conversation with full context awareness. ### Unified vs Distinct modes Agent handoff supports two modes that determine how the transition appears to the user: **Unified Mode (Default)** In Unified mode, assistants share the same context and voice, creating a seamless experience where specialists work behind the scenes. The user experiences one consistent agent, even though multiple specialized assistants are handling different parts of the conversation. - **Same voice**: All agents use the same voice configuration. - **Transparent transition**: User doesn't notice the handoff. - **Shared context**: Full conversation history available to all agents. - **Use case**: When you want a unified brand experience. **Distinct Mode** In Distinct mode, each assistant retains its voice configuration, creating a conference call experience with multiple distinct voices. The user hears different voices as they're transferred between specialists. - **Individual voices**: Each agent uses its own voice configuration. - **Explicit transition**: User hears "I'm transferring you to [specialist name]". - **Shared context**: Full conversation history available to all agents. - **Use case**: When you want to highlight specialist expertise. ### Key benefits - **Specialist routing**: Direct queries to domain experts who provide accurate, focused responses. - **Context preservation**: Full conversation history travels with the handoff, eliminating repetition. - **Reduced complexity**: Build focused agents instead of one mega-agent trying to do everything. - **Better user experience**: Users get the right expert for their specific needs. - **Easier maintenance**: Update individual specialist agents without affecting the entire system. - **Scalable architecture**: Add new specialists as your product or service expands. ### Common use cases - **Multi-domain support**: Transfer between technical support, billing, and sales departments with full context. - **Workflow automation**: Information gathering agent → processing agent → confirmation agent. - **Triage and routing**: Assessment agent evaluates needs, then routes to appropriate specialist. - **Language switching**: Detection agent identifies language, hands off to language-specific agent. - **Escalation tiers**: Level 1 support → Level 2 support → Level 3 specialist. - **Task segmentation**: Browse/Research agent → Purchase agent → Post-sale support agent. --- ## Best practices Following these best practices will help you design effective multi-agent systems and maximize the value of agent handoff. ### Agent architecture design **When to split agents vs one agent** Split into multiple agents when: - Domains require significantly different knowledge bases. - Agents need different tools or integrations. - Response patterns differ substantially. - You want to independently update different capabilities. Keep as one agent when: - Tasks are closely related with significant overlap. - Context switching would reduce quality. - User experience benefits from continuity. - The domain is narrow and well-defined. **Specialization patterns** - **By domain**: Technical support, billing, sales, product information. - **By task**: Information gathering, transaction processing, confirmation. - **By customer segment**: VIP customers, trial users, enterprise accounts. - **By complexity**: Simple queries, complex troubleshooting, escalations. - **By channel**: Phone, chat, email (if behavior differs significantly). **Context requirements** Each specialist agent should receive: - Full conversation history with previous agents. - User identification and account information. - Previous interactions and preferences. - Current intent and goals. - Any decisions or commitments made. ### Handoff trigger design **Clear criteria for handoff** Define explicit triggers: - **Intent-based**: "I need to talk to billing" → Billing agent. - **Keyword-based**: User mentions "refund" → Billing agent. - **Task completion**: Information gathered → Processing agent. - **Capability-based**: Technical question beyond triage scope → Technical specialist. - **Sentiment-based**: Frustrated customer → Senior support agent. **Avoiding handoff loops** Prevent agents from endlessly transferring to each other: - Define clear responsibility boundaries for each agent. - Implement handoff history tracking. - Set maximum handoff limits per conversation. - Design failsafe: after N handoffs, route to human agent. - Test handoff decision logic thoroughly. **Graceful transitions** Configure agents to announce handoffs clearly: **Unified mode**: ``` "Let me look into your billing question..." [Handoff to billing agent occurs seamlessly] [Billing agent continues without announcing the transition] ``` **Distinct mode**: ``` "I'm transferring you to Sarah, our billing specialist, who can help with your refund request." [Handoff occurs] "Hi, this is Sarah from billing. I can see you're asking about a refund..." ``` Use the [Transfer Call API](/api-reference/call-commands/transfer-call) to manage transfers between assistants programmatically. ### Context preservation strategies **What context to pass** Essential context to transfer: - **User information**: Name, account ID, customer tier. - **Conversation history**: What was discussed with previous agents. - **User intent**: What the user is trying to accomplish. - **Collected data**: Information gathered (order numbers, issue details, etc.). - **Agent actions**: What previous agents already tried. - **Sentiment**: User's emotional state (frustrated, satisfied, confused). **Using dynamic variables** Leverage dynamic variables to pass structured context: - `{{customer_name}}`, `{{account_id}}`, `{{customer_tier}}`. - `{{issue_type}}`, `{{priority_level}}`. - `{{previous_agent}}`, `{{handoff_reason}}`. - Custom variables specific to your domain. Learn more about [dynamic variables](https://developers.telnyx.com/docs/inference/ai-assistants/dynamic-variables). **Memory configuration** Configure agent memory settings to ensure context persistence: - Enable conversation memory for all agents in the handoff chain. - Use consistent memory keys across agents. - Set appropriate memory retention periods. Learn more about [memory configuration](https://developers.telnyx.com/docs/inference/ai-assistants/memory). ### Industry-specific templates **Healthcare: Triage to Specialist** ``` Scenario: Patient calls about symptoms Triage Agent: - Collects symptoms, medical history, urgency level. - Assesses severity and appropriate specialty. - Gathers insurance information. Handoff Trigger: - Specific condition identified (cardiology, dermatology, etc.). - Severity level requires specialist consultation. Specialist Agent (e.g., Cardiology): - Reviews triage notes and symptoms. - Provides condition-specific guidance. - Schedules appointment with cardiologist. - Explains what to expect. Benefits: - Triage agent handles initial screening efficiently. - Specialist agent provides expert medical guidance. - No need to repeat symptoms or medical history. ``` **E-commerce: Browse → Purchase → Support** ``` Scenario: Customer shopping for products Browse Agent: - Answers product questions. - Provides recommendations. - Helps compare options. - Adds items to cart. Handoff Trigger: User ready to purchase Purchase Agent: - Guides through checkout. - Handles payment processing. - Applies discounts/promo codes. - Confirms order. Handoff Trigger: Post-purchase question Support Agent: - Tracks order status. - Handles returns/exchanges. - Resolves delivery issues. Benefits: - Each agent optimized for specific task. - Smooth shopping experience. - Consistent context across journey. ``` **Financial Services: Authentication → Account → Fraud** ``` Scenario: Customer calling about suspicious activity Authentication Agent: - Verifies customer identity. - Security questions. - Multi-factor authentication. Handoff Trigger: Authentication successful Account Agent: - Accesses account information. - Reviews recent transactions. - Answers general account questions. Handoff Trigger: Suspicious activity detected Fraud Agent: - Investigates flagged transactions. - Freezes card if necessary. - Issues replacement card. - Sets up fraud monitoring. Benefits: - Security handled by specialized agent. - Fraud expertise when needed. - Reduces risk of errors. ``` **Customer Support: Tier 1 → Tier 2 → Tier 3** ``` Scenario: Technical support escalation Tier 1 Agent: - Handles common issues. - Tries basic troubleshooting. - Resolves 70% of queries. Handoff Trigger: Issue not resolved after standard troubleshooting Tier 2 Agent: - Advanced troubleshooting. - System configuration expertise. - Resolves complex technical issues. Handoff Trigger: System-level issue or bug identified Tier 3 Agent (Engineering Specialist): - Deep system knowledge. - Can create bug reports. - Provides workarounds for known issues. - Escalates to development team if needed. Benefits: - Efficient resource utilization. - Expertise matched to complexity. - Full troubleshooting history preserved. ``` --- ## Configuration Agent handoff can be configured through the Portal's AI Assistant builder or via API when creating or updating assistants. ### Portal configuration 1. Navigate to your AI Assistants in the [Telnyx Portal](https://portal.telnyx.com/#/ai/assistants). ![AI Assistants List](/img/ai-assistants-list.png) 2. In the Tools section, add a Handoff tool. ![Add Tool Handoff Menu](/img/add-tool-handoff-menu.png) 3. Choose your voice mode: **Unified** (seamless) or **Distinct** (conference call style). 4. Enter a display name for the target assistant. 5. Select the target assistant from the dropdown menu. 6. Click the plus (+) button to add additional target assistants if needed. 7. Save your configuration. ![AI Assistant Handoff Configuration](/img/handoff-config-page.png) ### How handoff appears to users **Unified Mode Experience:** The user experiences a single, consistent agent voice throughout the conversation. When a handoff occurs, the transition is completely seamless - the agent simply continues helping without announcing any change. ``` User: "I need help with my billing" Triage Agent (Voice A): "I can help you with that. What's your question about billing?" User: "I was charged twice for my last order" [Handoff to Billing Agent occurs silently] Billing Agent (Voice A): "I see you were charged twice. Let me look into that for you..." ``` **Distinct Mode Experience:** The user hears different voices as they're transferred between specialists, creating a conference call experience where each expert introduces themselves. ``` User: "I need help with my billing" Triage Agent (Voice A): "I'll transfer you to Sarah from our billing team who can help." [Handoff occurs] Billing Agent (Voice B): "Hi! This is Sarah from billing. I can see you have a question about being charged twice..." ``` --- ## API implementation ### Creating an assistant with agent handoff When creating an AI assistant via API, include the Handoff tool with target assistant IDs in the tools array: ```bash curl -L 'https://api.telnyx.com/v2/ai/assistants' \ -H 'Content-Type: application/json' \ -H 'Accept: application/json' \ -H 'Authorization: Bearer ' \ --data-raw '{ "name": "Customer Support Triage Agent", "instructions": "You are a triage agent for customer support. Listen to customer needs and determine if they need technical support, billing help, or sales assistance. Handoff to the appropriate specialist when needed.", "model": "moonshotai/Kimi-K2.5", "tools": [ { "type": "handoff", "handoff": { "voice_mode": "unified", "ai_assistants": [ { "name": "Technical Support", "id": "asst_tech_abc123" }, { "name": "Billing Support", "id": "asst_billing_def456" }, { "name": "Sales", "id": "asst_sales_ghi789" } ] } } ] }' ``` ### Distinct mode configuration To enable distinct voice mode where each agent retains its voice configuration: ```bash curl -L 'https://api.telnyx.com/v2/ai/assistants' \ -H 'Content-Type: application/json' \ -H 'Accept: application/json' \ -H 'Authorization: Bearer ' \ --data-raw '{ "name": "Triage Agent", "instructions": "You are a triage agent. When handing off, announce the specialist name.", "model": "moonshotai/Kimi-K2.5", "tools": [ { "type": "handoff", "handoff": { "voice_mode": "distinct", "ai_assistants": [ { "name": "Technical Specialist", "id": "asst_tech_xyz789" } ] } } ] }' ``` ### Updating handoff configuration You can update handoff settings for an existing assistant: ```bash curl -L 'https://api.telnyx.com/v2/ai/assistants/{assistant_id}' \ -X PATCH \ -H 'Content-Type: application/json' \ -H 'Accept: application/json' \ -H 'Authorization: Bearer ' \ --data-raw '{ "tools": [ { "type": "handoff", "handoff": { "voice_mode": "unified", "ai_assistants": [ { "name": "Priority Support", "id": "asst_priority_abc123" }, { "name": "Standard Support", "id": "asst_standard_def456" } ] } } ] }' ``` --- ## Troubleshooting ### Common issues #### Handoff loops **Symptoms**: Agents keep handing off to each other repeatedly, creating an endless loop. **Possible Causes**: - Overlapping agent responsibilities. - Unclear handoff criteria. - Agents don't recognize when they should handle the request. **Solutions**: - Define clear, mutually exclusive responsibility boundaries for each agent. - Document what each agent handles in their instructions. - Add specific instructions: "Only handoff if the request is truly outside your domain". #### Context loss after handoff **Symptoms**: Target agent doesn't have access to previous conversation history or user information. **Possible Causes**: - Dynamic variables webhook not configured for target agent. - Target agent's memory query doesn't include relevant past conversations. - Conversation metadata not being passed consistently. **Solutions**: - Configure dynamic variables webhook for all agents in the handoff chain. - Ensure all agents use the same conversation query logic to access relevant memory. - Use consistent conversation metadata across agents. - Test handoff flows in Portal conversation history. - Verify that dynamic variables are properly passed to target agents. #### Incorrect agent selection **Symptoms**: User gets handed off to wrong specialist, requiring additional transfers. **Possible Causes**: - Triage agent misunderstands user intent. - Agent instructions too vague about when to handoff. - Similar keywords trigger wrong agent selection. **Solutions**: - Refine triage agent instructions with specific examples. - Improve intent detection with more training examples. - Add confirmation step: "It sounds like you need billing help, is that correct?". - Review conversation logs to identify misrouting patterns. - Update agent instructions based on common mistakes. #### Handoff doesn't trigger **Symptoms**: Agent tries to handle request outside its expertise instead of handing off. **Possible Causes**: - Handoff tool not properly configured. - Agent instructions don't specify when to handoff. - Target assistant IDs incorrect or missing. **Solutions**: - Verify handoff tool is added to agent. - Check target assistant IDs are valid. - Add explicit handoff triggers in agent instructions. - Test with direct requests: "I need to talk to billing". - Review agent instructions for conflicting guidance. ### Testing strategies **Test handoff triggers**: - Provide various user requests to verify correct agent selection. - Try edge cases (ambiguous requests, multi-domain questions). - Test with different phrasings of the same intent. **Verify context preservation**: - Check that target agent has full conversation history. - Confirm user doesn't need to repeat information. - Validate that collected data (account numbers, etc.) transfers. **Monitor conversation flows**: - Review conversation transcripts in Portal. - Track average number of handoffs per conversation. - Identify common handoff patterns. - Measure resolution time with vs without handoffs. **Load testing**: - Test with multiple concurrent conversations. - Verify handoffs work reliably under load. - Check that context remains accurate with multiple handoffs. --- ## Related resources - [Workflow](/docs/inference/ai-assistants/workflows) - Visualize agent handoff flows and transitions in your workflow flowchart. - [Dynamic Variables](https://developers.telnyx.com/docs/inference/ai-assistants/dynamic-variables) - Customize context and personalize handoffs. - [Memory](https://developers.telnyx.com/docs/inference/ai-assistants/memory) - Configure persistent context across conversations. - [AI Assistants API Reference](https://developers.telnyx.com/api-reference/assistants/create-an-assistant#create-an-assistant) - Complete API documentation for assistants. - [Voice Assistant Quickstart](https://developers.telnyx.com/docs/inference/ai-assistants/no-code-voice-assistant#handoff) - See handoff tool in Portal. --- ### Voicemail Detection on Transfer > Source: https://developers.telnyx.com/docs/inference/ai-assistants/voicemail-detection-on-transfer.md When a Voice AI Assistant transfers a call, the destination may go to voicemail. Without detection, the caller experiences dead air while the voicemail greeting plays. Voicemail detection on transfer solves this by identifying when a transferred call reaches voicemail and triggering a configured action automatically. The assistant stays on the line with the caller throughout, so there is never a gap in the conversation. --- ## How it works 1. The AI Assistant initiates a transfer using the transfer tool. 2. AMD (Answering Machine Detection) monitors the transfer destination. 3. If voicemail is detected, the configured action triggers immediately. 4. The assistant returns to the caller and continues the conversation. The caller remains connected to the assistant during the entire process. If a human answers the transfer destination, the call proceeds as a normal transfer. --- ## Configuration options ### Detection mode The `detection_mode` field controls whether voicemail detection is active on the transfer: | Value | Description | | --- | --- | | `disabled` | No voicemail detection (default). | | `premium` | ML-based detection with high accuracy. Recommended for production use. | ### Actions when voicemail is detected The `on_voicemail_detected.action` field determines what happens when voicemail is detected: | Action | Behavior | | --- | --- | | `stop_transfer` | Cancels the transfer immediately and returns the assistant to the caller. | | `leave_message_and_stop_transfer` | Delivers a TTS message to the voicemail, then cancels the transfer and returns to the caller. | ### Voicemail message options When using `leave_message_and_stop_transfer`, the `on_voicemail_detected.voicemail_message` object configures what message is left: | Field | Value | Description | | --- | --- | --- | | `type` | `message` | Plays a custom TTS text as the voicemail message. | | `type` | `warm_transfer_instructions` | Uses the warm transfer audio instructions as the voicemail message. | | `message` | string | The TTS text to deliver when `type` is `message`. | --- ## Setting up via the Portal 1. Navigate to **AI, Storage and Compute** in the [Telnyx Portal](https://portal.telnyx.com). 2. Select an existing AI Assistant or create a new one. 3. In the **Tools** section, open the transfer tool configuration. 4. Under **Voicemail Detection**, set the detection mode to **Premium**. 5. Choose your preferred action: **Stop the transfer** or **Leave a message and stop transfer**. 6. If leaving a message, enter the TTS text you want delivered to voicemail. 7. Save and test with a call to a number that goes to voicemail. --- ## Setting up via API Configure voicemail detection within the transfer tool when creating or updating an assistant. ### Stop the transfer on voicemail This configuration cancels the transfer and returns the assistant to the caller when voicemail is detected: ```bash curl -L 'https://api.telnyx.com/v2/ai/assistants' \ -H 'Content-Type: application/json' \ -H 'Accept: application/json' \ -H 'Authorization: Bearer ' \ --data-raw '{ "name": "Support Agent", "instructions": "You are a support agent. Transfer calls to the support team when needed.", "model": "moonshotai/Kimi-K2.5", "tools": [ { "type": "transfer", "transfer": { "targets": [ { "name": "Support Team", "to": "+15551234567" } ], "from": "+15553456789", "voicemail_detection": { "detection_mode": "premium", "on_voicemail_detected": { "action": "stop_transfer" } } } } ] }' ``` ### Leave a message on voicemail This configuration delivers a TTS message to voicemail before returning to the caller: ```bash curl -L 'https://api.telnyx.com/v2/ai/assistants' \ -H 'Content-Type: application/json' \ -H 'Accept: application/json' \ -H 'Authorization: Bearer ' \ --data-raw '{ "name": "Support Agent", "instructions": "You are a support agent. Transfer calls to the support team when needed.", "model": "moonshotai/Kimi-K2.5", "tools": [ { "type": "transfer", "transfer": { "targets": [ { "name": "Support Team", "to": "+15551234567" } ], "from": "+15553456789", "voicemail_detection": { "detection_mode": "premium", "on_voicemail_detected": { "action": "leave_message_and_stop_transfer", "voicemail_message": { "type": "message", "message": "Hello, this is a message from Telnyx support. We attempted to reach you regarding your open ticket. We will try again shortly. Thank you." } } } } } ] }' ``` ### Leave warm transfer instructions on voicemail This configuration uses the warm transfer audio instructions as the voicemail message instead of custom TTS text: ```bash curl -L 'https://api.telnyx.com/v2/ai/assistants' \ -H 'Content-Type: application/json' \ -H 'Accept: application/json' \ -H 'Authorization: Bearer ' \ --data-raw '{ "name": "Support Agent", "instructions": "You are a support agent. Transfer calls to the support team when needed.", "model": "moonshotai/Kimi-K2.5", "tools": [ { "type": "transfer", "transfer": { "targets": [ { "name": "Support Team", "to": "+15551234567" } ], "from": "+15553456789", "voicemail_detection": { "detection_mode": "premium", "on_voicemail_detected": { "action": "leave_message_and_stop_transfer", "voicemail_message": { "type": "warm_transfer_instructions" } } } } } ] }' ``` --- ## Related resources - [Voice AI Assistant API Reference](/api-reference/assistants/create-an-assistant#transfertool) - Complete transfer tool API documentation including voicemail detection parameters. - [Agent Handoff](/docs/inference/ai-assistants/agent-handoff) - Configure AI-to-AI handoffs between specialized assistants. - [Answering Machine Detection](/docs/voice/programmable-voice/answering-machine-detection) - AMD for outbound calls using programmable voice. - [No-Code Voice Assistant](/docs/inference/ai-assistants/no-code-voice-assistant) - Get started with Voice AI Assistants in the Portal. --- ### Scheduled Events > Source: https://developers.telnyx.com/docs/inference/ai-assistants/scheduled-events.md Scheduled events let an AI Assistant kick off an outbound interaction at a fixed point in the future — either a phone call or an SMS message to a specific recipient. The platform stores the event, dispatches it at the scheduled time, and reports the outcome through callbacks. For phone calls, you can also configure automatic retries when the recipient does not engage (busy, no-answer, failed, canceled), so the assistant keeps trying without you having to re-create the event. Typical uses: - Outbound appointment reminders, follow-ups, and renewal calls. - After-hours callbacks scheduled by an inbound assistant. - Automated SMS confirmations queued from a workflow. - Re-attempting a missed call on a configurable interval until it connects. --- ## How it works Each scheduled event flows through a small state machine: 1. **`pending`** — the event has been created and is waiting for its scheduled time. 2. **`in_progress`** — the executor has dispatched the call or SMS to Telnyx. 3. **`completed`** — the recipient was reached (call answered and ended normally, or SMS delivered). 4. **`failed`** — the event could not be completed and no further attempts will be made. For phone calls, the terminal `CallStatus` returned by Telnyx decides whether the event is treated as a success or a retryable failure: | `CallStatus` | Treated as | | --- | --- | | `completed` | Success | | `busy` | Retryable client-side outcome | | `no-answer` | Retryable client-side outcome | | `failed` | Retryable client-side outcome | | `canceled` | Retryable client-side outcome | If retries are configured and budget remains, the event is re-queued with an updated `scheduled_at_fixed_datetime` and returned to `pending`. If the budget is exhausted, the event is marked `failed`. --- ## Configuring retries for external failures Two fields on the create-event request control the retry policy for recipient-side outcomes (busy, no-answer, failed, canceled). | Field | Type | Description | | --- | --- | --- | | `max_retries_client_errors` | integer, `0`–`10` | Number of additional dispatches allowed after the initial attempt. `0` (default) disables retries. | | `retry_interval_secs` | integer, `60`–`86400` | Seconds to wait between attempts. Required whenever `max_retries_client_errors > 0`. | Retries are **phone-call only**. Setting either field on an SMS event is rejected with a 400. ### Validation rules - `retry_interval_secs` must be in the range `60`–`86400` (1 minute to 24 hours). - `max_retries_client_errors` must be in the range `0`–`10`. - If `max_retries_client_errors > 0`, `retry_interval_secs` must also be set. - `retry_interval_secs` is only meaningful when `max_retries_client_errors > 0` — supplying one without the other returns a 400. - Both fields must be omitted (or `0` / `null`) for `sms_chat` events. ### How retries are scheduled When a phone-call event finishes with a retryable `CallStatus` and budget remains: - A new `CallAttempt` record is appended to the event's `call_attempts` array. - `scheduled_at_fixed_datetime` is advanced to **now + `retry_interval_secs`** (not the original time + interval — the clock starts when the previous attempt's terminal status is received). - `status` returns to `pending` and the executor will pick the event up again on its next sweep. When the budget is exhausted, the final attempt is recorded, `status` becomes `failed`, and an `errors` entry explains that the client-error retry budget was exhausted. ### Total attempts `max_retries_client_errors` counts retries **on top of** the initial dispatch. So: - `max_retries_client_errors: 0` → 1 total attempt (no retries). - `max_retries_client_errors: 3` → up to 4 total attempts. - `max_retries_client_errors: 10` → up to 11 total attempts. --- ## Inspecting attempt history Each phone-call event exposes a `call_attempts` array containing one entry per terminal dispatch: ```json { "scheduled_event_id": "8f3c…", "telnyx_conversation_channel": "phone_call", "status": "pending", "max_retries_client_errors": 3, "retry_interval_secs": 300, "call_attempts": [ { "attempt_number": 1, "attempted_at": "2026-05-07T14:30:02.812Z", "call_status": "no-answer", "call_duration": null, "telnyx_call_control_id": "v3:abc…" }, { "attempt_number": 2, "attempted_at": "2026-05-07T14:35:08.114Z", "call_status": "busy", "call_duration": null, "telnyx_call_control_id": "v3:def…" } ] } ``` The audit trail stays on a single row across the entire retry sequence — you do not need to query multiple events to reconstruct what happened. --- ## API reference All endpoints are scoped to a specific assistant. | Method | Path | Purpose | | --- | --- | --- | | `POST` | `/v2/ai/assistants/{assistant_id}/scheduled_events` | Create a scheduled event | | `GET` | `/v2/ai/assistants/{assistant_id}/scheduled_events` | List scheduled events (paginated, filterable) | | `GET` | `/v2/ai/assistants/{assistant_id}/scheduled_events/{event_id}` | Fetch a single event | | `DELETE` | `/v2/ai/assistants/{assistant_id}/scheduled_events/{event_id}` | Cancel a scheduled event | ### Create-event fields | Field | Required | Channels | Description | | --- | --- | --- | --- | | `telnyx_conversation_channel` | Yes | both | `phone_call` or `sms_chat`. | | `telnyx_end_user_target` | Yes | both | The recipient (phone number, E.164 or Sip URI). | | `telnyx_agent_target` | Yes | both | The number the assistant calls or sends from. | | `scheduled_at_fixed_datetime` | Yes | both | ISO-8601 timestamp; must be in the future. | | `text` | Yes for SMS | `sms_chat` | The message body. | | `conversation_metadata` | No | both | Free-form key/value metadata attached to the resulting conversation. | | `dynamic_variables` | No | both | Variables passed to the assistant for prompt interpolation. | | `max_retries_client_errors` | No | `phone_call` | Retry budget for recipient-side outcomes. | | `retry_interval_secs` | No | `phone_call` | Seconds between retries. | --- ## Examples ### Schedule a phone call with retries on busy / no-answer This event will be tried up to four times total (initial + 3 retries), with a five-minute gap between attempts. ```bash curl -L 'https://api.telnyx.com/v2/ai/assistants/{assistant_id}/scheduled_events' \ -H 'Content-Type: application/json' \ -H 'Accept: application/json' \ -H 'Authorization: Bearer ' \ --data-raw '{ "telnyx_conversation_channel": "phone_call", "telnyx_end_user_target": "+15551234567", "telnyx_agent_target": "+15553456789", "scheduled_at_fixed_datetime": "2026-05-08T15:00:00Z", "max_retries_client_errors": 3, "retry_interval_secs": 300, "conversation_metadata": { "campaign": "renewal-q2", "customer_id": "cust_42" } }' ``` ### Schedule a phone call without retries Omit the retry fields (or set `max_retries_client_errors` to `0`) for a single-attempt call. ```bash curl -L 'https://api.telnyx.com/v2/ai/assistants/{assistant_id}/scheduled_events' \ -H 'Content-Type: application/json' \ -H 'Accept: application/json' \ -H 'Authorization: Bearer ' \ --data-raw '{ "telnyx_conversation_channel": "phone_call", "telnyx_end_user_target": "+15551234567", "telnyx_agent_target": "+15553456789", "scheduled_at_fixed_datetime": "2026-05-08T15:00:00Z" }' ``` ### Schedule an SMS Retry fields do not apply to SMS events. ```bash curl -L 'https://api.telnyx.com/v2/ai/assistants/{assistant_id}/scheduled_events' \ -H 'Content-Type: application/json' \ -H 'Accept: application/json' \ -H 'Authorization: Bearer ' \ --data-raw '{ "telnyx_conversation_channel": "sms_chat", "telnyx_end_user_target": "+15551234567", "telnyx_agent_target": "+15553456789", "scheduled_at_fixed_datetime": "2026-05-08T15:00:00Z", "text": "Hi! This is a reminder about your appointment tomorrow at 10 AM." }' ``` ### Inspect retry history ```bash curl -L 'https://api.telnyx.com/v2/ai/assistants/{assistant_id}/scheduled_events/{event_id}' \ -H 'Accept: application/json' \ -H 'Authorization: Bearer ' ``` The response includes `status`, `call_attempts`, and (once retries finish) the final `call_status` and `errors` array. ### Cancel a pending event Deleting a `pending` event prevents any further dispatches. Deletion has no effect on attempts that have already completed. ```bash curl -X DELETE 'https://api.telnyx.com/v2/ai/assistants/{assistant_id}/scheduled_events/{event_id}' \ -H 'Authorization: Bearer ' ``` --- ## Choosing a retry policy A few rules of thumb when picking values: - **Short intervals (60–300s)** suit time-sensitive flows where the recipient is likely to be near their phone — verification calls, urgent reminders. They burn the retry budget quickly, which is sometimes what you want. - **Medium intervals (5–30 minutes)** are a good default for outreach campaigns. Long enough that a busy recipient may finish their previous call; short enough that the event still completes within an hour. - **Long intervals (1–24 hours)** match daily-cadence flows: missed bill reminders, callback queues, abandoned-cart nudges. - **Total reach time** = `max_retries_client_errors × retry_interval_secs`. Make sure that span fits inside the time window when calling is acceptable for your use case (and your jurisdiction's calling-hour rules). - **Higher `max_retries_client_errors` is not always better** — at some point the recipient is unlikely to answer regardless of how many times you try. Three to five retries is usually plenty. --- ## Related resources - [Voice AI Assistant API Reference](/api-reference/assistants/create-an-assistant) — Full assistant configuration. - [No-Code Voice Assistant](/docs/inference/ai-assistants/no-code-voice-assistant) — Get started with Voice AI Assistants in the Portal. - [Dynamic Variables](/docs/inference/ai-assistants/dynamic-variables) — Pass per-event context into the assistant prompt. - [Voicemail Detection on Transfer](/docs/inference/ai-assistants/voicemail-detection-on-transfer) — Handle voicemail on transferred calls (a separate but related feature for live calls). --- ### Transcription Settings > Source: https://developers.telnyx.com/docs/inference/ai-assistants/transcription-settings.md Telnyx AI Assistants support multiple speech-to-text (STT) models for transcribing caller audio. The model you choose affects transcription accuracy, supported languages, and response latency. You can also tune provider-specific transcription behavior, such as end-of-turn detection, formatting, keyterm boosting, and Azure region selection. --- ## Available models | Model | Engine | Best for | | --- | --- | --- | | `deepgram/flux` | Deepgram | Conversational AI, optimized for turn-taking with multilingual support | | `deepgram/nova-3` | Deepgram | Fast multilingual transcription, recommended for multilingual assistants | | `deepgram/nova-2` | Deepgram | Fast multilingual transcription on Deepgram's previous-generation model | | `azure/fast` | Azure | Fast multilingual transcription with optional Azure region and API key configuration | | `assemblyai/universal-streaming` | AssemblyAI | Conversational, multilingual streaming transcription with configurable turn detection | | `xai/grok-stt` | xAI | Multilingual transcription using Grok STT | `deepgram/flux` supports English, Spanish, French, German, Hindi, Russian, Portuguese, Japanese, Italian, and Dutch. For broader language coverage, use `deepgram/nova-3`, `deepgram/nova-2`, `azure/fast`, `assemblyai/universal-streaming`, or `xai/grok-stt`. --- ## Selecting a model ### Portal In the [AI Assistants tab](https://portal.telnyx.com/#/ai/assistants), edit your assistant and navigate to the **Voice** tab. Select your preferred STT model from the **Transcription Model** dropdown. ![AI Assistant Transcription Model Selection](/img/ai-assistant-transcription-model.png) When you change models in the Portal, related settings are reset to the defaults for that provider. For example: - `deepgram/flux` supports explicit languages, `auto`, and `multi` for its supported languages, and applies Flux end-of-turn defaults. - Other Deepgram models enable `smart_format` and `numerals` by default. - `assemblyai/universal-streaming` applies AssemblyAI turn detection defaults. - `azure/fast` defaults the Azure region to `latency`, which auto-selects the closest supported Telnyx-managed region. ### API Set the `transcription.model` field when creating or updating an assistant: ```bash curl -X POST https://api.telnyx.com/v2/ai/assistants \ -H "Authorization: Bearer $TELNYX_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "My Assistant", "model": "anthropic/claude-haiku-4-5", "instructions": "You are a helpful voice assistant.", "transcription": { "model": "deepgram/flux" } }' ``` You can also set the transcription language explicitly. If omitted or set to `auto`, supported models auto-detect the language: ```json "transcription": { "model": "deepgram/nova-3", "language": "es" } ``` --- ## Languages Supported language options depend on the selected model. | Model | Language behavior | | --- | --- | | `deepgram/flux` | English, Spanish, French, German, Hindi, Russian, Portuguese, Japanese, Italian, Dutch, plus `auto` and `multi` modes | | `deepgram/nova-3` | Auto-detect plus supported Deepgram Nova 3 language codes | | `deepgram/nova-2` | Auto-detect plus supported Deepgram Nova 2 language codes | | `azure/fast` | Explicit Azure locale codes, such as `en-US`, `es-MX`, or `fr-FR` | | `assemblyai/universal-streaming` | `auto` for multilingual detection or `en` for English | | `xai/grok-stt` | `auto` plus supported Grok STT language codes | If your assistant has a language filter set elsewhere in the Voice tab, the Portal only shows transcription models and language choices that are compatible with that language. --- ## Deepgram settings ### Deepgram Flux end-of-turn detection `deepgram/flux` is optimized for live voice agents. It provides end-of-turn detection so the assistant can start responding as soon as the caller finishes speaking. It also supports eager end-of-turn, which starts large language model (LLM) processing before the caller fully stops speaking to reduce perceived response latency. When you select `deepgram/flux` in the Portal, these settings are applied by default: | Field | Type | Range | Portal default | Description | | --- | --- | --- | --- | --- | | `eot_threshold` | number | 0.5-0.9 | 0.8 | Confidence required to trigger a final end of turn. Higher values require more confidence and may add latency. | | `eot_timeout_ms` | integer | 500-10000 | 5000 | Maximum silence duration, in milliseconds, before forcing an end of turn. | | `eager_eot_threshold` | number | 0.3-0.9 | 0.4 | Confidence required to start speculative LLM processing before final end-of-turn confirmation. Lower values trigger earlier. | `eager_eot_threshold` must be less than or equal to `eot_threshold`. Setting both thresholds to the same value effectively disables eager end-of-turn behavior because the system waits for final end-of-turn confirmation before starting LLM processing. The `eager_eot_threshold` field is controlled by the `FE-eager-eot-threshold` Portal feature flag. When that flag is disabled, the Portal hides the field, but API payloads can still include it if your account supports the setting. When using Flux, the Portal may also prompt you to lower the assistant's start speaking plan timings. Flux works best with low start speaking delays, such as `0.1` seconds for wait time and endpointing plan thresholds. ### Keyterm Boost `deepgram/flux` and `deepgram/nova-3` support `keyterm`, a comma-separated list of terms to boost during recognition. Use it for product names, customer names, acronyms, or domain-specific vocabulary. Keyterm Boost also supports [dynamic variables](/docs/inference/ai-assistants/dynamic-variables). Use variables when boosted terms are caller-specific, such as a customer name, participant names, account name, or product names passed into the assistant at conversation start. ```json "transcription": { "model": "deepgram/nova-3", "settings": { "keyterm": "Telnyx,VoIP,SIP,{{customer_name}},{{product_name}}" } } ``` ### Smart Format and Numerals For Deepgram models other than Flux, the Portal exposes these settings and enables both by default when you select the model: | Field | Type | Default | Description | | --- | --- | --- | --- | | `smart_format` | boolean | `true` | Automatically formats transcripts for readability, including punctuation and casing. | | `numerals` | boolean | `true` | Converts spoken numbers to digits, for example "five hundred" to "500". | --- ## AssemblyAI settings `assemblyai/universal-streaming` supports configurable turn detection. When you select it in the Portal, these defaults are applied: | Field | Type | Range | Portal default | Description | | --- | --- | --- | --- | --- | | `end_of_turn_confidence_threshold` | number | 0-1 | 0.4 | Confidence required to trigger an end of turn. Higher values require more certainty before ending a turn. | | `min_turn_silence` | integer | 100-5000 | 400 | Minimum silence duration, in milliseconds, before a turn can end. | | `max_turn_silence` | integer | 100-5000 | 1280 | Maximum silence duration, in milliseconds, before forcing an end of turn. | `min_turn_silence` must be less than or equal to `max_turn_silence`. ```json "transcription": { "model": "assemblyai/universal-streaming", "language": "auto", "settings": { "end_of_turn_confidence_threshold": 0.4, "min_turn_silence": 400, "max_turn_silence": 1280 } } ``` --- ## Azure settings `azure/fast` supports region selection and an optional Azure API key reference. | Field | Type | Description | | --- | --- | --- | | `region` | string | Azure transcription region. The Portal defaults to `latency`, which auto-selects the closest supported Telnyx-managed region. | | `api_key_ref` | string | Optional integration secret reference for your Azure API key. When provided, the Portal only shows regions that support custom API keys. | Common Telnyx-managed regions include `latency`, `australiaeast`, `centralindia`, `eastus`, `northcentralus`, `westeurope`, and `westus2`. Additional Azure regions are available when using your own Azure API key. ```json "transcription": { "model": "azure/fast", "language": "en-US", "region": "latency" } ``` --- ## Configure advanced settings via API Create an assistant with tuned transcription settings: ```bash curl -X POST https://api.telnyx.com/v2/ai/assistants \ -H "Authorization: Bearer $TELNYX_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "Low Latency Assistant", "model": "anthropic/claude-haiku-4-5", "instructions": "You are a helpful voice assistant.", "transcription": { "model": "deepgram/flux", "language": "en", "settings": { "eot_threshold": 0.8, "eot_timeout_ms": 5000, "eager_eot_threshold": 0.4, "keyterm": "Telnyx,VoIP,SIP,{{customer_name}},{{product_name}}" } } }' ``` --- ## Related resources - [Create an Assistant API Reference](/api-reference/assistants/create-an-assistant) - [Voice Assistant quickstart](/docs/inference/ai-assistants/no-code-voice-assistant) - [STT WebSocket Streaming](/docs/tts-stt/stt-websocket-streaming) --- ### Integrations > Source: https://developers.telnyx.com/docs/inference/ai-assistants/integrations.md Telnyx AI assistants can integrate with leading enterprise platforms to access customer data, create tickets, update records, and automate workflows directly during conversations. ## Available integrations | Integration | Description | Common Use Cases | |-------------|-------------|------------------| | **Salesforce** | CRM platform for sales, service, and marketing | Lead qualification, case management, contact updates | | **ServiceNow** | IT service management and workflow automation | Incident creation, ticket updates, service requests | | **Jira** | Project management and issue tracking | Bug reporting, task creation, project updates | | **HubSpot** | Marketing, sales, and service CRM | Contact management, deal tracking, ticket creation | | **Zendesk** | Customer service and support platform | Ticket creation, customer lookup, case management | | **Intercom** | Customer messaging and support platform | Conversation management, user data access | | **GitHub** | Code hosting and version control | Issue creation, repository access, pull request management | | **Greenhouse** | Applicant tracking and recruiting platform | Candidate lookup, interview scheduling, application management | | **Coval** | Simulation and evaluation for voice and chat agents | Automated testing, regression detection, production monitoring | ## Getting started ### Prerequisites Before connecting an integration, make sure you have: Active access with the right permissions on the target platform. Platform-specific keys or tokens required by the integration. A configured assistant ready to connect the integration. ### Connection workflow Navigate to [AI Assistants](https://portal.telnyx.com/#/ai/assistants) and open an existing assistant or create a new one. Select the **Integrations** tab for that assistant. Click **Add Integration**, choose a provider from the dropdown, and enter the required credentials. Enable the tools you need, adjust defaults, and save the assistant. ![Mission Control Portal showing Add Integration dropdown with available platforms including GitHub, Greenhouse, HubSpot, Intercom, Jira, Salesforce, and Zendesk](/img/ai-integration-dropdown-full.jpeg) ## See integrations in action Watch how Telnyx Voice AI Agents connect to enterprise platforms like ServiceNow to create, update, and resolve tickets through natural voice conversation: This demonstration shows the integration workflow in the Mission Control Portal and real-time ticket management capabilities that work across all supported platforms. ## Platform-specific setup Select your integration platform below. If you don't see your platform, scroll horizontally to view all available options. ### GitHub Connect your AI assistant to GitHub for code hosting, version control, and development workflows. #### Prerequisites - GitHub account with repository access. - Permissions to create personal access tokens. - Appropriate repository scopes for integration needs. #### Required credentials When connecting GitHub, you'll need to provide: Generate a GitHub Personal Access Token at https://github.com/settings/tokens by clicking **Generate new token → Generate new token (classic)** and selecting scopes such as `repo`, `read:user`, and `read:org`, then copy the token (it only appears once). ![GitHub integration credentials dialog showing Personal Access Token input field](/img/github-integration-credentials.png) #### Available tools After connecting GitHub, your assistant can: - **Create issues** - Log bugs, feature requests, or tasks in repositories. - **Search repositories** - Find repos by name, description, or topics. - **Manage pull requests** - Create, review, or update PRs. - **Access code** - Read file contents and repository structure. - **Manage labels** - Add or remove issue and PR labels. #### Use cases **Bug Reporting** ``` Developer: "We have a login bug affecting Safari users" Assistant: [Creates issue in GitHub repository] Assistant: [Adds labels: bug, priority:high, browser:safari] Assistant: "I've created issue #234 in the main repo and notified the team." ``` **Repository Search** ``` Developer: "Find our authentication libraries" Assistant: [Searches GitHub repos by keyword and topic] Assistant: "I found 3 authentication repos: auth-service, jwt-utils, and oauth-client." ``` ### Greenhouse Connect your AI assistant to Greenhouse for applicant tracking and recruiting workflows. #### Prerequisites - Greenhouse account with API access. - Access to Greenhouse Dev Center. - Harvest API permissions. #### Required credentials When connecting Greenhouse, you'll need to provide: Log in to Greenhouse, go to **Dev Center → API Credential Management**, create a Harvest API key with the permissions you need, and copy the generated token. ![Greenhouse integration credentials dialog showing API Key input field](/img/greenhouse-integration-credentials.png) #### Available tools After connecting Greenhouse, your assistant can: - **Candidate lookup** - Search for candidates by name, email, or application ID. - **Interview scheduling** - View and coordinate interview schedules. - **Application management** - Track application status and progress. - **Job posting access** - View open positions and job details. - **Scorecard review** - Access candidate evaluations and feedback. #### Use cases **Candidate Status Check** ``` Recruiter: "What's the status of the candidate who interviewed yesterday?" Assistant: [Searches Greenhouse by interview date] Assistant: "Sarah Johnson completed her technical interview yesterday. She's in the Reference Check stage." ``` **Interview Coordination** ``` Recruiter: "Schedule the next round for top candidates" Assistant: [Retrieves candidate list and interview availability] Assistant: [Checks interviewer calendars] Assistant: "I can schedule 3 final interviews for next Tuesday and Wednesday." ``` ### HubSpot Connect your AI assistant to HubSpot for marketing, sales, and customer service workflows. #### Prerequisites - HubSpot account with API access. - Private app access token or OAuth credentials. #### Required credentials When connecting HubSpot, you'll need to provide: Create a private app in HubSpot (Settings → Integrations → Private Apps) and copy the access token from the **Auth** tab. ![HubSpot integration credentials dialog showing Private app token input field](/img/hubspot-integration-credentials.png) #### Available tools After connecting HubSpot, your assistant can: - **Manage contacts** - Create, update, or search contacts. - **Deal tracking** - Create deals, update deal stages. - **Ticket management** - Create support tickets, update status. - **Company records** - Access and update company information. - **Engagement tracking** - Log calls, emails, and notes. #### Use cases **Lead Capture** ``` Prospect: "I'd like a demo of your product" Assistant: [Creates contact in HubSpot] Assistant: [Creates deal in pipeline] Assistant: [Schedules demo meeting] ``` **Support Ticketing** ``` Customer: "I have a billing question" Assistant: [Searches HubSpot for customer record] Assistant: [Creates ticket in support pipeline] Assistant: [Associates ticket with contact and deal] ``` ### Intercom Connect your AI assistant to Intercom for customer messaging and support workflows. #### Prerequisites - Intercom account with API access. - Permissions to create private apps. - Access token with appropriate scopes. #### Required credentials When connecting Intercom, you'll need to provide: Create a private app in Intercom (Settings → Developers → Developer Hub → Your Apps → New App) and copy the access token from the authentication section. ![Intercom integration credentials dialog showing Access Token input field](/img/intercom-integration-credentials.png) #### Available tools After connecting Intercom, your assistant can: - **Access conversation history** - Retrieve past customer interactions and messages. - **Create notes** - Add internal notes to customer conversations. - **Update customer attributes** - Modify user data and custom attributes. - **Search users** - Find customers by email, user ID, or other identifiers. - **Manage tags** - Add or remove conversation tags for organization. #### Use cases **Customer Support Context** ``` Customer: "I need help with my subscription" Assistant: [Searches Intercom for customer by phone/email] Assistant: [Reviews conversation history] Assistant: "I can see you upgraded to Pro last month. How can I help with your subscription?" ``` **User Data Management** ``` Customer: "Please update my company name" Assistant: [Updates customer attributes in Intercom] Assistant: [Adds note documenting the change] Assistant: "I've updated your company name in our system." ``` ### Jira Connect your AI assistant to Jira for project management, issue tracking, and software development workflows. #### Prerequisites - Jira account (Cloud or Server). - API token or password. - Project access and permissions. #### Required credentials When connecting Jira, you'll need to provide: The Jira username or email that has access to the project. Generate a token at https://id.atlassian.com/manage-profile/security/api-tokens by selecting **Create API token**. Provide the base URL of your Jira instance (for example `yourcompany.atlassian.net`) without the `https://` prefix. ![Jira integration credentials dialog showing Email, API token, and Site URL input fields](/img/jira-integration-credentials.png) #### Available tools After connecting Jira, your assistant can: - **Create issues** - Create bugs, tasks, stories, or epics. - **Update issues** - Change status, assignee, or priority. - **Search issues** - Find issues by project, assignee, or status. - **Add comments** - Comment on existing issues. - **Transition issues** - Move issues through workflow states. #### Use cases **Bug Reporting** ``` Developer: "Users are reporting a login error" Assistant: [Creates bug in Jira] Assistant: [Sets priority to High, component to Authentication] Assistant: "Created PROJ-1234. I've assigned it to the on-call engineer." ``` **Task Management** ``` Manager: "Create a task to update the documentation" Assistant: [Creates task in Jira project] Assistant: [Sets due date based on conversation] ``` ### Salesforce Connect your AI assistant to Salesforce to access customer records, create cases, update opportunities, and more. #### Prerequisites - Salesforce account with API access. - Username and password. - Security token (reset in Personal Settings → Reset My Security Token). - Organization ID (found in Setup → Company Settings → Company Information). #### Required credentials When connecting Salesforce, you'll need to provide: Your Salesforce hostname such as `acme.my.salesforce.com` (production) or `acme.sandbox.my.salesforce.com` (sandbox) without `https://` or trailing `/`. The Salesforce username or email with integration access. The password for that Salesforce user. The security token from Personal Settings → Reset My Security Token (it arrives via email). Your org ID from Setup → Company Settings → Company Information. ![Salesforce integration credentials dialog showing Instance domain, Username, Password, Security token, and Organization ID input fields](/img/salesforce-integration-credentials.png) #### Available tools After connecting Salesforce, your assistant can use tools to: - **Search records** - Find accounts, contacts, leads, opportunities. - **Create records** - Create new cases, leads, tasks, or opportunities. - **Update records** - Modify existing records with new information. - **Query data** - Run SOQL queries for custom data retrieval. #### Use cases **Customer Service** ``` Customer: "I need help with my recent order" Assistant: [Searches Salesforce for customer by phone number] Assistant: "I found your account, let me check your recent orders..." ``` **Lead Qualification** ``` Prospect: "I'm interested in your enterprise plan" Assistant: [Creates lead in Salesforce with details from conversation] Assistant: [Updates lead score based on budget and timeline discussed] ``` **Case Management** ``` Customer: "My service is down" Assistant: [Creates high-priority case in Salesforce] Assistant: "I've created case #12345 for you. Our team will reach out within 2 hours." ``` ### ServiceNow Connect your AI assistant to ServiceNow for IT service management, incident tracking, and workflow automation. #### Prerequisites - ServiceNow instance with API access. - User account with appropriate roles (e.g., itil, admin). - Instance URL and credentials. #### Required credentials Provide the ServiceNow hostname, for example `acme.service-now.com` or `acme-dev.service-now.com`. Supply the ServiceNow user with the necessary roles. Enter the password for that user. ![ServiceNow integration credentials dialog showing Instance URL, Username, and Password input fields](/img/servicenow-integration-credentials.png) #### Available tools After connecting ServiceNow, your assistant can: - **Create incidents** - Log IT incidents with priority and categorization. - **Update tickets** - Modify incident status, assignment, or details. - **Search knowledge base** - Find KB articles for issue resolution. - **Query records** - Access CMDB, user records, or service catalogs. #### Use cases **IT Support** ``` Employee: "My laptop won't connect to WiFi" Assistant: [Creates incident in ServiceNow] Assistant: [Categorizes as Network → WiFi] Assistant: "I've logged incident INC0012345. IT will assist you shortly." ``` **Service Requests** ``` Employee: "I need access to the marketing drive" Assistant: [Creates service request in ServiceNow] Assistant: [Routes to appropriate approval group] ``` ### Zendesk Connect your AI assistant to Zendesk for customer service and support workflows. #### Prerequisites - Zendesk account with API access. - Admin access to generate API tokens. - Subdomain and email credentials. #### Required credentials When connecting Zendesk, you'll need to provide: Enter only the subdomain portion (e.g., `company` if your portal is `company.zendesk.com`); the rest is added automatically. The Zendesk account email that owns the API token. Generate a token in Admin Center → Apps and integrations → APIs → Zendesk API. ![Zendesk integration credentials dialog showing Subdomain, Email, and API token input fields](/img/zendesk-integration-credentials.png) #### Available tools After connecting Zendesk, your assistant can: - **Create tickets** - Log customer support requests with priority and categorization. - **Search customer history** - Find previous tickets and interactions by customer. - **Update ticket status** - Modify ticket status, assignment, or priority. - **Access knowledge base** - Search KB articles for issue resolution. #### Use cases **Support Ticketing** ``` Customer: "I'm having an issue with my account login" Assistant: [Creates ticket in Zendesk] Assistant: [Categorizes as Account → Login Issues] Assistant: "I've created ticket #12345. Our support team will reach out within 2 hours." ``` **Customer History Lookup** ``` Customer: "What's the status of my previous request?" Assistant: [Searches Zendesk by phone number or email] Assistant: "I found your ticket #12340 from last week. It was resolved on Monday." ``` ### Coval Connect your AI assistant to [Coval](https://www.coval.dev/) for automated simulation, evaluation, and production monitoring of voice and chat agents. Coval is a **testing and evaluation** tool — unlike the other integrations on this page, it does not add tools your assistant uses during conversations. Instead, it tests and monitors the assistant itself, helping you catch regressions and validate behavior at scale. #### Prerequisites - Coval account at [coval.dev](https://www.coval.dev/). - A configured Telnyx AI assistant to evaluate. #### Required credentials When connecting Coval, you'll need to provide: Log in to your Coval dashboard, navigate to your workspace settings, and copy your API key. #### Capabilities After connecting Coval, you can: - **Automated scenario simulation** — Generate thousands of test scenarios from a few seed cases, covering edge cases and unexpected user behaviors. - **CI/CD regression testing** — Automatically detect performance regressions on every code change before deploying to production. - **Production monitoring** — Log live calls, receive instant alerts for performance drops, and replay transcripts and audio. - **Built-in evaluation metrics** — Measure latency, accuracy, tool-call effectiveness, and instruction compliance across all interactions. #### Use cases **Pre-deployment validation** ``` Run hundreds of simulated conversations against your assistant to verify it handles greetings, edge cases, and tool calls correctly before going live. ``` **Regression testing in CI/CD** ``` Add Coval evaluation steps to your deployment pipeline so every assistant update is automatically tested against your scenario library. ``` **Production quality monitoring** ``` Monitor live assistant conversations for performance drops, then replay specific calls to diagnose issues. ``` ## Managing integrations ### Viewing connected integrations Navigate to the assistant you want to review in the Mission Control Portal. Open the **Integrations** tab. View everything listed under **Connected Integrations**. ![Integrations section displaying Jira under Connected Integrations with description and unassign button](/img/connected-integration-view.png) ### Disconnecting an integration To disconnect an integration from your assistant: Navigate to your assistant and open the **Integrations** tab. Find it within the **Connected Integrations** list. Click the chain-link **unassign** button. Approve the confirmation dialog to finish disconnecting. ![Jira integration card in Connected Integrations showing unassign button (chain link icon)](/img/jira-connected-integration-unassign.png) After disconnecting: - The integration is removed from this assistant. - All associated tools are disabled for this assistant. - The integration moves to **Available Integrations** and can be reconnected later. Disconnecting an integration only removes it from the current assistant. The integration remains in your account and can be connected to other assistants or reconnected to this one. ### Deleting an integration To permanently delete an integration from your account: Navigate to your AI assistant and open the **Integrations** tab. Look under **Available Integrations**. Click the trash icon next to the integration. Approve the deletion to remove stored credentials permanently. ![Jira integration card in Available Integrations showing connect button and delete button (trash icon)](/img/jira-available-integrations-actions.png) Deleting an integration permanently removes it from your account, including all stored credentials. You will need to set it up again from scratch if you want to use it in the future. ## Best practices ### Security Create integration-specific profiles with only the permissions the workflow needs. Update API tokens and passwords on a routine cadence. Review integration activity through platform audit logs. Grant only the scopes required for each integration use case. Test and validate integrations in non-production environments first. ### Configuration Enable search/read capabilities first, then gradually introduce write actions. Document when and how each tool should be used so assistants respond correctly. Validate integration behavior across multiple conversation scenarios. Configure sensible defaults (e.g., priority, project) to reduce user input errors. Define fallback behavior when integration calls fail or return unexpected results. ### Performance Avoid duplicate searches or redundant requests where possible. Store reusable values (for example, via dynamic variables) for the duration of a session. Configure timeout thresholds that balance responsiveness with reliability. Track provider limits and design workflows to stay within allocation. ## Troubleshooting ### Connection failures **Symptom**: Unable to connect integration, credentials rejected. **Solutions**: - Verify credentials are correct and have not expired. - Check that the user account has API access enabled. - Ensure security tokens or API keys are current. - For Salesforce: Confirm security token is included. - For cloud platforms: Verify instance URL format (no `https://` or trailing `/`). ### Tools not appearing **Symptom**: Integration connected but no tools available. **Solutions**: - Refresh the page and check again. - Verify the integration account has required permissions. - Check that the platform subscription includes API access. - Disconnect and reconnect the integration. ### Authentication errors during calls **Symptom**: Tools fail with authentication errors during conversations. **Solutions**: - Regenerate API tokens or security tokens. - Update stored credentials in the integration. - Verify account has not been locked or suspended. - Check IP allowlists (if applicable). ### Missing data or records **Symptom**: Assistant cannot find expected records. **Solutions**: - Verify the integration account can access the records. - Check record permissions and sharing settings. - Confirm records exist in the platform. - Verify search parameters and filters. ### Rate limiting **Symptom**: Integration calls fail with rate limit errors. **Solutions**: - Reduce frequency of integration calls. - Implement caching for frequently accessed data. - Contact platform support to increase limits. - Distribute calls across multiple service accounts. ## Next steps - **[Voice Assistant Quickstart](https://developers.telnyx.com/docs/inference/ai-assistants/no-code-voice-assistant)** - Learn how to create and configure AI assistants. - **[Workflow](/docs/inference/ai-assistants/workflows)** - Visualize how your integrations and tools connect in your assistant's conversation flow. - **[Agent Handoff](https://developers.telnyx.com/docs/inference/ai-assistants/agent-handoff)** - Enable multiple specialized assistants with integrations. - **[Dynamic Variables](https://developers.telnyx.com/docs/inference/ai-assistants/dynamic-variables)** - Pass integration-specific context to your assistant. - **[API Reference](/api-reference/assistants/list-assistants)** - Programmatic assistant management. ## Related resources - [Integration Secrets](https://portal.telnyx.com/#/integration-secrets) - Securely store API keys and tokens. - [AI Assistants Portal](https://portal.telnyx.com/#/ai/assistants) - Configure assistants and integrations. --- ### Testing, Versions & Traffic Distribution > Source: https://developers.telnyx.com/docs/inference/ai-assistants/version-testing-traffic-distribution.md This guide walks you through testing your AI assistant before production deployment and managing live traffic distribution between different versions. You'll learn how to create tests, iterate on your assistant, and safely roll out changes using A/B testing. --- ## Creating Your First Assistant Start by creating a new assistant using a template to establish a baseline for testing. 1. Navigate to the [AI Assistants page](https://portal.telnyx.com/#/ai/assistants) 2. Click "Create Assistant" and select the "Weather Assistant" template 3. This template provides a good foundation with a standard greeting and weather functionality ![Create Weather Assistant](/img/create-weather-assistant-template.png) Take note of the default greeting message - we'll be testing and modifying this later. --- ## Setting Up Your First Test Testing your assistant ensures it behaves correctly before going live with users. ### Creating a Test 1. Navigate to the [AI Tests page](https://portal.telnyx.com/#/ai/tests) 2. Click "Create Test" to set up your first test scenario ![Create AI Test](/img/create-ai-test-setup.png) ### Configuring Test Criteria 3. Configure your test with the following: - **Test Name**: "Weather Assistant Greeting Test" - **Assistant**: Select your weather assistant - **Success Criteria**: Add criteria to validate the greeting message content and that temperature is described by the assistant. ![Test Criteria Configuration](/img/test-criteria-setup.png) ### Running Your Test 1. Click "Run Test" to execute your test scenario 2. Monitor the test progress in real-time 3. Review the detailed results once the test completes ![Test Execution Run](/img/ai-test-run.png) ![Test Execution Results](/img/test-execution-results.png) ![Test Execution Results](/img/test-conversation-history.png) The results will show whether your assistant met all the defined criteria, helping you identify any issues before deployment. You can also review the conversation itself. --- ## Creating Assistant Versions Now you'll create a new version of your assistant with modified behavior to demonstrate A/B testing. To make it obvious that the A/B test is working, we make two visibly distinct versions of the AI Assistant using the frontend widget feature. Versions are not limited to the frontend, though. You can make versions from any configuration on the assistant including updated tools, instructions, and more. ### Modifying the Assistant 1. Return to your weather assistant in the [AI Assistants page](https://portal.telnyx.com/#/ai/assistants) 2. Click the edit icon (pencil) next to your assistant 3. Make the following changes to create a visually distinct version: - **Enable the frontend widget**: Navigate to the Widget tab and click enable - **Widget Appearance**: Navigate back to the Widget tab and change the widget theme from dark mode to light mode in the appearance settings ![Enable Assistant Widget](/img/enable-assistant-widget.png) ### Creating a New Version 1. After making your changes, click "Save as New Version" 2. Give your version a descriptive name: "Light Theme with New Greeting" 3. Add version notes describing the changes made ![Edit Assistant Widget](/img/edit-assistant-widget.png) You now have two versions of your assistant: - **Version 1**: Original greeting with dark theme widget - **Version 2**: New greeting with light theme widget ![View New Assistant Version](/img/view-assistant-versions.png) --- ## Production Traffic Distribution Once you've validated a new version, use version routing to control which live calls receive it. Traffic routing now uses ordered rules, similar to feature-flag targeting. Each rule has: - **If** conditions that match the end user target for the conversation - **Serve** behavior that sends matching calls to one version, or splits them across several versions Rules are evaluated from top to bottom. The first matching rule wins. If no target rule matches, the assistant serves the main version unless you configure a default rule. ### Open the traffic routing editor 1. Open your assistant from the [AI Assistants page](https://portal.telnyx.com/#/ai/assistants). 2. Go to the assistant's version or deployment controls. 3. Open **Traffic distribution** to configure routing rules. ### Add a target rule Use target rules when you want a specific end user, customer, or test endpoint to receive a version. The target value is the same value exposed to assistants as [`{{telnyx_end_user_target}}`](/docs/inference/ai-assistants/dynamic-variables#telnyx-system-variables): the phone number, SIP URI, or other identifier associated with the end user. This works for both call directions: - **Inbound calls**: the end user target is the caller's phone number, SIP URI, or identifier. - **Outbound calls**: the end user target is the destination the assistant calls. For example, you can route calls to your own number to a test version. 1. Click **Add rule**. 2. In the **If** section, choose **End user target**. 3. Select an operator: - **is one of** for exact targets, such as `+13125550123` or `sip:customer@example.com` - **is not one of** to exclude specific targets - **starts with** for prefixes, such as `+1312` or `sip:qa-` ![AI Assistant traffic routing new end user target rule](/img/ai-assistant-routing-new-target-rule.png) If your Traffic distribution editor still shows **Origination number**, treat it as the end user target. The field is being renamed because the same routing behavior applies to inbound callers and outbound call destinations. 4. Enter one or more target values. You can separate values with commas or new lines. 5. In the **Serve** section, choose **Send all matched calls to one version** and select the version that matching calls should receive. ![AI Assistant single-version end user target rule](/img/ai-assistant-routing-single-version-rule.png) Target rules can contain multiple conditions. Conditions in the same rule are AND-joined, so every condition must match for the rule to apply. If multiple rules could match, only the first matching rule is used. ![AI Assistant end user target rule with multiple conditions](/img/ai-assistant-routing-multiple-conditions.png) ### Split matching traffic by percentage For gradual rollouts, set a rule's **Serve** behavior to **Split by percentage**. Add version slots and assign each one a percentage. The allocation bar shows how matching traffic is split. Percentages must add up to less than 100. Any remaining percentage routes to the main version, which gives you a built-in safety fallback during canary releases. ![AI Assistant weighted rollout configuration](/img/ai-assistant-routing-weighted-rollout.png) For example, if a target rule sends 25% of matching calls to Version 2 and 25% to Version 3, the remaining 50% of matching calls continue to use the main version. ### Configure the default rule The default rule handles calls that do not match any target rule. By default, unmatched calls serve the main version. Use **Configure default** when you want unmatched calls to receive another version or a percentage split. Use **Reset to main** to remove the custom default and return all unmatched traffic to the main version. ### Save, reorder, or rollback - Drag target rules to change their priority. Rule order matters because the first match wins. - Click **Save** to apply the routing configuration. - Click **Rollback** to clear all routing rules and send traffic back to the main version. This setup allows you to: - Test new versions with internal phone numbers or SIP URIs before a broad rollout - Run percentage-based canaries for matching call segments - Keep the main version as the fallback for unmatched calls and remaining rollout percentage - Quickly rollback if issues arise - Promote a validated version to main when you're ready ### Testing Live Traffic Distribution To verify your routing is working correctly, make repeated test calls with end user targets that should match each rule: 1. For inbound testing, call the assistant from a phone number or SIP URI listed in a target rule and confirm the routed version answers. 2. For outbound testing, have the assistant call a target listed in a rule, such as your own phone number, and confirm you receive the test version. 3. Call from or to a target that should not match the target rules and confirm the default behavior applies. 4. For weighted rollouts, make enough calls to confirm that matching traffic is distributed according to your configured percentages. --- ## Automated Evaluation with Coval The manual testing and A/B traffic distribution described above work well for targeted checks and gradual rollouts. For automated evaluation at scale, Telnyx integrates with [Coval](https://www.coval.dev/) — a simulation and evaluation platform purpose-built for voice and chat agents. ### What Coval adds | Capability | How it complements built-in testing | |---|---| | **Scenario simulation** | Generate thousands of test conversations from a few seed cases, covering edge cases that are difficult to script manually. | | **CI/CD evaluations** | Automatically run your scenario library on every assistant change and block deployments that introduce regressions. | | **Production monitoring** | Log live calls, surface performance drops in real time, and replay transcripts or audio for debugging. | | **Built-in metrics** | Measure latency, accuracy, tool-call effectiveness, and instruction compliance without custom instrumentation. | ### Getting started with Coval 1. Set up the integration on the [Integrations tab](/docs/inference/ai-assistants/integrations#coval) of your assistant. 2. Create seed scenarios in Coval that reflect your most important conversation paths. 3. Run simulations to validate assistant behavior before deploying new versions. 4. Add Coval evaluation steps to your CI/CD pipeline to catch regressions automatically. For setup details and required credentials, see the [Coval integration guide](/docs/inference/ai-assistants/integrations#coval). --- ### Importing Assistants > Source: https://developers.telnyx.com/docs/inference/ai-assistants/importing.md If you have voice assistants with another provider, you can import them to Telnyx in the portal or [via API](https://developers.telnyx.com/api-reference/assistants/import-assistants-from-external-provider#import-assistants-from-external-provider). ## Supported Providers Telnyx currently supports importing assistants from: - **Vapi** - Import your Vapi voice assistants with all configurations. - **ElevenLabs** - Import your ElevenLabs conversational AI agents. - **Retell** - Import your single- and multi-prompt Retell agents. --- ## Video Tutorial Watch this quick demonstration of importing a Vapi assistant to Telnyx: --- ## Import Walkthrough ### Importing from Vapi #### Viewing your Assistants From the [assistants page](https://portal.telnyx.com/#/ai/assistants), you can select **Import Assistants**. ![AI Assistant List Import](/img/list-assistants-import-vapi.png) #### Selecting your Provider On the next page, select **Vapi** from the provider list and securely store your Vapi API Key with Telnyx to initiate the import. *Any assistant that has already been imported will be overwritten with its latest version from Vapi.* ![Import Assistants Page](/img/import-assistants-vapi.png) #### Selecting your Assistants You can then select which Vapi assistants to import. *[Trial accounts](https://developers.telnyx.com/docs/account-setup/account-upgrade) have a limit of 1 AI Assistant.* ![Import Assistants Page](/img/select-vapi-ai-import.png) #### Testing your imported Assistant You should now see your new assistants listed and can click the pencil icon to edit or the telephone icon to test them in the portal. Check out the [quickstart tutorial](https://developers.telnyx.com/docs/inference/ai-assistants/no-code-voice-assistant) for more information on editing or creating our Assistants. ### Importing from ElevenLabs #### Viewing your Assistants From the [assistants page](https://portal.telnyx.com/#/ai/assistants), you can select **Import Assistants**. ![AI Assistant List Import](/img/list-assistants-import.png) #### Selecting your Provider On the next page, select **ElevenLabs** from the provider list and securely store your ElevenLabs API Key with Telnyx to initiate the import. *Any assistant that has already been imported will be overwritten with its latest version from ElevenLabs.* ![Import Assistants Page](/img/import-ai.png) #### Selecting your Assistants You can then select which ElevenLabs agents to import. *[Trial accounts](https://developers.telnyx.com/docs/account-setup/account-upgrade) have a limit of 1 AI Assistant.* ![Import Assistants Page](/img/select-ai-import.png) #### Testing your imported Assistant You should now see your new assistants listed and can click the pencil icon to edit or the telephone icon to test them in the portal. Check out the [quickstart tutorial](https://developers.telnyx.com/docs/inference/ai-assistants/no-code-voice-assistant) for more information on editing or creating our Assistants. --- # Supported Import Functionality ### Instructions Your assistant's instructions will be imported as is. ### Greeting (a.k.a. first message) Your assistant's greeting will be imported as is. ### LLM By default, the LLM assistant will be set to our flagship on-prem LLM, which provides optimal latency, cost, and intelligence. You can BYO LLM with third-party providers by storing an API Key. ### Voice When importing from Vapi or ElevenLabs, the voice will be imported as is. Otherwise, the LLM assistant will be set to our flagship on-prem TTS, which provides optimal latency, cost, and realism. You can BYO TTS with other third-party providers by storing an API Key. ### Dynamic Variables References to dynamic variables and their defaults will be imported as is. See our [guide](https://developers.telnyx.com/docs/inference/ai-assistants/dynamic-variables) on how to set these values per call. ### Tools Hangup, transfer, and webhook tools will be imported as is. ### MCP Servers (a.k.a. Integrations) Integrations with MCP Servers will be imported as is. ### Insights (a.k.a. analysis, success criteria, structured data...) Structured and unstructured call analysis configuration is imported as is. ### Data Retention If you have disabled data retention with another provider, this setting will be imported as is. ### Knowledge Bases Telnyx will not import your knowledge base by default. You can easily drag and drop files or import website content for your assistant in the assistant builder. ![Knowledge Base Page](/img/knowledge-base-upload.png) ### Secrets Telnyx will create placeholder integration secrets for you with the same name as they are configured in the importing provider. You will need to resupply the value of the secret on the Telnyx [integration secrets page](https://portal.telnyx.com/#/integration-secrets). ![Knowledge Base Page](/img/edit-integration-secret.png) --- ### Custom LLMs for Assistants > Source: https://developers.telnyx.com/docs/inference/ai-assistants/custom-llm.md In addition to standard third-party LLM providers like OpenAI, Gemini, and Groq, you can also power your AI Assistant with any public OpenAI-compatible chat completions endpoint. This includes models hosted using AWS Bedrock, Azure OpenAI, and Baseten or open source inference engines like vLLM and SGLang. You must have a publicly accessible OpenAI-compatible chat completions endpoint before proceeding with deployment. --- ## When to use custom LLM providers Custom LLM providers are ideal for scenarios where you need: - **Specific model requirements** - Access to proprietary models, fine-tuned models, or the latest releases not yet available through standard providers. - **Data residency and compliance** - Ensure your data stays within specific geographic regions or private cloud environments. - **Cost optimization** - Leverage enterprise agreements, reserved capacity, or self-hosted infrastructure for better economics at scale. - **Advanced model control** - Fine-tune parameters, adjust inference settings, or use specialized configurations for your use case. --- ## Azure For this guide, we will deploy [gpt-4o on Azure AI Foundry](https://ai.azure.com/). First, create or select a resource. ![Azure AI Foundry homepage showing resource selection interface](/img/foundry-homepage.png) Then, drill into the resource to find the API key and Azure OpenAI endpoint. ![Azure deployment details page displaying API key and endpoint URL](/img/foundry-deployment.png) Create or edit a Telnyx AI Assistant and in the `Agent` tab, check `Use Custom LLM`. Input the endpoint URL as the `Base URL` and append `/openai/v1` and create a new Integration Secret with your API Key. You will see a dropdown of all possible Azure models but only ones that you have deployed will validate an LLM connection. ![Telnyx portal Agent tab with Azure custom LLM configuration and model dropdown](/img/azure-in-portal.png) Once you save your assistant you will be able to immediately use your assistant in the Telnyx portal with the `Test Assistant` dropdown. ## Forward metadata to your custom LLM By default, Telnyx does not include your assistant's dynamic variables in requests to a custom LLM endpoint. If your model gateway or application needs those values, enable `forward_metadata` on the assistant's `external_llm` configuration. ```json { "external_llm": { "llm_api_key_ref": "integration_secret_id", "base_url": "https://your-llm-gateway.example.com/openai/v1", "model": "your-model-name", "forward_metadata": true } } ``` When `forward_metadata` is `true`, Telnyx adds a top-level `extra_metadata` object to the OpenAI-compatible chat completions request body sent to your custom LLM endpoint when dynamic variables are available. The field defaults to `false` when omitted. ```json { "model": "your-model-name", "messages": [ { "role": "system", "content": "..." }, { "role": "user", "content": "..." } ], "extra_metadata": { "customer_name": "Jane", "account_id": "acct_789", "telnyx_agent_target": "+13125550100", "telnyx_end_user_target": "+13125550123" } } ``` Use this when your external LLM service needs request context for routing, logging, personalization, or retrieval. `extra_metadata` is separate from OpenAI's native `metadata` field, so your endpoint must explicitly read `extra_metadata` from the request body. ## Baseten For this guide, we will deploy [Llama 3.3 70B on Baseten](https://www.baseten.co/library/llama-3-3-70b-instruct/). First, click `Deploy Now`. ![Baseten model library page showing Llama 3.3 70B with Deploy Now button](/img/baseten-deploy-now.png) Navigate to the deployment. ![Baseten deployment overview page for Llama 3.3 70B model](/img/baseten-deployment.png) Click the API Endpoint button to see the endpoint and generate an API Key. Save these details. ![Baseten deployment page showing API endpoint URL and key generation button](/img/baseten-api-key.png) After about 15 minutes, the deployment should be live. When it is complete, create or edit a Telnyx AI Assistant and in the `Agent` tab, check `Use Custom LLM`. Input the Baseten Endpoint URL for your deployment as the `Base URL` and create a new Integration Secret with your Baseten API Key. ![Telnyx portal custom LLM configuration showing Base URL input field](/img/custom-url.png) If your base URL supports an OpenAI-compatible /models endpoint the Model Name dropdown will populate automatically. Baseten deployments do not support this endpoint, so you can enter any name for your model here. You can also validate the connection is live before saving your assistant. ![Telnyx portal custom LLM validation interface with connection test button](/img/validate-custom-llm.png) Once you save your assistant you will be able to immediately use your assistant in the Telnyx portal with the `Test Assistant` dropdown as well as review metrics in your Baseten deployment. ![Baseten deployment dashboard displaying usage metrics and performance data](/img/baseten-metrics.png) --- ### Data Residency & Compliance FAQ > Source: https://developers.telnyx.com/docs/inference/data-residency.md This page answers common customer questions about where data is processed and stored, which providers are used, retention, and model training. Telnyx AI spans two products that handle **processing location** differently: - **Inference API** — chat completions, the responses endpoint, and related model APIs. - **Voice AI Assistants** — telephony-based conversational agents. The single most important thing to understand: Telnyx offers **hard controls for data at rest** (storage location and retention), but **does not offer hard controls for processing location**. Where a request is *processed* is **latency-based and best-effort** — it is influenced, not guaranteed. Under failover or capacity events, processing shifts to the next-best region rather than failing. This FAQ describes how the product works technically. It is not a legal commitment. Contractual terms — including data processing terms, training opt-outs, and any region commitments — are handled through your account team, a Data Processing Agreement (DPA), and applicable Telnyx terms. For written confirmations, [contact support](mailto:support@telnyx.com) or your Telnyx account manager. --- ## Processing vs. storage: the key distinction | | Processing in transit (best-effort, not guaranteed) | Storage at rest (hard control) | | --- | --- | --- | | **Inference API** | Latency-based, influenced by the **ingress domain** you call (`api.telnyx.com`, `api.telnyx.eu`, `api.telnyx.com.au`). Not tied to data locality. Not guaranteed. | Chat completions: **not stored**. Responses endpoint (stores conversations): governed by your **data locality** setting. | | **Voice AI Assistants** | Influenced by the **anchorsite** on the assistant's TeXML application. Best-effort, not guaranteed. | Governed by your **data locality** flag, plus the **data-retention** setting for conversation content. | [Data Locality](/docs/account-setup/data-locality) governs **storage at rest** for covered data types. Neither the data locality flag nor the anchorsite is a hard guarantee of where live **processing** happens. --- ## Inference API ### Where is Inference processing performed? Inference **processing in transit is latency-based and best-effort, influenced by the ingress domain you call**, not by your data locality setting: | Ingress domain | Preferred region | | --- | --- | | `api.telnyx.com` | US | | `api.telnyx.eu` | EU | | `api.telnyx.com.au` | APAC | Calling a regional ingress domain (for example, `api.telnyx.eu`) directs requests to the nearest GPU region for that domain on a best-effort basis. Telnyx does **not guarantee** the processing location: during failover or capacity events, requests are processed at the next-lowest-latency region rather than failing. See [Inference Regions & Availability](/docs/inference/models/regions) for the underlying GPU regions. ### Does Inference store my data? It depends on the endpoint: - **Chat completions endpoint** — **does not store** request or response data. - **Responses endpoint** — **stores conversations**. For stored data, your [Data Locality](/docs/account-setup/data-locality) setting dictates the storage region. ### Can Inference traffic be pinned to a specific region? Not as a hard guarantee. Routing is **latency-based and best-effort**: calling a regional ingress domain (for example, `api.telnyx.eu`) directs requests to that region under normal conditions, but Telnyx does **not guarantee** processing location. During failover or capacity events, requests are processed at the next-lowest-latency region rather than failing. If you have a strict compliance requirement for guaranteed processing location, [contact support](mailto:support@telnyx.com) to discuss what is possible for your account. --- ## Voice AI Assistants ### Where is Voice AI Assistant processing performed? For Voice AI Assistants, processing location is **influenced by the anchorsite** configured on the assistant's **TeXML application** — not by the data locality flag. Setting the anchorsite (for example, Frankfurt for the EU) directs media/processing to that region under normal conditions. The anchorsite is **best-effort, not a hard control**. Telnyx does not guarantee processing location: under failover or capacity events, processing can shift to another region rather than failing the call. ### Where is Voice AI Assistant data stored? **Storage location at rest is a hard control, governed by your [Data Locality](/docs/account-setup/data-locality) flag.** Retention of conversation content is further controlled by the **data-retention** setting (see [Data retention](#data-retention-and-model-training)). Recording storage can also be directed to your own storage destination, which Telnyx respects. ### Are call audio, transcripts, prompts, responses, summaries, or recordings ever handled outside the configured region? - **Processing** location is influenced by the assistant's anchorsite, but is **best-effort and not guaranteed**. - **Storage at rest** is a hard control, following your **data locality** flag. Recordings can be directed to a customer-controlled storage destination, which Telnyx respects. Telnyx does **not** contractually guarantee blanket "EU-only processing." Processing controls are best-effort only, "processing" is defined very broadly, and some components — for example, third-party STT/TTS providers, or operational/security/fraud handling — may involve activity outside a single region. The specifics depend on the providers and features you enable. Confirm written data commitments with your account team and DPA before making representations to your own customers. ### Example: EU-focused Voice AI setup A typical EU-oriented configuration combines: - **Data locality:** EU (Germany) — a hard control over storage at rest - **Anchorsite on the TeXML app:** an EU site (for example, Frankfurt) — best-effort influence over media/processing location - **Voice API endpoint:** `api.telnyx.eu` - **SIP endpoint:** `sip.telnyx.eu` This keeps storage in the EU (a hard control via data locality) and steers processing toward the EU (best-effort via the anchorsite). STT/TTS provider choice also matters — some providers are self-hosted by Telnyx and some are third parties (see below). --- ## STT, TTS, and LLM providers (Voice AI) For Voice AI Assistants, the STT, TTS, and LLM providers in use depend on the models and voices **you select**. Some are **self-hosted by Telnyx** (run on Telnyx-operated infrastructure); others are **third-party** services that Telnyx integrates with. This distinction matters for compliance: self-hosted models keep that processing step within Telnyx infrastructure, whereas third-party models route that step to the vendor. Hosting (self-hosted vs. third-party) is about *which infrastructure* performs the step, not a guarantee of *region*. Processing region is best-effort for all providers — see the [processing vs. storage](#processing-vs-storage-the-key-distinction) note above. ### Speech-to-text (STT) | Model | Provider | Hosting | | --- | --- | --- | | `deepgram/flux` | Deepgram | Self-hosted by Telnyx | | `deepgram/nova-3` | Deepgram | Self-hosted by Telnyx | | `deepgram/nova-2` | Deepgram | Self-hosted by Telnyx | | `assemblyai/universal-streaming` | AssemblyAI | Self-hosted by Telnyx | | `speechmatics/standard` | Speechmatics | Self-hosted by Telnyx | | `distil-whisper/distil-large-v2` | Whisper (English-only) | Self-hosted by Telnyx | | `azure/fast` | Azure | Third-party | | `soniox/stt-rt-v4` | Soniox | Third-party | | `xai/grok-stt` | xAI | Third-party | ### Text-to-speech (TTS) TTS is delivered through Telnyx's TTS gateway, which integrates multiple providers. The provider depends on the voice you select: | Provider | Hosting | | --- | --- | | Telnyx (in-house voices, including Telnyx Ultra) | Self-hosted by Telnyx | | Rime | Self-hosted by Telnyx | | Resemble | Self-hosted by Telnyx | | ElevenLabs | Third-party | | AWS | Third-party | | Azure | Third-party | | Minimax | Third-party | | Inworld | Third-party | | xAI | Third-party | See [Text to Speech voices](/docs/tts-stt/tts-available-voices) for the current voice catalog. ### Large language model (LLM) The assistant's model is served through Telnyx's inference platform. The model in use is the one you configure on the assistant. **Self-hosted by Telnyx** (open models served on Telnyx infrastructure) include the **Qwen** and **Moonshot (Kimi)** model families — for example, `Qwen/Qwen3-235B-A22B`, `moonshotai/Kimi-K2.5`, and `moonshotai/Kimi-K2.6`. **Third-party** models — including those from **Anthropic** (Claude), **OpenAI** (GPT), and **Google** (Gemini) — are **not self-hosted**. When you select one of these, the prompt is sent to that external provider to generate the response. The available models evolve over time — for the current catalog and which models are recommended for assistants, see [Models](/docs/inference/models). If data residency or third-party data sharing is a concern, choose a self-hosted model (a Qwen or Moonshot/Kimi model) to keep prompt and response generation on Telnyx infrastructure. Region remains best-effort even for self-hosted models. ### Can STT, TTS, or LLM processing be restricted to the EU? There is **no hard guarantee** of processing region for any provider — processing is best-effort. In addition: - **Self-hosted** providers keep that processing step on Telnyx infrastructure, but region remains best-effort. - **Third-party** providers route that step to the vendor, whose own region behavior applies. If you need STT, TTS, or LLM processing constrained to a specific region, [contact support](mailto:support@telnyx.com) so we can advise which self-hosted provider/model combinations best fit your requirement. Hard region guarantees are not offered for processing. --- ## Recordings ### Are call recordings disabled by default? No — for Voice AI Assistants, **call recordings are enabled by default**, and you can turn them off. When recordings are enabled, the recording is stored as Media Storage, which is subject to your [Data Locality](/docs/account-setup/data-locality) setting. Disable recording on the assistant (or per call) if you do not want recordings retained. --- ## Data retention and model training ### What does the data-retention setting control? Voice AI Assistants expose a **data-retention** privacy setting (`privacy_settings.data_retention`). It is **enabled by default**. When you disable it, the assistant stops persisting conversation **content** while continuing the minimum processing needed to run and bill the call. When `data_retention` is **disabled**, conversation content is **not retained**: | Item | Behavior when retention is off | | --- | --- | | Conversation messages / transcripts | Not persisted to the conversations store | | Insights | Not retained. An insight may be computed transiently in-memory to support live conversation behavior, but the conversation and its insights are not stored | | Transcript and assistant answer in observability logs | Not retained; replaced with placeholders (for example, `[transcript not available]` / `[answer not available]`) | | LLM request/response content logging | Disabled | | TTS cache | Disabled, so synthesized audio is not cached | A limited set of records is still **retained** even when conversation retention is off, because they are required to operate and bill the service: | Item | Behavior when retention is off | | --- | --- | | Latency / timing metrics | Retained (timing only, no conversation content) | | Billing, security, and fraud-prevention records | Retained as required for legitimate business and compliance purposes | The data-retention flag governs retention of **conversation content** for Voice AI Assistants. Disabling it stops persistence of conversation content and insights; it does not change where data that *is* retained lives — storage region is controlled by [Data Locality](/docs/account-setup/data-locality). Recordings are governed separately by the recording setting (see [Recordings](#recordings) above). For a guarantee tailored to your exact configuration (audio, tool inputs/outputs, memory, observability traces, and third-party provider logs), confirm in writing with your account team and DPA. ### Can a customer opt out of model improvement / training / evaluation? Customer data handling for model training is governed by Telnyx's applicable terms and DPA. If you require an opt-out from model improvement, training, or evaluation — for both input and output data, and covering Telnyx and any third-party AI providers in your configuration — [contact your account team](mailto:support@telnyx.com) to confirm the governing terms and document the opt-out. --- ## Usage reporting and billing ### Can usage be broken down by assistant, phone number, or metadata/tag? Usage and conversation data can be attributed using identifiers such as the assistant, the associated phone number, and metadata. For subscriber-level or per-tag billing breakdowns, [contact support](mailto:support@telnyx.com) to confirm which dimensions are available and how to structure metadata/tags for clean attribution. See [Agent Observability](/docs/inference/ai-assistants/agent-observability) and [Session Analysis](/docs/reporting/session-analysis). --- ## Related resources - [Data Locality](/docs/account-setup/data-locality) - [Inference Regions & Availability](/docs/inference/models/regions) - [Models](/docs/inference/models) - [Transcription Settings](/docs/inference/ai-assistants/transcription-settings) - [Text to Speech voices](/docs/tts-stt/tts-available-voices) - [Agent Observability](/docs/inference/ai-assistants/agent-observability) --- ### Session Analysis > Source: https://developers.telnyx.com/docs/reporting/session-analysis.md # Session Analysis API Usage Guide Understand the full cost and event tree for any Telnyx AI or voice session — from a single inference call to complex multi-product conversations. --- ## What is Session Analysis? Session Analysis gives you a complete view of costs and events for any usage session on the Telnyx platform: - **Full event tree** — parent events, child events, and their relationships - **Rollup costs** — per-event costs plus cumulative totals across the session - **Product linkages** — how different Telnyx products interacted in a single session This is especially valuable for AI-powered sessions, where a single call can involve multiple Telnyx products — AI Voice Assistant, inference, speech-to-text, text-to-speech, call control, SIP trunking, and more — all layered into one conversation. ### When to Use It | Use Case | Why Session Analysis Helps | | :---- | :---- | | **AI session cost breakdown** | See the full cost tree for an AI assistant call, broken down by inference turns, STT/TTS, and infrastructure costs | | **Troubleshooting** | Trace the complete event chain to debug unexpected call flow or billing issues | | **Usage auditing** | Verify which products and services were consumed during a session | | **Cost optimization** | Identify which components of an AI session drive the most cost and optimize accordingly | --- ## Core Concepts ### Record Types A **record type** identifies the category of usage event for a Telnyx product. Each record type maps to a specific product and has defined relationships to other record types. Example: `ai-voice-assistant` is the record type for AI Voice Assistant sessions, which have `inference` events as children. ### Sessions A **session** is a tree of related usage events. The **root event** is the primary event you're analyzing. **Child events** are related usage that occurred as part of that session. **Example:** An AI Voice Assistant call involves a `call-session` root with a `call-control` child, which in turn has an `ai-voice-assistant` child with `inference` events underneath — each representing one LLM turn in the conversation. ### Relationships Events are connected via **relationships**: | Relationship | Direction | Example | | :---- | :---- | :---- | | `child_of` | Parent → Child | An `ai-voice-assistant` event has `inference` children (one per LLM turn) | | `parent_of` | Child → Parent | An `inference` event has a parent `ai-voice-assistant` session | Relationships are defined via **field mappings** — the child's `local_field` matches the parent's `parent_field`. For example, `inference` events link to their `ai-voice-assistant` parent via the `conversation_id` field. ### Cost Rollup Each event node includes: - **`event_cost`** — Cost of this individual event - **`cumulative_cost`** — This event's cost plus all descendant costs The root event's `cumulative_cost` equals the total session cost. For AI sessions, this means the `call-session` cumulative cost includes everything — the AI assistant, each inference turn, STT, TTS, call control, and SIP trunking — rolled up into one number. --- ## Endpoints ### 1. List All Record Types ``` GET /v2/session_analysis/metadata ``` Returns all available record types, their relationships, and query parameter options. **Response includes:** - `record_types[]` — Array of all record types with their parent/child relationships - `query_parameters` — Valid parameters for session analysis queries - `meta` — Total count and last updated timestamp **Example:** ```shell curl "https://api.telnyx.com/v2/session_analysis/metadata" \ -H "Authorization: Bearer YOUR_API_KEY" ``` ### 2. Get Record Type Details ``` GET /v2/session_analysis/metadata/{record_type} ``` Returns metadata for a specific record type with additional details: - Valid child and parent relationships - Example query URLs - Recommended max depth for traversal **Example:** ```shell curl "https://api.telnyx.com/v2/session_analysis/metadata/ai-voice-assistant" \ -H "Authorization: Bearer YOUR_API_KEY" ``` ### 3. Get Session Analysis ``` GET /v2/session_analysis/{record_type}/{event_id} ``` Retrieves the full session analysis tree for a specific event. **Path Parameters:** | Parameter | Type | Description | | :---- | :---- | :---- | | `record_type` | string | The record type identifier (see [Record Types Reference](#record-types-reference) below) | | `event_id` | UUID | The event's unique identifier | You can find event IDs in the individual webhooks or detail records for the applicable event. **Which record type should I use?** If you're analyzing a call made with your AI assistant, start with `call-session`. This gives you the full picture — AI assistant sessions, inference events, call control, SIP trunking, and all other products in one tree. Use the metadata endpoint to explore other starting points for different products. **Query Parameters:** | Parameter | Type | Default | Description | | :---- | :---- | :---- | :---- | | `include_children` | boolean | `true` | Include child events in the response | | `max_depth` | integer | `2` | Maximum traversal depth (1-5) | | `expand` | string | `record` | Only `record` (include full record data) or `none` (omit record data) | | `date_time` | ISO 8601 | — | Timestamp to narrow index selection (improves performance) | **Example:** ```shell curl "https://api.telnyx.com/v2/session_analysis/call-session/a1b2c3d4-5e6f-7a8b-9c0d-1e2f3a4b5c6d?include_children=true&max_depth=3&date_time=2026-03-17" \ -H "Authorization: Bearer YOUR_API_KEY" ``` --- ## Response Structure ### Top-Level Fields ```json { "session_id": "4c9d22b0-1e4b-11f1-83a0-02420aef51a1", "cost": { "total": "0.009800", "currency": "USD" }, "root": { /* EventNode */ }, "meta": { "event_count": 3, "products": ["ai-voice-assistant", "inference", "callcontrol-cdrs"] } } ``` | Field | Description | | :---- | :---- | | `session_id` | Identifier for the analyzed session | | `cost.total` | Total session cost (sum of all events) | | `cost.currency` | ISO 4217 currency code | | `root` | Root event node (tree structure) | | `meta.event_count` | Total events in the tree | | `meta.products` | Unique products involved in the session | ### Event Node Structure Each event in the tree follows this structure: ```json { "id": "4c9d22b0-1e4b-11f1-83a0-02420aef51a1", "product": "callcontrol-cdrs", "event_name": "callcontrol-cdrs", "relationship": null, "cost": { "event_cost": "0.000000", "cumulative_cost": "0.009800", "currency": "USD" }, "links": { "self": "/v2/session_analysis/call-control/4c9d22b0-...", "records": "/v2/detail_records?record_type=call-control&id=4c9d22b0-..." }, "record": { /* Full detail record */ }, "children": [ /* Array of child EventNodes */ ] } ``` | Field | Description | | :---- | :---- | | `id` | Event identifier | | `product` | Internal product identifier | | `event_name` | Event type name | | `relationship` | How this event relates to its parent (null for root) | | `cost.event_cost` | Cost of this individual event | | `cost.cumulative_cost` | This event + all descendants | | `links.self` | Link to this analysis node | | `links.records` | Link to underlying detail records | | `record` | Full detail record data (varies by record type) | | `children` | Child event nodes | ### Relationship Object For child events, the `relationship` field explains the connection: ```json { "type": "child_of", "via": { "local_field": "conversation_id", "parent_field": "conversation_id" }, "parent_id": "4c9063e0-1e4b-11f1-b789-02420aef51a1" } ``` | Field | Description | | :---- | :---- | | `type` | Relationship type (`child_of` or `parent_of`) | | `via.local_field` | Field on this record that links to parent | | `via.parent_field` | Field on the parent record that's matched | | `parent_id` | Identifier of the parent event | Note: The `via` fields vary by relationship. AI-related linkages often use `conversation_id`, while voice/telephony relationships typically use `telnyx_session_id` or `call_session_id`. --- ## Example: AI Voice Assistant Session A customer makes an inbound call to an AI Voice Assistant using the Telnyx Web Dialer. The assistant handles a multi-turn conversation over ~152 seconds. **Query:** ```shell curl "https://api.telnyx.com/v2/session_analysis/call-session/a1b2c3d4-5e6f-7a8b-9c0d-1e2f3a4b5c6d?max_depth=5&include_children=true&date_time=2026-03-17" \ -H "Authorization: Bearer YOUR_API_KEY" ``` **Session tree:** ``` call-session (SIP Trunking) event: $0.0000 cumulative: $0.2128 ├── recording event: $0.0051 cumulative: $0.0051 ├── webrtc event: $0.0052 cumulative: $0.0052 ├── sip-trunking event: $0.0052 cumulative: $0.0052 └── call-control event: $0.0052 cumulative: $0.1973 └── ai-voice-assistant event: $0.1800 cumulative: $0.1921 ├── inference event: $0.0010 ├── inference event: $0.0015 ├── inference event: $0.0029 ├── inference event: $0.0033 └── inference event: $0.0034 ``` **Summary:** | Field | Value | | :---- | :---- | | Total session cost | $0.2128 | | Event count | 11 | | Duration | ~152 seconds | | Products involved | siptrunking-cdrs, recording-cdrs, webrtc-cdrs, callcontrol-cdrs, ai-voice-assistant, inference | **Key observations:** - The **call-session** root has $0.00 event cost — it's a container for the session, not a billable event itself. - **ai-voice-assistant** is the dominant cost at $0.18 (billed at $0.06/min for 3 billed minutes). Its cumulative cost of $0.1921 includes 5 inference events underneath. - Each **inference** event represents a single LLM call during the conversation — one per assistant turn. The cost per turn varies based on the model and token count. - **call-control** has a $0.0052 event cost but a $0.1973 cumulative cost — the difference is the ai-voice-assistant subtree rolling up underneath it. - **recording**, **webrtc**, and **sip-trunking** are leaf nodes with no children, so their event cost equals their cumulative cost. ### Understanding the AI Cost Structure In an AI-powered call, costs come from multiple layers: | Layer | Record Type | What You're Billed For | | :---- | :---- | :---- | | **AI Assistant** | `ai-voice-assistant` | Time-based (per minute of assistant usage) | | **LLM Inference** | `inference` | Per-turn (each LLM call is a separate event) | | **Call Control** | `call-control` | Time-based (per minute of call control usage) | | **Transport** | `sip-trunking`, `webrtc` | Time-based (per minute for the voice leg) | | **Add-ons** | `recording`, `speech-to-text`, `text-to-speech` | Per-use or per-minute depending on the product | Session Analysis rolls all of these into one tree, so you can see exactly where cost accumulates — whether it's the AI assistant minute rate, the number of inference turns, or the underlying voice transport. --- ## Common Use Cases ### Debugging High-Cost AI Sessions 1. Query the session with `max_depth=5` to get the full tree 2. Look at `meta.products` to see which products were involved 3. Compare `event_cost` across children — if `ai-voice-assistant` is the driver, check how many inference turns occurred underneath 4. Check `record` fields for rate information (`rate`, `rate_measured_in`) and duration ### Finding Related Events 1. Start from any event in the tree 2. Use `relationship.via` to understand the field linkage 3. Query the parent or children directly using their `id` ### Exporting for Analysis 1. Use `expand=none` to get just the tree structure without full records 2. Flatten the tree programmatically for spreadsheet import 3. Or keep full records and parse specific fields (e.g., `rate`, `duration_sec`) --- ## Best Practices ### Performance - **Provide `date_time`** when you know it — this narrows the index search and improves response time - **Use appropriate `max_depth`** — start with `2`, increase only if needed. AI sessions with inference events typically need `max_depth=4` or `5` to reach the leaf level. - **Use `expand=none`** if you only need cost rollups, not full records ### Querying - **Start from the root event** — the most complete picture comes from the top-level event - **Check metadata first** — `/metadata/{record_type}` shows expected children and recommended depth - **Use `links.records`** to get the underlying detail records for deeper analysis ### Error Handling | Status Code | Meaning | Action | | :---- | :---- | :---- | | 400 | Invalid parameters | Check `record_type` spelling, `event_id` format | | 403 | Forbidden | Verify API key has access to this user's data | | 404 | Event not found | Event may not exist, or `date_time` is incorrect | | 500 | Internal error | Retry with exponential backoff | --- ## Record Types Reference ### AI & Voice Intelligence | Record Type | Product | Description | Common Children | | :---- | :---- | :---- | :---- | | `ai-voice-assistant` | AI Voice Assistant | AI assistant sessions | `inference` | | `inference` | Inference | AI inference events (one per LLM turn) | — | | `inference-speech-to-text` | Inference | STT-specific inference | — | | `inference-text-to-speech` | Inference | TTS-specific inference | — | | `summarization` | Inference | Text summarization | — | | `embedding` | Inference | Vector embeddings | — | | `speech-to-text` | Speech-to-Text | STT transcription records | — | | `text-to-speech` | Text-to-Speech | TTS synthesis records | — | | `amd` | Answering Machine Detection | AMD invocations | — | | `noise-suppression` | Noise Suppression | Audio enhancement records | — | ### Voice & Telephony | Record Type | Product | Description | Common Children | | :---- | :---- | :---- | :---- | | `call-session` | N/A | Root-level record for grouping related voice product usage records | `call-control`, `sip-trunking`, `recording`, `webrtc`, `siprec-client`, `fax-api`, `conference-participant` | | `call-control` | Call Control (Programmable Voice) | Voice API call records | `ai-voice-assistant`, `amd`, `media-streaming`, `speech-to-text`, `text-to-speech`, `noise-suppression` | | `sip-trunking` | SIP Trunking | SIP trunking detail records | — | | `webrtc` | WebRTC | WebRTC call records | — | ### Media & Storage | Record Type | Product | Description | Common Children | | :---- | :---- | :---- | :---- | | `recording` | Call Recording | Call recording CDRs | — | | `media_storage` | Media Storage | Stored media files | — | | `media-streaming` | Media Streaming | Real-time audio streams | — | | `forking` | Media Forking | Forked media records | — | ### Conferencing | Record Type | Product | Description | Common Children | | :---- | :---- | :---- | :---- | | `conference` | Conference | Conference sessions | `conference-participant` | | `conference-participant` | Conference | Individual participant records | — | ### Messaging | Record Type | Product | Description | Common Children | | :---- | :---- | :---- | :---- | | `message` | Messaging | SMS/MMS message records | `ai-messaging-assistant` | ### Video | Record Type | Product | Description | Common Children | | :---- | :---- | :---- | :---- | | `room-session-event` | Programmable Video | Video room sessions | — | | `room-session-participant-event` | Programmable Video | Video participants | — | | `room-session-recording-event` | Programmable Video | Video recordings | — | | `room-session-composition-event` | Programmable Video | Video compositions | — | ### Other Products | Record Type | Product | Description | Common Children | | :---- | :---- | :---- | :---- | | `verify` | Verify (2FA) | Verification attempts | — | | `fax-api` | Fax API | Fax transmission records | — | | `branded-calling` | Branded Calling | Caller ID branding | — | | `siprec-client` | SIPREC | SIPREC recording sessions | — | | `sim_card_usage` | Wireless | SIM card data usage | — | | `customer-service-record` | CSR | Customer service records | — | --- ## Missions ### Voice Outreach > Source: https://developers.telnyx.com/docs/inference/missions.md Your AI agent can do more than answer questions — it can go out into the world and get things done. With **AI Missions** and the **Telnyx Missions skill**, you can give your agent a task like _"Find catering companies in Chicago and call them to negotiate quotes for a corporate event"_ and watch it execute the entire workflow: research, phone calls, follow-ups, and a final summary. This guide walks you through the full lifecycle — from setting up your agent with the Missions skill to reviewing call results in the Telnyx Portal. --- ## What you'll need Before you start, make sure you have: 1. **An OpenClaw agent** — This is the AI agent that will execute your mission. If you don't have one yet, follow the [OpenClaw quickstart](https://docs.openclaw.ai) to get set up. 2. **A Telnyx account with an API key** — Your agent needs API access to create assistants, schedule calls, and track missions. [Create a Telnyx account](https://telnyx.com/sign-up) if you don't have one, then [generate an API key](https://portal.telnyx.com/#/app/api-keys). 3. **The Telnyx Missions skill installed** — Install it from [ClawHub](https://clawhub.ai/dotcom-squad/telnyx-toolkit) and configure your `TELNYX_API_KEY` as an environment variable for your agent. That's it. The skill handles phone number selection and assignment automatically — your agent will pick an available number from your account when it needs to make calls. --- ## How it works Your agent uses several Telnyx APIs to orchestrate the work: - **Missions API** — creates a mission, plans steps, logs events, and tracks status - **Assistants API** — creates a voice assistant with custom prompts and schedules calls - **Numbers API** — finds an available phone number and assigns it to the assistant The Missions API tracks every step — what your agent planned, what it did, what succeeded, and what failed. You get a complete audit trail without having to monitor the agent in real time. --- ## Give your agent a mission Once your agent has the Telnyx Missions skill installed, you can describe tasks in natural language. The agent handles the rest. ### Example: Catering quote negotiation Here's a real example — asking your agent to find caterers and negotiate pricing: > _"Find catering companies in Chicago for a corporate event with 50 people. Call Lakefront Catering, Chicago Grill Co, and Prairie Table Events. Get a baseline quote from the first one, then use that to negotiate better rates with the others. Compare all quotes and recommend the best option."_ Your agent will: 1. **Create a mission** in the Telnyx Missions API to track the work 2. **Build an execution plan** with steps (assistant setup → baseline call → negotiation calls → comparison) 3. **Create a voice assistant** — a Telnyx AI Assistant configured as a "Catering Quote Negotiator" with a professional script tailored to your request 4. **Assign a phone number** from your account to the assistant for outbound calling 5. **Call the first caterer** to establish a baseline price 6. **Call remaining caterers** armed with the best quote so far, negotiating for better rates 7. **Monitor call completions** and capture conversation insights 8. **Summarize results** — comparing quotes and recommending the best option ### What the agent plans When the mission starts, your agent creates a structured plan. In the Portal, you can see each step and its status: ![Mission plan view showing completed steps: create assistant, call Lakefront Catering (baseline), call Chicago Grill Co (with best quote), call Prairie Table Events (with best quote), compare and recommend](/img/missions-plan-view.png) Each step has a status (`pending`, `in_progress`, `completed`, `failed`) so you can see exactly where things stand at any point. In this example, the agent's plan was: 1. Create catering negotiation assistant with dynamic variable placeholders 2. Call Lakefront Catering (baseline, no leverage) 3. Call Chicago Grill Co (with best quote from call 1) 4. Call Prairie Table Events (with best quote so far) 5. Compare quotes and recommend best option Notice how the agent strategically ordered the calls — getting a baseline first, then using that price as leverage in subsequent negotiations. --- ## Monitor progress in the Portal You don't need to watch your agent work. The Telnyx Portal gives you a dashboard view of every mission. ### Mission overview Navigate to the **AI Missions** section in the [Telnyx Portal](https://portal.telnyx.com) to see all missions, their status, and when they were last updated. The dashboard shows recent runs with result summaries at a glance: ![AI Missions overview showing recent runs including Corporate Catering Quotes (succeeded), Restaurant Reservation Screening, and Weather IVR Sweep missions](/img/missions-overview.png) ### Run details Click **View Run** to see the full run detail — status, timing, the original input, and structured result payload. This is where you'll find the agent's final analysis: ![Mission Run Detail showing succeeded status, result summary with quotes from 3 caterers, and structured result payload with per-person pricing and recommendation](/img/missions-run-detail.png) The result payload includes structured data — per-person quotes, negotiation notes, conversation IDs linking back to call recordings, and the agent's recommendation. This data is also accessible via the API. ### Linked agents and conversation history Scrolling down, the Portal shows which Telnyx AI Assistants your agent created and linked to the mission run. Below that, you can see every conversation — including which caterer was called, the call channel, and timestamps. ### Conversation playback Click any conversation to open the full transcript with audio playback. Here's the Chicago Grill Co negotiation — watch the agent use the Lakefront's $65 quote as leverage to negotiate down to $62: ![Conversation detail showing the Catering Quote Negotiator negotiating with Chicago Grill Co — starting at $58/person without dessert, then negotiating to $62 all-inclusive using the $65 competitor quote as leverage](/img/missions-conversation-detail.png) The conversation panel shows the full back-and-forth, audio waveform with playback controls, and latency metrics (STT, LLM, TTS) for each assistant turn. --- ## Review call results After calls complete, your agent captures conversation insights — structured summaries of what was discussed. ### Conversation insights For each completed call, the Missions skill can extract structured information using the [AI Insights API](/docs/inference/ai-insights/creating-insights/index) — pulling out key data points like pricing, availability, and terms into a structured format. ### The final summary When all calls are done, your agent compiles everything into a summary with recommendations. Here's what that looks like for our catering example: > **Mission Complete: Catering Quote Negotiation** > > Called 3 caterers for a 50-person corporate event: > > | Caterer | Quote (per person) | Negotiated? | Notes | > |---------|-------------------|-------------|-------| > | Lakefront Catering | $65 | Baseline | First call, established reference price | > | Chicago Grill Co | $62 | ✅ Yes, down from $68 | Beat the Lakefront quote when presented with it | > | Prairie Table Events | $65 | ❌ No match | Couldn't beat $65, matched it | > > **Recommendation:** Chicago Grill Co offers the best value at $62/person — $150 savings over the baseline for 50 guests. This summary is also stored as the mission's `result_summary` and `result_payload`, so it's permanently accessible via the API and Portal. --- ## How the voice calls work Behind the scenes, your agent uses the Telnyx AI Assistants platform to make calls. Here's what it sets up: ### Voice assistant creation The agent creates a Telnyx AI Assistant with: - A **system prompt** tailored to your specific request (e.g., _"You are calling catering companies to negotiate quotes for a 50-person corporate event..."_) - A **greeting** that opens the conversation naturally (e.g., _"Hi, I'm calling to inquire about catering for a corporate event. Do you have a moment?"_) - **Dynamic variables** that update between calls (e.g., injecting the current best quote before each negotiation call) - **Telephony features** enabled for voice calls ### Phone number assignment The agent finds an available phone number from your Telnyx account and assigns it to the assistant's voice connection. This is the caller ID that businesses will see. You need at least one phone number in your Telnyx account. If none are available, the agent will let you know and link you to the [number purchase page](https://portal.telnyx.com/#/app/numbers/search-numbers). ### Call scheduling Calls are scheduled during business hours to maximize the chance of reaching someone. The agent handles timezone awareness and won't schedule calls at 3 AM. ### Call monitoring After scheduling, the agent polls for call completion and retrieves: - **Call status** — completed, failed, no answer, busy - **Conversation transcript** — full text of what was said - **Conversation insights** — structured extraction of key data points (quotes, availability, terms) --- ## Tips for effective missions ### Be specific in your request The more detail you give, the better your agent performs: ``` ❌ "Find me a caterer" ✅ "Find 3 catering companies in downtown Chicago for a 50-person corporate lunch. Call each one, get per-person pricing, ask about dietary accommodation options, and negotiate using the best quote you get as leverage." ``` ### Let the agent plan first Your agent will create a plan before executing. If you want to review the plan before calls go out, say so: > _"Find caterers and create a plan, but wait for my approval before making any calls."_ ### Check the Portal for real-time status You don't need to keep chatting with your agent. The Portal shows live progress — check back when you get a notification that the mission is complete. ### Start small Try a mission with 2-3 calls first to see how it works. You can always scale up once you're comfortable with the workflow. --- ## What's next - **[AI Missions API Reference](/docs/inference/missions/index)** — Full API documentation for creating and managing missions programmatically - **[AI Assistants](/docs/inference/ai-assistants/async-tools/index)** — Learn about async tools and advanced assistant configurations - **[AI Insights](/docs/inference/ai-insights/creating-insights/index)** — Extract structured data from conversations - **[Telnyx Missions Skill on ClawHub](https://clawhub.ai/dotcom-squad/telnyx-toolkit)** — Install the skill for your OpenClaw agent --- ## Analytics ### Creating insights > Source: https://developers.telnyx.com/docs/inference/ai-insights/creating-insights.md This guide walks you through creating individual [AI Insights](https://portal.telnyx.com/#/ai/insights) in the Mission Control Portal. You'll learn how to define analysis instructions, use variables, and configure both normal and structured insights. ## Prerequisites - Access to the Telnyx Mission Control Portal. - At least one AI Assistant configured (recommended for testing). ## Accessing AI Insights 1. Log in to the [Mission Control Portal](https://portal.telnyx.com). 2. Navigate to **AI, Storage and Compute** > **AI Insights**. 3. You'll see a list of existing insights with their IDs, names, instructions, and creation dates. ![AI Insights Main Page](/img/ai-insights-main-page.png) ## Creating an insight Insights return free-form text responses based on your instructions. ### Step 1: Open the create dialog Click the **Create Insight** button in the top-right corner of the AI Insights page. ### Step 2: Configure basic settings In the Create Insight modal, you'll see: ![Create Insight Modal](/img/create-insight-modal.png) 1. **Name** (Required) - A descriptive identifier for your insight. - Example: "Conversation Summary", "Customer Sentiment", "Issue Classification". 2. **Instructions** - Detailed prompt describing what to analyze and extract. - Be specific about what information you want. - Include output format expectations. - Reference conversation elements (transcript, metadata, etc.). ### Step 3: Write effective instructions Good instructions are clear, specific, and actionable. Here are some examples: **Example 1: Conversation summary** ``` Summarize the conversation for use as future context. Include: - Key facts mentioned. - Decisions made. - User preferences expressed. - Action items or follow-ups needed. Keep the summary concise (2-3 sentences) and focus on information that would be useful in future conversations with this user. ``` **Example 2: Sentiment analysis** ``` Measure the positivity & negativity of the call and rate it from 1-5 in ascending order. Positivity: How positive, satisfied, or happy was the customer? (1=very negative, 5=very positive) Negativity: How frustrated, angry, or dissatisfied was the customer? (1=no negativity, 5=very negative) Provide your ratings and a brief explanation of why you assigned those scores. ``` **Example 3: Issue categorization** ``` Analyze the conversation and identify the primary issue or request. Categorize it into one of the following: - Technical Support. - Billing Question. - Feature Request. - General Inquiry. - Complaint. - Other. Also provide a brief description of the specific issue within that category. ``` ### Step 4: Use variables (optional) You can include dynamic variables in your instructions to provide context about the specific conversation. Click the **Add a variable** dropdown to see available options. **System variables:** - `{{telnyx_current_time}}` - Date and time of the conversation. - `{{telnyx_conversation_channel}}` - Channel type (phone_call, web_call, sms_chat). - `{{telnyx_agent_target}}` - Assistant's phone number or identifier. - `{{telnyx_end_user_target}}` - User's phone number or identifier. **Example with variables:** ``` Analyze this {{telnyx_conversation_channel}} conversation from {{telnyx_current_time}}. Identify if the user at {{telnyx_end_user_target}} expressed interest in any of our products or services. If so, list the products mentioned and their level of interest (high/medium/low). ``` You can also reference [custom dynamic variables](https://developers.telnyx.com/docs/inference/ai-assistants/dynamic-variables) that you've configured for your assistant. ### Step 5: Save the insight 1. Review your configuration. 2. Click **Save**. 3. The insight will appear in your insights list. ## Managing insights ### Editing an insight 1. Find the insight in the list. 2. Click the **edit icon** (pencil) on the right side of the row. 3. Modify the name or instructions. 4. Click **Save**. ### Copying an insight ID Each insight has a unique ID that you can use with the Memory API or for programmatic access: 1. Click the **copy icon** next to the insight ID. 2. The ID will be copied to your clipboard. 3. Use this ID in API calls or memory queries. Example ID format: `cfcc865c-d3d4-4823-8a4b-f0df57d9f56f` ## Configuring webhook delivery You can configure webhook URLs to automatically receive insight results when conversations complete. ### Via Insight Groups Set a webhook URL when creating or editing an Insight Group: 1. Navigate to [AI Insight Groups](https://portal.telnyx.com/#/ai/insights-groups). 2. Click **Create Insight Group** or edit an existing group. 3. Enter your webhook URL in the **Webhook URL** field. 4. Save the group. ![Create Insight Group Modal](/img/create-insight-group-modal.png) **Example:** ``` Webhook URL: https://api.mycompany.com/webhooks/ai-insights ``` All assistants using this group will send insights to this URL unless overridden. ### Per assistant override Override the group's webhook URL for a specific assistant: 1. Navigate to your AI Assistant configuration. 2. Go to the **Analysis** tab and select **Insights**. 3. Select an Insight Group. 4. Click **Edit selected** to modify the group settings. 5. Enter a different webhook URL for this assistant. 6. Save the assistant configuration. This is useful when different assistants need insights delivered to different systems or for testing with staging webhooks. ## Best practices ### Writing clear instructions 1. **Be Specific** - Clearly state what you want to extract. - ❌ "Analyze the call". - ✅ "Identify the customer's main complaint and rate the urgency from 1-5". 2. **Define Output Format** - Specify how you want the response structured. - ❌ "Tell me about sentiment". - ✅ "Rate sentiment from 1-10 and provide a one-sentence explanation". 3. **Provide Context** - Explain why the information matters. - ❌ "List products mentioned". - ✅ "List products the customer showed interest in purchasing, noting their budget concerns". 4. **Use Examples** - Show the format you expect ``` Categorize the issue as one of: - Billing (e.g., incorrect charges, payment questions). - Technical (e.g., service not working, setup help). - Account (e.g., upgrades, cancellations). ``` ### Testing your insights 1. **Start with Test Conversations** - Try your insight on a few sample conversations first. 2. **Review Results** - Check if the extracted information matches your expectations. 3. **Refine Instructions** - Adjust wording based on the results. 4. **Validate Accuracy** - Ensure consistent, reliable extraction across different conversation types. ### Naming conventions Use clear, descriptive names that indicate: - **What** is being analyzed: "Customer Sentiment", "Product Interest", "Issue Type". - **Why** it matters: "Escalation Needed", "Follow-up Required", "Compliance Check". - **Scope**: "Healthcare Compliance", "Sales Qualification", "Support Quality". ## Next steps - **[Use Structured Data](https://developers.telnyx.com/docs/inference/ai-insights/structured-insights)** - Create insights with consistent JSON schemas. - **[Create Insight Groups](https://developers.telnyx.com/docs/inference/ai-insights/insight-groups)** - Organize insights for your assistants. - **[Explore Use Cases](https://developers.telnyx.com/docs/inference/ai-insights/use-cases)** - Industry-specific examples. ## Related resources - [Dynamic Variables](https://developers.telnyx.com/docs/inference/ai-assistants/dynamic-variables) - Custom variables for your insights. - [Memory](https://developers.telnyx.com/docs/inference/ai-assistants/memory) - Using insight IDs in memory queries. - [Voice Assistant Configuration](https://developers.telnyx.com/docs/inference/ai-assistants/no-code-voice-assistant#insights) - Assigning insights to assistants. --- ### Use cases > Source: https://developers.telnyx.com/docs/inference/ai-insights/use-cases.md This guide provides complete, industry-specific examples of AI Insights implementations. Each use case includes insight configurations, group organization, webhook integration, and best practices. ## Healthcare ### Use case: Patient call quality & compliance **Business Need:** Ensure regulatory compliance (HIPAA) while monitoring patient interaction quality and identifying follow-up needs. ### Insight configurations **1. HIPAA compliance check** (structured) ``` Name: HIPAA Compliance Verification Instructions: Verify HIPAA compliance requirements were met during this call. Parameters: 1. disclosures_made - Type: boolean - Required: Yes - Description: Were required privacy disclosures made at the start of the call? 2. consent_obtained - Type: boolean - Required: Yes - Description: Was patient consent obtained before discussing PHI? 3. phi_handled_properly - Type: boolean - Required: Yes - Description: Was Protected Health Information handled according to HIPAA guidelines? 4. violations - Type: array - Required: No - Description: List any potential HIPAA violations or concerns 5. compliant - Type: boolean - Required: Yes - Description: Overall assessment: was this call HIPAA compliant? ``` **2. Patient care quality** (structured) ``` Name: Patient Care Assessment Instructions: Evaluate the quality of patient care and interaction. Parameters: 1. empathy_score - Type: number - Required: Yes - Description: Rate the assistant's empathy from 1-5 (1=robotic, 5=highly empathetic) 2. clarity_score - Type: number - Required: Yes - Description: Rate explanation clarity from 1-5 (1=confusing, 5=very clear) 3. questions_answered - Type: boolean - Required: Yes - Description: Were all patient questions adequately answered? 4. follow_up_needed - Type: boolean - Required: Yes - Description: Does this patient require follow-up contact? 5. urgency_level - Type: string - Required: Yes - Description: Urgency classification: "routine", "moderate", "urgent", "emergency" ``` **3. Appointment summary** (unstructured) ``` Name: Call Summary for Patient Record Instructions: Create a concise summary for the patient's medical record. Include: - Reason for call. - Symptoms or concerns discussed. - Instructions provided. - Any appointments scheduled. - Follow-up requirements. Keep summary professional and factual for medical record inclusion. ``` ### Insight group configuration ``` Name: Healthcare Patient Calls Webhook URL: https://ehr.healthcorp.com/api/webhooks/ai-insights Insights: - HIPAA Compliance Verification - Patient Care Assessment - Call Summary for Patient Record ``` ### Webhook integration example ```python @app.route('/api/webhooks/ai-insights', methods=['POST']) def handle_patient_call_insights(): data = request.json conversation_id = data['conversation_id'] insights = {i['insight_name']: i['result'] for i in data['insights']} # Check compliance compliance = insights.get('HIPAA Compliance Verification', {}) if not compliance.get('compliant'): # Alert compliance team alert_compliance_team( conversation_id=conversation_id, violations=compliance.get('violations', []) ) # Check if urgent follow-up needed care = insights.get('Patient Care Assessment', {}) if care.get('urgency_level') in ['urgent', 'emergency']: # Create urgent task create_urgent_followup( conversation_id=conversation_id, urgency=care['urgency_level'] ) # Store summary in EHR summary = insights.get('Call Summary for Patient Record') if summary: ehr.add_patient_note( conversation_id=conversation_id, note=summary, note_type='ai_assistant_call' ) return jsonify({'status': 'processed'}), 200 ``` ### Expected results ```json { "insights": [ { "insight_name": "HIPAA Compliance Verification", "result": { "disclosures_made": true, "consent_obtained": true, "phi_handled_properly": true, "violations": [], "compliant": true } }, { "insight_name": "Patient Care Assessment", "result": { "empathy_score": 5, "clarity_score": 4, "questions_answered": true, "follow_up_needed": true, "urgency_level": "routine" } }, { "insight_name": "Call Summary for Patient Record", "result": "Patient called regarding follow-up on recent lab results. Explained test findings indicating normal thyroid function. Patient had questions about medication dosing which were addressed. Scheduled 6-month follow-up appointment. No immediate concerns noted." } ] } ``` --- ## Customer support ### Use case: Support ticket automation & quality monitoring **Business Need:** Automatically categorize and route support calls, monitor agent performance, and ensure customer satisfaction. ### Insight configurations **1. Ticket classification** (structured) ``` Name: Support Ticket Classifier Instructions: Classify and triage the support request for ticket creation. Parameters: 1. issue_type - Type: string - Required: Yes - Description: Primary issue category: "technical", "billing", "account", "feature_request", "bug_report", "general" 2. product_area - Type: string - Required: No - Description: Which product or service is affected? 3. priority - Type: string - Required: Yes - Description: Priority level: "critical", "high", "medium", "low" 4. resolved - Type: boolean - Required: Yes - Description: Was the issue completely resolved during this call? 5. resolution_time_minutes - Type: number - Required: No - Description: If resolved, approximately how many minutes did resolution take? 6. tags - Type: array - Required: No - Description: Relevant tags (e.g., "password_reset", "refund_request", "api_error") ``` **2. Customer satisfaction** (structured) ``` Name: Customer Satisfaction Score Instructions: Assess customer satisfaction from the conversation. Parameters: 1. csat_score - Type: number - Required: Yes - Description: Customer satisfaction from 1-5 (1=very dissatisfied, 5=very satisfied) 2. sentiment - Type: string - Required: Yes - Description: Overall sentiment: "positive", "neutral", "negative" 3. frustration_level - Type: number - Required: Yes - Description: Customer frustration from 1-5 (1=not frustrated, 5=extremely frustrated) 4. likely_to_churn - Type: boolean - Required: Yes - Description: Based on the conversation, is the customer likely to cancel service? 5. positive_feedback - Type: array - Required: No - Description: Specific things the customer praised or appreciated ``` **3. Agent performance** (structured) ``` Name: Agent Quality Assessment Instructions: Evaluate the AI assistant's performance in handling this support request. Parameters: 1. professionalism_score - Type: number - Required: Yes - Description: Professional tone and communication from 1-5 2. problem_solving_score - Type: number - Required: Yes - Description: Effectiveness in solving the issue from 1-5 3. efficiency_score - Type: number - Required: Yes - Description: How efficiently was the issue handled from 1-5 4. escalation_needed - Type: boolean - Required: Yes - Description: Should this have been escalated to human agent? 5. improvement_areas - Type: array - Required: No - Description: Areas where the assistant could improve ``` ### Insight group configuration ``` Name: Support Call Analytics Webhook URL: https://support.mycompany.com/api/webhooks/ai-insights Insights: - Support Ticket Classifier - Customer Satisfaction Score - Agent Quality Assessment ``` ### Webhook integration example ```javascript app.post('/api/webhooks/ai-insights', async (req, res) => { res.status(200).send('OK'); const { conversation_id, insights } = req.body; const insightMap = Object.fromEntries( insights.map(i => [i.insight_name, i.result]) ); const classification = insightMap['Support Ticket Classifier']; const satisfaction = insightMap['Customer Satisfaction Score']; const performance = insightMap['Agent Quality Assessment']; // Create ticket if unresolved if (!classification.resolved) { await ticketing.create({ conversation_id, type: classification.issue_type, priority: classification.priority, product: classification.product_area, tags: classification.tags, customer_sentiment: satisfaction.sentiment }); } // Alert on potential churn if (satisfaction.likely_to_churn) { await alerts.send({ type: 'churn_risk', conversation_id, csat_score: satisfaction.csat_score, frustration: satisfaction.frustration_level }); } // Log agent performance metrics await analytics.track('agent_performance', { conversation_id, professionalism: performance.professionalism_score, problem_solving: performance.problem_solving_score, efficiency: performance.efficiency_score, escalation_needed: performance.escalation_needed }); }); ``` --- ## Sales ### Use case: Lead qualification & pipeline management **Business Need:** Automatically qualify inbound leads, score opportunities, and route to appropriate sales representatives. ### Insight configurations **1. Lead qualification** (structured) ``` Name: Sales Lead Qualifier Instructions: Assess the quality and readiness of this sales lead. Parameters: 1. budget_discussed - Type: boolean - Required: Yes - Description: Did the prospect discuss budget or pricing? 2. budget_range - Type: string - Required: No - Description: Budget range if mentioned: "under_10k", "10k_50k", "50k_100k", "over_100k", "not_disclosed" 3. decision_timeframe - Type: string - Required: Yes - Description: When will they decide: "immediate", "this_month", "this_quarter", "next_quarter", "exploring", "unknown" 4. authority_level - Type: string - Required: Yes - Description: Decision-making authority: "decision_maker", "influencer", "end_user", "researcher", "unknown" 5. pain_points - Type: array - Required: Yes - Description: Specific problems or needs mentioned by the prospect 6. competitor_mentions - Type: array - Required: No - Description: Names of competing solutions mentioned 7. lead_score - Type: number - Required: Yes - Description: Overall qualification score from 1-10 based on BANT criteria 8. recommended_action - Type: string - Required: Yes - Description: Next step: "immediate_followup", "schedule_demo", "send_proposal", "nurture", "disqualify" ``` **2. Product interest** (structured) ``` Name: Product Interest Tracking Instructions: Identify which products or features the prospect showed interest in. Parameters: 1. products_mentioned - Type: array - Required: Yes - Description: List of products discussed during the call 2. primary_interest - Type: string - Required: Yes - Description: The product they seemed most interested in 3. feature_priorities - Type: array - Required: No - Description: Specific features they asked about or emphasized 4. use_case - Type: string - Required: Yes - Description: Brief description of their intended use case 5. integration_requirements - Type: array - Required: No - Description: Systems they need to integrate with ``` **3. Objections & concerns** (unstructured) ``` Name: Sales Objections Analysis Instructions: Identify any objections, concerns, or hesitations the prospect expressed. Include: - Price/budget concerns. - Feature gaps or limitations. - Competitive comparisons. - Implementation concerns. - Trust or credibility questions. Provide actionable insights for sales team to address these objections. ``` ### Insight group configuration ``` Name: Sales Call Intelligence Webhook URL: https://crm.salesteam.com/api/webhooks/leads Insights: - Sales Lead Qualifier - Product Interest Tracking - Sales Objections Analysis ``` ### Webhook integration example ```javascript app.post('/api/webhooks/leads', async (req, res) => { res.status(200).send('OK'); const { conversation_id, insights } = req.body; const insightMap = Object.fromEntries( insights.map(i => [i.insight_name, i.result]) ); const qualification = insightMap['Sales Lead Qualifier']; const interest = insightMap['Product Interest Tracking']; const objections = insightMap['Sales Objections Analysis']; // Create or update lead in CRM const lead = await crm.leads.upsert({ source: 'ai_assistant', conversation_id, score: qualification.lead_score, budget_range: qualification.budget_range, timeframe: qualification.decision_timeframe, authority: qualification.authority_level, pain_points: qualification.pain_points, primary_interest: interest.primary_interest, products: interest.products_mentioned, use_case: interest.use_case, objections: objections }); // Route high-quality leads immediately if (qualification.lead_score >= 8) { const rep = await assignSalesRep(interest.primary_interest); await crm.tasks.create({ assigned_to: rep, lead_id: lead.id, type: 'immediate_followup', priority: 'high', notes: `High-quality lead (score: ${qualification.lead_score}). ${qualification.recommended_action}` }); // Send Slack notification await slack.notify({ channel: '#sales', message: `🔥 Hot lead! Score: ${qualification.lead_score}/10. Assigned to ${rep.name}.` }); } // Add to appropriate nurture campaign if (qualification.recommended_action === 'nurture') { await marketing.addToCampaign(lead.email, { campaign: `nurture_${qualification.decision_timeframe}`, interests: interest.products_mentioned }); } }); ``` --- ## E-commerce ### Use case: Customer service & order management **Business Need:** Handle order inquiries, identify dissatisfaction early, and capture product feedback. ### Insight configurations **1. Order inquiry classification** (structured) ``` Name: Order Support Classifier Instructions: Classify the type of order-related inquiry or issue. Parameters: 1. inquiry_type - Type: string - Required: Yes - Description: Type of inquiry: "order_status", "return_request", "product_question", "shipping_issue", "payment_problem", "cancel_order", "modify_order" 2. order_number - Type: string - Required: No - Description: Order number if mentioned 3. urgency - Type: string - Required: Yes - Description: Urgency level: "urgent", "moderate", "low" 4. resolution_provided - Type: boolean - Required: Yes - Description: Was a resolution or answer provided? 5. compensation_offered - Type: boolean - Required: No - Description: Was any compensation (refund, discount, credit) offered? 6. next_action - Type: string - Required: Yes - Description: Required next step: "none", "process_return", "issue_refund", "escalate", "ship_replacement", "cancel_order" ``` **2. Customer sentiment** (structured) ``` Name: E-commerce Customer Sentiment Instructions: Gauge customer satisfaction with their shopping experience. Parameters: 1. satisfaction_level - Type: string - Required: Yes - Description: Overall satisfaction: "very_satisfied", "satisfied", "neutral", "dissatisfied", "very_dissatisfied" 2. likely_to_repurchase - Type: boolean - Required: Yes - Description: Based on the conversation, is customer likely to purchase again? 3. likely_to_recommend - Type: number - Required: Yes - Description: NPS score from 0-10: How likely to recommend to a friend? 4. complaint_severity - Type: string - Required: No - Description: If complaining, severity: "minor", "moderate", "major" 5. praise_areas - Type: array - Required: No - Description: What aspects did they appreciate or praise? ``` **3. Product feedback** (unstructured) ``` Name: Product Feedback Collection Instructions: Capture any product feedback, suggestions, or quality issues mentioned. Include: - Specific products mentioned. - Positive feedback or features they loved. - Negative feedback or problems encountered. - Feature requests or suggestions. - Quality concerns. Focus on actionable insights for product and marketing teams. ``` ### Insight group configuration ``` Name: E-commerce Customer Insights Webhook URL: https://api.shop.com/webhooks/customer-insights Insights: - Order Support Classifier - E-commerce Customer Sentiment - Product Feedback Collection ``` ### Webhook integration example ```javascript app.post('/webhooks/customer-insights', async (req, res) => { res.status(200).send('OK'); const { conversation_id, insights } = req.body; const insightMap = Object.fromEntries( insights.map(i => [i.insight_name, i.result]) ); const orderClassification = insightMap['Order Support Classifier']; const sentiment = insightMap['E-commerce Customer Sentiment']; const feedback = insightMap['Product Feedback Collection']; // Process order actions switch (orderClassification.next_action) { case 'process_return': await orders.initiateReturn(orderClassification.order_number); break; case 'issue_refund': await orders.processRefund(orderClassification.order_number); break; case 'cancel_order': await orders.cancel(orderClassification.order_number); break; case 'ship_replacement': await orders.shipReplacement(orderClassification.order_number); break; } // Alert on poor experiences if (sentiment.satisfaction_level === 'very_dissatisfied') { await alerts.send({ type: 'customer_dissatisfaction', conversation_id, order_number: orderClassification.order_number, severity: orderClassification.complaint_severity, nps_score: sentiment.likely_to_recommend }); } // Collect product feedback if (feedback) { await productFeedback.create({ conversation_id, feedback: feedback, source: 'ai_assistant', sentiment: sentiment.satisfaction_level }); } // Segment for marketing if (!sentiment.likely_to_repurchase) { await marketing.addToWinBackCampaign({ conversation_id, reason: orderClassification.inquiry_type }); } }); ``` --- ## Financial services ### Use case: Fraud detection & compliance **Business Need:** Identify potential fraud, ensure regulatory compliance, and categorize financial inquiries. ### Insight configurations **1. Fraud risk assessment** (structured) ``` Name: Fraud Risk Detector Instructions: Assess potential fraud risk indicators in this conversation. Parameters: 1. risk_level - Type: string - Required: Yes - Description: Overall risk: "no_risk", "low", "medium", "high", "critical" 2. risk_indicators - Type: array - Required: No - Description: Specific fraud indicators detected (e.g., "urgency_pressure", "unusual_request", "inconsistent_info", "social_engineering_attempt") 3. verification_requested - Type: boolean - Required: Yes - Description: Did the caller request account changes or sensitive actions? 4. authentication_status - Type: string - Required: Yes - Description: Authentication status: "verified", "partially_verified", "not_verified", "failed_verification" 5. recommended_action - Type: string - Required: Yes - Description: Recommended action: "approve", "additional_verification", "escalate_to_fraud_team", "block_immediately" 6. confidence_score - Type: number - Required: Yes - Description: Confidence in risk assessment from 0-100% ``` **2. Compliance check** (structured) ``` Name: Financial Compliance Verification Instructions: Verify regulatory compliance requirements for financial services. Parameters: 1. required_disclosures_made - Type: boolean - Required: Yes - Description: Were all required financial disclosures made? 2. customer_consent_obtained - Type: boolean - Required: No - Description: Was consent obtained for account changes or data sharing? 3. pii_handled_properly - Type: boolean - Required: Yes - Description: Was Personally Identifiable Information handled securely? 4. recording_disclosure - Type: boolean - Required: Yes - Description: Was call recording disclosure made? 5. compliance_violations - Type: array - Required: No - Description: Any potential compliance violations or concerns 6. compliant - Type: boolean - Required: Yes - Description: Overall compliance status ``` **3. Inquiry categorization** (structured) ``` Name: Financial Inquiry Classifier Instructions: Categorize the type of financial inquiry or request. Parameters: 1. inquiry_category - Type: string - Required: Yes - Description: Primary category: "account_balance", "transaction_dispute", "card_activation", "fraud_report", "account_opening", "loan_inquiry", "investment_advice", "general_question" 2. account_type - Type: string - Required: No - Description: Account type involved: "checking", "savings", "credit_card", "loan", "investment", "multiple" 3. transaction_amount - Type: number - Required: No - Description: Dollar amount if transaction-related inquiry 4. resolution_status - Type: string - Required: Yes - Description: Resolution status: "resolved", "pending", "escalated", "requires_callback" 5. callback_required - Type: boolean - Required: Yes - Description: Does customer need a callback from specialist? ``` ### Insight group configuration ``` Name: Financial Services Security & Compliance Webhook URL: https://api.bank.com/webhooks/insights Insights: - Fraud Risk Detector - Financial Compliance Verification - Financial Inquiry Classifier ``` ### Webhook integration example ```python @app.route('/webhooks/insights', methods=['POST']) def handle_financial_insights(): data = request.json conversation_id = data['conversation_id'] insights = {i['insight_name']: i['result'] for i in data['insights']} fraud_risk = insights.get('Fraud Risk Detector', {}) compliance = insights.get('Financial Compliance Verification', {}) inquiry = insights.get('Financial Inquiry Classifier', {}) # Handle high fraud risk immediately if fraud_risk.get('risk_level') in ['high', 'critical']: fraud_team.alert({ 'conversation_id': conversation_id, 'risk_level': fraud_risk['risk_level'], 'indicators': fraud_risk['risk_indicators'], 'recommended_action': fraud_risk['recommended_action'], 'confidence': fraud_risk['confidence_score'] }) # Block if critical if fraud_risk['risk_level'] == 'critical': security.block_account_temporarily(conversation_id) # Handle compliance violations if not compliance.get('compliant'): compliance_team.alert({ 'conversation_id': conversation_id, 'violations': compliance['compliance_violations'], 'severity': 'high' }) # Log for audit audit_log.create({ 'event': 'compliance_violation', 'conversation_id': conversation_id, 'details': compliance }) # Route inquiry to appropriate team if inquiry.get('callback_required'): routing.create_callback({ 'conversation_id': conversation_id, 'category': inquiry['inquiry_category'], 'account_type': inquiry.get('account_type'), 'priority': 'high' if fraud_risk['risk_level'] != 'no_risk' else 'normal' }) return jsonify({'status': 'processed'}), 200 ``` --- ## Best practices across use cases ### 1. Start with core insights Begin with 2-3 essential insights: - ✅ Sentiment/satisfaction. - ✅ Primary classification. - ✅ Action required. Add more specialized insights once core metrics are validated. ### 2. Balance structured and unstructured Use structured insights for: - Metrics and scores. - Categories and classifications. - Boolean flags. Use unstructured insights for: - Summaries and context. - Open-ended feedback. - Nuanced analysis. ### 3. Configure appropriate webhooks - **Real-time action required** → Webhook to operational system. - **Analytics only** → Webhook to analytics platform. - **Manual review** → No webhook, use Portal. ### 4. Test with real conversations Before production: 1. Test insights on 10-20 real conversations. 2. Review accuracy of classifications. 3. Validate webhook integration. 4. Adjust instructions based on results. ### 5. Monitor and iterate Track these metrics: - Insight accuracy. - Webhook delivery success rate. - False positive/negative rates for classifications. - Time to process insights. Refine instructions monthly based on performance. ### 6. Secure sensitive data For regulated industries: - Use HTTPS for all webhooks. - Implement proper authentication. - Encrypt data at rest. - Maintain audit logs. - Follow industry-specific compliance requirements. ## Related resources - [Voice Assistant Quickstart](https://developers.telnyx.com/docs/inference/ai-assistants/no-code-voice-assistant) - Set up your AI assistant. - [Memory API](https://developers.telnyx.com/docs/inference/ai-assistants/memory) - Use insights in conversation memory. - [API Reference](/api-reference/assistants/list-assistants) - Programmatic insights access. --- ### Structured insights > Source: https://developers.telnyx.com/docs/inference/ai-insights/structured-insights.md Structured insights extract data in a predefined JSON schema format, providing consistent, machine-readable results perfect for analytics, dashboards, and downstream processing. ## When to use structured insights Choose structured insights when you need: - **Quantitative Metrics** - Scores, ratings, counts, percentages. - **Categorical Data** - Status values, types, priorities, classifications. - **Boolean Flags** - Yes/no decisions, presence checks, compliance indicators. - **Consistent Format** - Data that feeds into databases, dashboards, or analytics systems. - **Multiple Related Fields** - Complex data with multiple attributes. | Use Case | Unstructured | Structured | |----------|--------------|------------| | Open-ended summaries | ✅ | ❌ | | Sentiment scoring (1-5) | ❌ | ✅ | | Issue categorization | ❌ | ✅ | | Descriptive analysis | ✅ | ❌ | | Compliance flags | ❌ | ✅ | | Dashboard metrics | ❌ | ✅ | ## Creating a structured insight ### Step 1: Start creating an insight 1. Navigate to [AI Insights](https://portal.telnyx.com/#/ai/insights) in the Portal. 2. Click **Create Insight**. 3. Enter a name and basic instructions. ### Step 2: Enable structured data mode Click the **Collect as structured data** button to reveal the schema configuration interface. ![Structured Data Mode](/img/create-insight-structured-data.png) ### Step 3: Define parameters For each piece of data you want to extract, add a parameter with: 1. **Name** - The field name in the JSON output (e.g., `sentiment_score`, `issue_type`). 2. **Type** - The data type (string, number, boolean, etc.). 3. **Required** - Whether this field must always be present. 4. **Description** - Instructions for extracting this specific field. Click **Add parameter** to add additional fields to your schema. ## Parameter types ### String Text values - use for categories, descriptions, identifiers. **Example:** ``` Name: issue_category Type: string Description: The primary category of the customer's issue. Must be one of: "billing", "technical", "account", "general" ``` **Output:** ```json { "issue_category": "technical" } ``` ### Enum Predefined categories - use when you have a fixed set of possible values. The AI agent will select one value from your provided list. **Example:** ``` Name: sentiment Type: enum Enum Values (comma separated): positive, negative, neutral, mixed Description: Overall sentiment of the customer during the conversation ``` **Output:** ```json { "sentiment": "positive" } ``` Enums provide better accuracy than asking the AI agent to choose from options in a string description. They enforce strict value validation and make the AI agent's task clearer. ### Number Numeric values - use for scores, ratings, counts, percentages. **Example:** ``` Name: satisfaction_score Type: number Description: Customer satisfaction rating from 1-10 based on their tone and statements ``` **Output:** ```json { "satisfaction_score": 8 } ``` ### Integer Whole number values - use when you need integers without decimals (counts, quantities, IDs). **Example:** ``` Name: message_count Type: integer Description: Total number of messages sent by the customer in this conversation ``` **Output:** ```json { "message_count": 5 } ``` Use `integer` instead of `number` when you specifically need whole numbers. This provides clearer intent and can help the AI agent avoid returning decimal values. ### Boolean True/false values - use for flags, presence checks, yes/no decisions. **Example:** ``` Name: escalation_needed Type: boolean Description: True if the conversation requires escalation to a supervisor or specialist ``` **Output:** ```json { "escalation_needed": false } ``` ### Array Lists of values - use for multiple items of the same type. **Example:** ``` Name: products_mentioned Type: array Description: List of product names or SKUs mentioned during the conversation ``` **Output:** ```json { "products_mentioned": ["Widget Pro", "Widget Basic", "Extended Warranty"] } ``` ### Array (string) Lists of text values - use for multiple string items with enforced type safety. **Example:** ``` Name: issue_keywords Type: array (string) Description: Key terms or phrases mentioned by the customer that describe their issue ``` **Output:** ```json { "issue_keywords": ["billing", "refund", "overcharge"] } ``` ### Array (number) Lists of numeric values - use for multiple numbers with enforced type safety. **Example:** ``` Name: mentioned_prices Type: array (number) Description: All price points or dollar amounts mentioned during the conversation ``` **Output:** ```json { "mentioned_prices": [29.99, 49.99, 99.99] } ``` ### Array (boolean) Lists of true/false values - use for multiple boolean flags with enforced type safety. **Example:** ``` Name: feature_preferences Type: array (boolean) Description: Customer preferences for features in order: email notifications, SMS alerts, push notifications ``` **Output:** ```json { "feature_preferences": [true, false, true] } ``` Typed arrays (string, number, boolean) provide better type safety than the generic `array` type. Use them when you know all elements will be of the same specific type. ### Object Nested structures - use for complex related data. **Example:** ``` Name: customer_info Type: object Description: Extracted customer information including name, account number, and contact preference ``` **Output:** ```json { "customer_info": { "name": "Jane Smith", "account_number": "ACC-12345", "preferred_contact": "email" } } ``` ## Complete examples ### Example 1: Sentiment analysis **Configuration:** ``` Name: Customer Sentiment Analysis Instructions: Analyze the customer's emotional state throughout the conversation. Parameters: 1. positivity_score - Type: number - Required: Yes - Description: Rate positive sentiment from 1-5 (1=very negative, 5=very positive) 2. frustration_level - Type: number - Required: Yes - Description: Rate customer frustration from 1-5 (1=not frustrated, 5=very frustrated) 3. overall_sentiment - Type: string - Required: Yes - Description: Overall sentiment classification: "positive", "neutral", or "negative" 4. key_emotions - Type: array - Required: No - Description: List of specific emotions detected (e.g., "happy", "confused", "angry", "satisfied") ``` **Sample Output:** ```json { "positivity_score": 4, "frustration_level": 2, "overall_sentiment": "positive", "key_emotions": ["satisfied", "relieved", "appreciative"] } ``` ### Example 2: Sales qualification **Configuration:** ``` Name: Lead Qualification Instructions: Assess the sales opportunity from this conversation. Parameters: 1. budget_mentioned - Type: boolean - Required: Yes - Description: Did the prospect mention or discuss budget? 2. budget_range - Type: string - Required: No - Description: If mentioned, what budget range? (e.g., "under $1000", "$1000-$5000", "over $5000") 3. decision_timeframe - Type: string - Required: Yes - Description: When do they need to make a decision? ("immediate", "this_month", "this_quarter", "exploring", "unknown") 4. pain_points - Type: array - Required: Yes - Description: List of specific problems or needs mentioned 5. competitor_mentions - Type: array - Required: No - Description: Names of competing solutions mentioned 6. lead_score - Type: number - Required: Yes - Description: Qualification score from 1-10 based on buying signals ``` **Sample Output:** ```json { "budget_mentioned": true, "budget_range": "$1000-$5000", "decision_timeframe": "this_month", "pain_points": [ "Manual data entry taking too long", "Errors in current system", "No mobile access" ], "competitor_mentions": ["CompetitorX", "LegacyTool"], "lead_score": 8 } ``` ### Example 3: Support ticket categorization **Configuration:** ``` Name: Support Ticket Classification Instructions: Categorize and triage the support request. Parameters: 1. issue_type - Type: string - Required: Yes - Description: Primary issue type: "technical", "billing", "account", "feature_request", "bug_report" 2. priority - Type: string - Required: Yes - Description: Urgency level: "critical", "high", "medium", "low" 3. affected_service - Type: string - Required: No - Description: Which product/service is affected? 4. resolved - Type: boolean - Required: Yes - Description: Was the issue resolved during this conversation? 5. resolution_time_minutes - Type: number - Required: No - Description: If resolved, how many minutes did it take? 6. follow_up_needed - Type: boolean - Required: Yes - Description: Does this require follow-up action? 7. tags - Type: array - Required: No - Description: Relevant tags for categorization (e.g., "password_reset", "billing_dispute", "api_error") ``` **Sample Output:** ```json { "issue_type": "technical", "priority": "high", "affected_service": "API Integration", "resolved": true, "resolution_time_minutes": 12, "follow_up_needed": false, "tags": ["api_error", "authentication", "resolved"] } ``` ## Advanced mode Enable **Advanced mode** (checkbox at the top of the structured data section) to access additional schema configuration options: - Custom validation rules. - Enum constraints for string values. - Min/max constraints for numbers. - Pattern matching for strings. - Nested object definitions. ## Best practices ### 1. Keep schemas focused Don't try to extract everything in one insight. Create multiple focused insights instead: - ✅ Separate insights for "Sentiment" and "Issue Classification" - ❌ One massive insight trying to capture sentiment, classification, entities, summary, etc. ### 2. Make instructions clear Each parameter's description should be crystal clear: ``` ❌ Bad: "sentiment score" ✅ Good: "Rate overall sentiment from 1-10, where 1 is very negative, 5 is neutral, and 10 is very positive" ``` ### 3. Use enums for categories When you have a fixed set of categories, use the **enum** type instead of listing values in a string description: **❌ Less effective (using string with description):** ``` Type: string Description: Classification must be one of: "billing_question", "technical_support", "feature_request", "complaint", "general_inquiry" ``` **✅ Better (using enum type):** ``` Type: enum Enum Values (comma separated): billing_question, technical_support, feature_request, complaint, general_inquiry Description: The primary classification of the customer inquiry ``` Using the enum type provides better accuracy and enforces strict value validation. See the [Enum parameter type](#enum) section for more details. ### 4. Mark optional appropriately Only mark fields as required if they should always be extractable: - **Required**: `overall_sentiment` - should always be detectable - **Optional**: `competitor_mentioned` - may not come up in every conversation ### 5. Provide value ranges For numeric fields, specify the range: ``` Description: Urgency score from 1-5, where 1 is low priority and 5 is critical/urgent ``` ### 6. Test with edge cases Test your structured insights with: - Very short conversations. - Conversations where some information is missing. - Ambiguous or unclear discussions. - Multiple topics in one conversation. ## Next steps - **[Create Insight Groups](https://developers.telnyx.com/docs/inference/ai-insights/insight-groups)** - Organize your structured insights. - **[Explore Use Cases](https://developers.telnyx.com/docs/inference/ai-insights/use-cases)** - Industry-specific structured insight examples. ## Related resources - [Creating Insights](https://developers.telnyx.com/docs/inference/ai-insights/creating-insights) - Basic insight creation. - [Insight Groups](https://developers.telnyx.com/docs/inference/ai-insights/insight-groups) - Organizing insights. - [API Reference](/api-reference/assistants/list-assistants) - Programmatic access to insights. --- ### Insight groups > Source: https://developers.telnyx.com/docs/inference/ai-insights/insight-groups.md Insight Groups allow you to organize related insights into collections that can be assigned to AI Assistants and configured with webhook delivery. Groups provide a modular, reusable approach to conversation analysis. ## Overview An **Insight Group** is a named collection of insights with optional webhook configuration. Key features: - **Reusable** - Assign the same group to multiple assistants. - **Modular** - Mix and match insights across different groups. - **Flexible Delivery** - Configure unique webhook URLs per group. - **Organized** - Group insights by use case, department, or business function. ## Accessing insight groups 1. Log in to the [Mission Control Portal](https://portal.telnyx.com). 2. Navigate to **AI, Storage and Compute** > **AI Insights**. 3. Click the **AI Insight Groups** tab. ![AI Insight Groups Page](/img/ai-insight-groups-page.png) The page displays: - **ID** - Unique identifier for the group (copyable). - **Name** - Group name. - **Webhook URL** - Configured webhook endpoint (or "-" if none). - **Insights Count** - Number of insights in the group. - **Created At** - When the group was created. ## Creating an insight group ### Step 1: Open the create dialog Click the **Create Insight Group** button in the top-right corner. ### Step 2: Configure the group ![Create Insight Group Modal](/img/create-insight-group-modal.png) Fill in the following fields: #### Name (required) A descriptive name that indicates the group's purpose. **Examples:** - "Customer Service Analytics". - "Sales Qualification Metrics". - "Healthcare Compliance Checks". - "E-commerce Customer Insights". - "Support Ticket Classification". **Best practices:** - Use clear, business-oriented names. - Include the use case or department. - Avoid generic names like "Group 1" or "Test Group". #### Webhook URL (optional) The HTTPS endpoint where insight results will be sent after each conversation. **Format:** `https://your-domain.com/webhooks/insights` **When to use:** - You want real-time insight delivery to your application. - You're building dashboards or analytics systems. - You need to trigger actions based on insights. - You're integrating with external systems. **When to skip:** - You only need to view insights in the Portal. - You're using the API to fetch insights on-demand. - You're still testing and refining your insights. #### Insights (multi-select) Select which insights to include in this group: 1. Click the **Select an insight** dropdown. 2. Search or browse available insights. 3. Click to add an insight to the group. 4. Repeat to add multiple insights. **You can:** - Add multiple insights to one group. - Use the same insight in multiple groups. - Create groups with a single insight. - Modify group membership after creation. ### Step 3: Save the group 1. Review your configuration. 2. Click **Save**. 3. The group will appear in the Insight Groups list. ## Example configurations ### Example 1: Customer support group ``` Name: Customer Support Analytics Webhook URL: https://api.mycompany.com/webhooks/support-insights Insights: - Customer Sentiment Analysis - Issue Classification - Resolution Status - Follow-up Required ``` **Use Case:** Automatically analyze support calls and send results to your ticketing system. ### Example 2: Sales qualification group ``` Name: Sales Lead Qualification Webhook URL: https://crm.mycompany.com/webhooks/lead-insights Insights: - Budget Discussion - Decision Timeframe - Pain Points Identified - Competitor Mentions - Lead Score ``` **Use Case:** Score sales calls and update CRM with qualification data. ### Example 3: Compliance monitoring group ``` Name: Healthcare Compliance Webhook URL: https://compliance.healthcorp.com/insights Insights: - HIPAA Compliance Check - Required Disclosures Made - Consent Verification - Patient Information Handled ``` **Use Case:** Ensure regulatory compliance and maintain audit trail. ### Example 4: Quality assurance group ``` Name: Call Quality Metrics Webhook URL: - Insights: - Agent Performance - Script Adherence - Professional Tone - Issue Resolution Quality ``` **Use Case:** Manual review of call quality without webhook integration. ## Managing insight groups ### Editing a group 1. Find the group in the list. 2. Click the **edit icon** (pencil) on the right. 3. Modify name, webhook URL, or insights. 4. Click **Save**. **Note:** Changes to the group will apply to all assistants using this group. ### Copying a group ID Each group has a unique ID for API access: 1. Click the **copy icon** next to the group ID. 2. The ID will be copied to your clipboard. Example ID format: `a2708926-c060-480a-8631-041cb7304117` ## Assigning groups to assistants ### Via portal (during assistant creation) When creating or editing an AI Assistant: 1. Navigate to the **Insights** under **Analysis** tab. 2. Select an Insight Group from the dropdown. 3. Save the assistant configuration. See the [Voice Assistant Quickstart](https://developers.telnyx.com/docs/inference/ai-assistants/no-code-voice-assistant#insights) for detailed steps. ### Via API Use the `insight_settings` field when creating or updating an assistant: ```bash curl -X POST https://api.telnyx.com/v2/ai/assistants \ -H "Authorization: Bearer YOUR_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "Customer Support Assistant", "model": "moonshotai/Kimi-K2.5", "instructions": "You are a helpful support assistant...", "insight_settings": { "insight_group_id": "a2708926-c060-480a-8631-041cb7304117" } }' ``` ## Using insights from groups ### In conversation history View insight results in the conversation history: 1. Navigate to **AI Assistants** > select an assistant. 2. Go to the **Conversation History** under **Analysis** tab. 3. Click on a conversation. 4. Scroll to the **Insights** section to see results. ### With memory API Filter which insights are included when querying conversation memory: ```json { "memory": { "conversation_query": "metadata->user_id=eq.123&limit=5", "insight_query": "insight_ids=insight_abc,insight_def,insight_xyz" } } ``` **Get insight IDs from your group:** 1. View the group details in the Portal. 2. Copy the ID of each insight in the group. 3. Include those IDs in the `insight_query` parameter. Learn more in the [Memory documentation](https://developers.telnyx.com/docs/inference/ai-assistants/memory). ### Via webhooks If you configured a webhook URL, results are automatically delivered: ```json { "event_type": "conversation.insights.completed", "conversation_id": "conv_abc123", "insight_group_id": "a2708926-c060-480a-8631-041cb7304117", "insight_group_name": "Customer Support Analytics", "insights": [ { "insight_id": "insight_xyz789", "insight_name": "Customer Sentiment", "result": { "score": 8, "sentiment": "positive" } }, { "insight_id": "insight_abc456", "insight_name": "Issue Type", "result": { "category": "technical", "priority": "high" } } ] } ``` ## Organization strategies ### By use case Group insights by business function: - "Sales Qualification". - "Customer Support". - "Compliance Monitoring". - "Quality Assurance". **Benefits:** - Clear ownership by department. - Focused analytics per use case. - Easy to assign to specialized assistants. ### By delivery destination Group insights by webhook endpoint: - "CRM Integration Group" → `https://crm.company.com/insights`. - "Analytics Dashboard Group" → `https://analytics.company.com/insights`. - "Ticket System Group" → `https://tickets.company.com/insights`. **Benefits:** - Simplified webhook management. - Targeted data delivery. - System-specific insight collections. ### By analysis type Group insights by the kind of analysis: - "Quantitative Metrics" (scores, ratings, counts). - "Categorical Classification" (types, statuses, priorities). - "Qualitative Analysis" (summaries, descriptions). **Benefits:** - Consistent data structures. - Easier downstream processing. - Clear analytical purpose. ### Hybrid approach Combine strategies for complex scenarios: - "Sales - Quantitative" (lead scoring, budget ranges). - "Sales - Qualitative" (pain points, objections). - "Support - Urgent" (critical issues only). - "Support - Complete" (all support metrics). ## Best practices ### 1. Start small Begin with a focused group: - ✅ 2-4 related insights. - ❌ 15+ insights covering everything. You can always add more insights later. ### 2. Test without webhooks first When creating a new group: 1. Leave webhook URL empty initially. 2. Assign to a test assistant. 3. Review results in the Portal. 4. Add webhook once validated. ### 3. Use descriptive names Make group purpose immediately clear: - ✅ "Healthcare Compliance Checks". - ✅ "E-commerce Order Analysis". - ❌ "Group 1". - ❌ "Test". ### 4. Document webhook endpoints Maintain documentation of: - What each webhook URL expects. - Who owns the endpoint. - What system processes the insights. - How to troubleshoot delivery issues. ### 5. Version your groups When making significant changes: 1. Create a new group (e.g., "Support Analytics v2"). 2. Test with a subset of assistants. 3. Migrate fully once validated. 4. Retire the old group. ### 6. Monitor insights count Keep groups manageable: - **1-5 insights**: Focused, fast analysis. - **5-10 insights**: Comprehensive, still efficient. - **10+ insights**: May be slower, consider splitting. ## Troubleshooting ### Insights not appearing **Check:** - Is the group assigned to the assistant? - Did a conversation complete after assignment? - Are the individual insights configured correctly? ### Webhook not receiving data **Check:** - Is the webhook URL correct and accessible? - Is the endpoint returning 200 OK status? - Check webhook logs in your application. ### Wrong insights in group **Solution:** 1. Edit the group. 2. Remove incorrect insights. 3. Add correct insights. 4. Save changes. Changes apply to future conversations immediately. ## Next steps - **[Explore Use Cases](https://developers.telnyx.com/docs/inference/ai-insights/use-cases)** - Industry-specific group examples. - **[Assign to Assistants](https://developers.telnyx.com/docs/inference/ai-assistants/no-code-voice-assistant#insights)** - Connect groups to your AI assistants. ## Related resources - [Creating Insights](https://developers.telnyx.com/docs/inference/ai-insights/creating-insights) - Build insights for your groups. - [Structured Insights](https://developers.telnyx.com/docs/inference/ai-insights/structured-insights) - Define consistent data schemas. - [Memory](https://developers.telnyx.com/docs/inference/ai-assistants/memory) - Use insights with conversation memory. - [API Reference](/api-reference/assistants/list-assistants) - Programmatic group management. --- ### Embeddings > Source: https://developers.telnyx.com/docs/inference/embeddings.md In this tutorial, you'll learn how to: - Upload documents to [Telnyx Storage](https://telnyx.com/products/cloud-storage) - Transform these documents into embeddings, enabling a language model to retrieve relevant sections of your documents - Provide the storage bucket as context for the language model ## Upload your documents You can upload objects to Telnyx's S3-Compatible storage API using our [quickstart](https://developers.telnyx.com/docs/cloud-storage/quick-start) or with our [drag-and-drop interface in the portal](https://portal.telnyx.com/#/storage/buckets). ## Embed your documents Once you've uploaded your documents, you can [embed them via API](https://developers.telnyx.com/api-reference/embeddings/embed-url-content#embed-url-content) or by clicking the "Embed for AI Use" button in the portal while viewing your storage bucket's contents. Behind the scenes, your documents will be processed into sections and each section will be "embedded" based on its contents. Later, when a user asks a language model a question, it will automatically be provided with the most relevant sections of documents from the bucket to help answer the question. ## Chat over your documents Once your documents are embedded, you can try it out in the [AI Playground in the portal](https://portal.telnyx.com/#/ai/playground) by selecting your embedded bucket from the storage dropdown. You can also use embeddings via our [chat completions API](https://developers.telnyx.com/api-reference/openai-chat/create-a-chat-completion-openai-compatible). Here is a Python example. Make sure you have set the `TELNYX_API_KEY` environment variable. Also, update the `question` and `bucket` variables in the sample code. ```python import os from openai import OpenAI client = OpenAI( api_key=os.getenv("TELNYX_API_KEY"), base_url="https://api.telnyx.com/v2/ai/openai", ) question = "" bucket = "" chat_completion = client.chat.completions.create( messages=[ { "role": "user", "content": question } ], model="zai-org/GLM-5.1-FP8", stream=True, tools=[ { "type": "retrieval", "retrieval": { "bucket_ids": [bucket] } } ] ) for chunk in chat_completion: if chunk.choices[0].delta.content: print(chunk.choices[0].delta.content, end="", flush=True) ``` --- ### Clusters > Source: https://developers.telnyx.com/docs/inference/clusters.md In this tutorial, you'll learn: - How [Embeddings](https://developers.telnyx.com/api-reference/embeddings/embed-url-content#embed-url-content) and [Clusters](https://developers.telnyx.com/api-reference/clusters/compute-new-clusters#compute-new-clusters) work - How to leverage them to identify common themes in your data # Embeddings and Clusters Embeddings are numerical representations of concepts within text, image, or audio data. [The representation is a real-valued vector that encodes the meaning of the word in such a way that the words that are closer in the vector space are expected to be similar in meaning](https://en.wikipedia.org/wiki/Word_embedding) Quantifying the semantic similarity of your data opens up several possibilities. For instance, by embedding a Telnyx storage bucket, you can [search for similar content](https://developers.telnyx.com/api-reference/embeddings/embed-url-content#embed-url-content) within your bucket. This tutorial is focused on another application of embeddings: analyzing how your semantic data is [clustered](https://en.wikipedia.org/wiki/Cluster_analysis), which provides insight into common themes and niche subtopics. For example, pictured below are clusters of embeddings computed for the novel The Great Gatsby. ![Gatsby clusters](/img/gatsby-cluster.png) # Clustering content with Telnyx ## Embed your documents Embedding your content in a Telnyx storage bucket is a prerequisite for computing these clusters. For more information, check out our [Embeddings](https://developers.telnyx.com/docs/inference/embeddings) tutorial. ## Identify clusters Once your documents are embedded, you can [compute clusters](https://developers.telnyx.com/api-reference/clusters/compute-new-clusters#compute-new-clusters) via API. The optional `prefix` and `files` parameters allow you to specfiy a subset of your bucket you would like to cluster. The `min_cluster_size` and `min_subcluster_size` parameters control how clusters are identified. Top-level clusters should be thought of as identifying broad themes in your data. Choose `min_cluster_size` based on the minimum data points you would like to constitute a broader theme. Sub-clusters should be thought of as identifying more specific topics within a broader theme. Choose `min_subcluster_size` based on the minimum data points you would like to constitute a more niche subtopic. ## Identifying themes in The Great Gatsby To demonstrate embedding and clustering a Telnyx storage bucket, we will be using the text from The Great Gatsby. ### Upload to Telnyx Storage You can upload objects to Telnyx's S3-Compatible storage API using our [quickstart](https://developers.telnyx.com/docs/cloud-storage/quick-start) or with our [drag-and-drop interface in the portal](https://portal.telnyx.com/#/storage/buckets). ### Embed your documents Once you've uploaded your documents, you can [embed them via API](https://developers.telnyx.com/api-reference/embeddings/embed-url-content#embed-url-content) or by clicking the "Embed for AI Use" button in the portal while viewing your storage bucket's contents. Behind the scenes, your documents will be processed into chunks and each chunk will be "embedded" based on its contents. Each chunk will be a single data point used in the clustering step. ### Compute clusters You can compute multiple clusterings on the same data. This is helpful to tweak the parameters to find the best clusters for your data. Below is an example API request ``` $ curl --request POST \ --url https://api.telnyx.com/v2/ai/clusters \ --header "Authorization: Bearer $TELNYX_API_KEY" \ --header 'Content-Type: application/json' \ --data '{ "bucket": "cluster-gatsby", "min_cluster_size": 50, "min_subcluster_size": 10 }' ``` And the response ``` {"data":{"task_id":"04dd624f-c9b3-4fc8-8cec-492c8696e9ea"}} ``` ### Inspect clusters You can then take that `task_id` and view the clusters structured as JSON via ``` $ curl --request GET \ --url "https://api.telnyx.com/v2/ai/clusters/04dd624f-c9b3-4fc8-8cec-492c8696e9ea?show_subclusters=true" \ --header "Authorization: Bearer $TELNYX_API_KEY" ``` If you want to see example data from each cluster, you can also pass the `top_n_nodes` query parameter which will include the top N most central data points for each cluster. You can also view a simple graph of the clusters via ``` $ curl --request GET \ --url "https://api.telnyx.com/v2/ai/clusters/04dd624f-c9b3-4fc8-8cec-492c8696e9ea/graph" \ --header "Authorization: Bearer $TELNYX_API_KEY" --output clusters.png ``` If you want to look at a cluster's subclusters, you can pass the `cluster_id` query parameter. Here is a closer look at the sub-clusters related to the cluster for "Daisy's Past" using this endpoint ![Gatsby clusters](/img/gatsby-daisy-subcluster.png) The initial parameters can have a large effect on the computed clusters, and the "right" clusters depend heavily on your data set and your goals, so you may have to play around a bit to find what works best. The general idea is that raising `min_cluster_size` will result in broader, more generic clusters. You can also compute as many configurations over your data as you like so you have multiple ways of clustering your data if you'd like. --- ## API Reference (Assistants) ### Assistants - [List assistants](https://developers.telnyx.com/api-reference/assistants/list-assistants.md): Retrieve a list of all AI Assistants configured by the user. - [Create an assistant](https://developers.telnyx.com/api-reference/assistants/create-an-assistant.md): Create a new AI Assistant. - [Get an assistant](https://developers.telnyx.com/api-reference/assistants/get-an-assistant.md): Retrieve an AI Assistant configuration by `assistant_id`. - [Update an assistant](https://developers.telnyx.com/api-reference/assistants/update-an-assistant.md): Update an AI Assistant's attributes. - [Delete an assistant](https://developers.telnyx.com/api-reference/assistants/delete-an-assistant.md): Delete an AI Assistant by `assistant_id`. - [Assistant Chat (BETA)](https://developers.telnyx.com/api-reference/assistants/assistant-chat-beta.md): This endpoint allows a client to send a chat message to a specific AI Assistant. The assistant processes the message and returns a relevant reply based on the… - [Assistant Sms Chat](https://developers.telnyx.com/api-reference/assistants/assistant-sms-chat.md): Send an SMS message for an assistant. This endpoint: - [Clone Assistant](https://developers.telnyx.com/api-reference/assistants/clone-assistant.md): Clone an existing assistant, excluding telephony and messaging settings. - [Import assistants from external provider](https://developers.telnyx.com/api-reference/assistants/import-assistants-from-external-provider.md): Import assistants from external providers. Any assistant that has already been imported will be overwritten with its latest version from the importing provider. - [List scheduled events](https://developers.telnyx.com/api-reference/assistants/list-scheduled-events.md): Get scheduled events for an assistant with pagination and filtering - [Create a scheduled event](https://developers.telnyx.com/api-reference/assistants/create-a-scheduled-event.md): Create a scheduled event for an assistant - [Get a scheduled event](https://developers.telnyx.com/api-reference/assistants/get-a-scheduled-event.md): Retrieve a scheduled event by event ID - [Delete a scheduled event](https://developers.telnyx.com/api-reference/assistants/delete-a-scheduled-event.md): If the event is pending, this will cancel the event. Otherwise, this will simply remove the record of the event. - [List assistant tests with pagination](https://developers.telnyx.com/api-reference/assistants/list-assistant-tests-with-pagination.md): Retrieves a paginated list of assistant tests with optional filtering capabilities - [Create a new assistant test](https://developers.telnyx.com/api-reference/assistants/create-a-new-assistant-test.md): Creates a comprehensive test configuration for evaluating AI assistant performance - [Get all test suite names](https://developers.telnyx.com/api-reference/assistants/get-all-test-suite-names.md): Retrieves a list of all distinct test suite names available to the current user - [Get test suite run history](https://developers.telnyx.com/api-reference/assistants/get-test-suite-run-history.md): Retrieves paginated history of test runs for a specific test suite with filtering options - [Trigger test suite execution](https://developers.telnyx.com/api-reference/assistants/trigger-test-suite-execution.md): Executes all tests within a specific test suite as a batch operation - [Get assistant test by ID](https://developers.telnyx.com/api-reference/assistants/get-assistant-test-by-id.md): Retrieves detailed information about a specific assistant test - [Update an assistant test](https://developers.telnyx.com/api-reference/assistants/update-an-assistant-test.md): Updates an existing assistant test configuration with new settings - [Delete an assistant test](https://developers.telnyx.com/api-reference/assistants/delete-an-assistant-test.md): Permanently removes an assistant test and all associated data - [Get test run history for a specific test](https://developers.telnyx.com/api-reference/assistants/get-test-run-history-for-a-specific-test.md): Retrieves paginated execution history for a specific assistant test with filtering options - [Trigger a manual test run](https://developers.telnyx.com/api-reference/assistants/trigger-a-manual-test-run.md): Initiates immediate execution of a specific assistant test - [Get specific test run details](https://developers.telnyx.com/api-reference/assistants/get-specific-test-run-details.md): Retrieves detailed information about a specific test run execution - [Get all versions of an assistant](https://developers.telnyx.com/api-reference/assistants/get-all-versions-of-an-assistant.md): Retrieves all versions of a specific assistant with complete configuration and metadata - [Get a specific assistant version](https://developers.telnyx.com/api-reference/assistants/get-a-specific-assistant-version.md): Retrieves a specific version of an assistant by assistant_id and version_id - [Update a specific assistant version](https://developers.telnyx.com/api-reference/assistants/update-a-specific-assistant-version.md): Updates the configuration of a specific assistant version. Can not update main version - [Delete a specific assistant version](https://developers.telnyx.com/api-reference/assistants/delete-a-specific-assistant-version.md): Permanently removes a specific version of an assistant. Can not delete main version - [Promote an assistant version to main](https://developers.telnyx.com/api-reference/assistants/promote-an-assistant-version-to-main.md): Promotes a specific version to be the main/current version of the assistant. This will delete any existing canary deploy configuration and send all live produc… - [Get Canary Deploy](https://developers.telnyx.com/api-reference/assistants/get-canary-deploy.md): Endpoint to get a canary deploy configuration for an assistant. - [Create Canary Deploy](https://developers.telnyx.com/api-reference/assistants/create-canary-deploy.md): Endpoint to create a canary deploy configuration for an assistant. - [Update Canary Deploy](https://developers.telnyx.com/api-reference/assistants/update-canary-deploy.md): Endpoint to update a canary deploy configuration for an assistant. - [Delete Canary Deploy](https://developers.telnyx.com/api-reference/assistants/delete-canary-deploy.md): Endpoint to delete a canary deploy configuration for an assistant. - [Get assistant texml](https://developers.telnyx.com/api-reference/assistants/get-assistant-texml.md): Get an assistant texml by `assistant_id`. - [Test Assistant Tool](https://developers.telnyx.com/api-reference/assistants/test-assistant-tool.md): Test a webhook tool for an assistant - [Get All Tags](https://developers.telnyx.com/api-reference/assistants/get-all-tags.md) - [Add Assistant Tag](https://developers.telnyx.com/api-reference/assistants/add-assistant-tag.md) - [Remove Assistant Tag](https://developers.telnyx.com/api-reference/assistants/remove-assistant-tag.md) - [Add Assistant Tool](https://developers.telnyx.com/api-reference/assistants/add-assistant-tool.md) - [Remove Assistant Tool](https://developers.telnyx.com/api-reference/assistants/remove-assistant-tool.md) ### Integrations - [List Integrations](https://developers.telnyx.com/api-reference/integrations/list-integrations.md): List all available integrations. - [List User Integrations](https://developers.telnyx.com/api-reference/integrations/list-user-integrations.md): List user setup integrations - [Get User Integration connection By Id](https://developers.telnyx.com/api-reference/integrations/get-user-integration-connection-by-id.md): Get user setup integrations - [Delete Integration Connection](https://developers.telnyx.com/api-reference/integrations/delete-integration-connection.md): Delete a specific integration connection. - [List Integration By Id](https://developers.telnyx.com/api-reference/integrations/list-integration-by-id.md): Retrieve integration details ### MCP Servers - [List MCP Servers](https://developers.telnyx.com/api-reference/mcp-servers/list-mcp-servers.md): Retrieve a list of MCP servers. - [Create MCP Server](https://developers.telnyx.com/api-reference/mcp-servers/create-mcp-server.md): Create a new MCP server. - [Get MCP Server](https://developers.telnyx.com/api-reference/mcp-servers/get-mcp-server.md): Retrieve details for a specific MCP server. - [Update MCP Server](https://developers.telnyx.com/api-reference/mcp-servers/update-mcp-server.md): Update an existing MCP server. - [Delete MCP Server](https://developers.telnyx.com/api-reference/mcp-servers/delete-mcp-server.md): Delete a specific MCP server. ### Missions - [List missions](https://developers.telnyx.com/api-reference/missions/list-missions.md): List all missions for the organization - [Create mission](https://developers.telnyx.com/api-reference/missions/create-mission.md): Create a new mission definition - [List recent events](https://developers.telnyx.com/api-reference/missions/list-recent-events.md): List recent events across all missions - [List recent runs](https://developers.telnyx.com/api-reference/missions/list-recent-runs.md): List recent runs across all missions - [Get mission](https://developers.telnyx.com/api-reference/missions/get-mission.md): Get a mission by ID (includes tools, knowledge_bases, mcp_servers) - [Update mission](https://developers.telnyx.com/api-reference/missions/update-mission.md): Update a mission definition - [Delete mission](https://developers.telnyx.com/api-reference/missions/delete-mission.md): Delete a mission - [Clone mission](https://developers.telnyx.com/api-reference/missions/clone-mission.md): Clone an existing mission - [List runs for mission](https://developers.telnyx.com/api-reference/missions/list-runs-for-mission.md): List all runs for a specific mission - [Start a run](https://developers.telnyx.com/api-reference/missions/start-a-run.md): Start a new run for a mission - [Get run details](https://developers.telnyx.com/api-reference/missions/get-run-details.md): Get details of a specific run - [Update run](https://developers.telnyx.com/api-reference/missions/update-run.md): Update run status and/or result - [Cancel run](https://developers.telnyx.com/api-reference/missions/cancel-run.md): Cancel a running or paused run - [List events](https://developers.telnyx.com/api-reference/missions/list-events.md): List events for a run (paginated) - [Log event](https://developers.telnyx.com/api-reference/missions/log-event.md): Log an event for a run - [Get event details](https://developers.telnyx.com/api-reference/missions/get-event-details.md): Get details of a specific event - [Pause run](https://developers.telnyx.com/api-reference/missions/pause-run.md): Pause a running run - [Get plan](https://developers.telnyx.com/api-reference/missions/get-plan.md): Get the plan (all steps) for a run - [Create initial plan](https://developers.telnyx.com/api-reference/missions/create-initial-plan.md): Create the initial plan for a run - [Add step(s) to plan](https://developers.telnyx.com/api-reference/missions/add-steps-to-plan.md): Add one or more steps to an existing plan - [Get step details](https://developers.telnyx.com/api-reference/missions/get-step-details.md): Get details of a specific plan step - [Update step status](https://developers.telnyx.com/api-reference/missions/update-step-status.md): Update the status of a plan step - [Resume run](https://developers.telnyx.com/api-reference/missions/resume-run.md): Resume a paused run - [List linked Telnyx agents](https://developers.telnyx.com/api-reference/missions/list-linked-telnyx-agents.md): List all Telnyx agents linked to a run - [Link Telnyx agent to run](https://developers.telnyx.com/api-reference/missions/link-telnyx-agent-to-run.md): Link a Telnyx AI agent (voice/messaging) to a run - [Unlink Telnyx agent](https://developers.telnyx.com/api-reference/missions/unlink-telnyx-agent.md): Unlink a Telnyx agent from a run - [List knowledge bases](https://developers.telnyx.com/api-reference/missions/list-knowledge-bases.md): List all knowledge bases for a mission - [Create knowledge base](https://developers.telnyx.com/api-reference/missions/create-knowledge-base.md): Create a new knowledge base for a mission - [Get knowledge base](https://developers.telnyx.com/api-reference/missions/get-knowledge-base.md): Get a specific knowledge base by ID - [Update knowledge base](https://developers.telnyx.com/api-reference/missions/update-knowledge-base.md): Update a knowledge base definition - [Delete knowledge base](https://developers.telnyx.com/api-reference/missions/delete-knowledge-base.md): Delete a knowledge base from a mission - [List MCP servers](https://developers.telnyx.com/api-reference/missions/list-mcp-servers.md): List all MCP servers for a mission - [Create MCP server](https://developers.telnyx.com/api-reference/missions/create-mcp-server.md): Create a new MCP server for a mission - [Get MCP server](https://developers.telnyx.com/api-reference/missions/get-mcp-server.md): Get a specific MCP server by ID - [Update MCP server](https://developers.telnyx.com/api-reference/missions/update-mcp-server.md): Update an MCP server definition - [Delete MCP server](https://developers.telnyx.com/api-reference/missions/delete-mcp-server.md): Delete an MCP server from a mission - [List tools](https://developers.telnyx.com/api-reference/missions/list-tools.md): List all tools for a mission - [Create tool](https://developers.telnyx.com/api-reference/missions/create-tool.md): Create a new tool for a mission - [Get tool](https://developers.telnyx.com/api-reference/missions/get-tool.md): Get a specific tool by ID - [Update tool](https://developers.telnyx.com/api-reference/missions/update-tool.md): Update a tool definition - [Delete tool](https://developers.telnyx.com/api-reference/missions/delete-tool.md): Delete a tool from a mission ### Conversations - [List conversations](https://developers.telnyx.com/api-reference/conversations/list-conversations.md): Retrieve a list of all AI conversations configured by the user. Supports PostgREST-style query parameters for filtering. Examples are included for the standard… - [Create a conversation](https://developers.telnyx.com/api-reference/conversations/create-a-conversation.md): Create a new AI Conversation. - [Get Insight Template Groups](https://developers.telnyx.com/api-reference/conversations/get-insight-template-groups.md): Get all insight groups - [Create Insight Template Group](https://developers.telnyx.com/api-reference/conversations/create-insight-template-group.md): Create a new insight group - [Get Insight Template Group](https://developers.telnyx.com/api-reference/conversations/get-insight-template-group.md): Get insight group by ID - [Update Insight Template Group](https://developers.telnyx.com/api-reference/conversations/update-insight-template-group.md): Update an insight template group - [Delete Insight Template Group](https://developers.telnyx.com/api-reference/conversations/delete-insight-template-group.md): Delete insight group by ID - [Assign Insight Template To Group](https://developers.telnyx.com/api-reference/conversations/assign-insight-template-to-group.md): Assign an insight to a group - [Unassign Insight Template From Group](https://developers.telnyx.com/api-reference/conversations/unassign-insight-template-from-group.md): Remove an insight from a group - [Get Insight Templates](https://developers.telnyx.com/api-reference/conversations/get-insight-templates.md): Get all insights - [Create Insight Template](https://developers.telnyx.com/api-reference/conversations/create-insight-template.md): Create a new insight - [Get Insight Template](https://developers.telnyx.com/api-reference/conversations/get-insight-template.md): Get insight by ID - [Update Insight Template](https://developers.telnyx.com/api-reference/conversations/update-insight-template.md): Update an insight template - [Delete Insight Template](https://developers.telnyx.com/api-reference/conversations/delete-insight-template.md): Delete insight by ID - [Get a conversation](https://developers.telnyx.com/api-reference/conversations/get-a-conversation.md): Retrieve a specific AI conversation by its ID. - [Update conversation metadata](https://developers.telnyx.com/api-reference/conversations/update-conversation-metadata.md): Update metadata for a specific conversation. - [Delete a conversation](https://developers.telnyx.com/api-reference/conversations/delete-a-conversation.md): Delete a specific conversation by its ID. - [Get insights for a conversation](https://developers.telnyx.com/api-reference/conversations/get-insights-for-a-conversation.md): Retrieve insights for a specific conversation - [Create Message](https://developers.telnyx.com/api-reference/conversations/create-message.md): Add a new message to the conversation. Used to insert a new messages to a conversation manually ( without using chat endpoint ) - [Get conversation messages](https://developers.telnyx.com/api-reference/conversations/get-conversation-messages.md): Retrieve messages for a specific conversation, including tool calls made by the assistant. ### Embeddings - [Get Tasks by Status](https://developers.telnyx.com/api-reference/embeddings/get-tasks-by-status.md): Retrieve tasks for the user that are either `queued`, `processing`, `failed`, `success` or `partial_success` based on the query string. Defaults to `queued` an… - [Embed documents](https://developers.telnyx.com/api-reference/embeddings/embed-documents.md): Perform embedding on a Telnyx Storage Bucket using the a embedding model. - [List embedded buckets](https://developers.telnyx.com/api-reference/embeddings/list-embedded-buckets.md): Get all embedding buckets for a user. - [Get file-level embedding statuses for a bucket](https://developers.telnyx.com/api-reference/embeddings/get-file-level-embedding-statuses-for-a-bucket.md): Get all embedded files for a given user bucket, including their processing status. - [Disable AI for an Embedded Bucket](https://developers.telnyx.com/api-reference/embeddings/disable-ai-for-an-embedded-bucket.md): Deletes an entire bucket's embeddings and disables the bucket for AI-use, returning it to normal storage pricing. - [Search for documents](https://developers.telnyx.com/api-reference/embeddings/search-for-documents.md): Perform a similarity search on a Telnyx Storage Bucket, returning the most similar `num_docs` document chunks to the query. - [Embed URL content](https://developers.telnyx.com/api-reference/embeddings/embed-url-content.md): Embed website content from a specified URL, including child pages up to 5 levels deep within the same domain. The process crawls and loads content from the mai… - [Get an embedding task's status](https://developers.telnyx.com/api-reference/embeddings/get-an-embedding-tasks-status.md): Check the status of a current embedding task. Will be one of the following: ### Clusters - [List all clusters](https://developers.telnyx.com/api-reference/clusters/list-all-clusters.md) - [Compute new clusters](https://developers.telnyx.com/api-reference/clusters/compute-new-clusters.md): Starts a background task to compute how the data in an embedded storage bucket is clustered. This helps identify common themes and patterns in the data. - [Fetch a cluster](https://developers.telnyx.com/api-reference/clusters/fetch-a-cluster.md) - [Delete a cluster](https://developers.telnyx.com/api-reference/clusters/delete-a-cluster.md) - [Fetch a cluster visualization](https://developers.telnyx.com/api-reference/clusters/fetch-a-cluster-visualization.md)