How I built a conversational AI bot using Telnyx
Learn how Enzo, a Telnyx Software Engineer, built a conversational AI bot using Telnyx and OpenAI.
By Enzo Piacenza
Inspired by the Air.ai demo that recently went viral on X (formerly Twitter) of an AI bot making a sales call, I set out to build my own conversational AI bot using the Telnyx Voice API.
I’m Enzo Piacenza, a Telnyx software engineer. In this project, I combined Telnyx's telephony capabilities with OpenAI's powerful language models to make an AI expert for inbound callers interested in Telnyx’s offerings.
Here's a step-by-step guide on how I brought this vision to life.
Getting started and developer setup
Before diving into the steps I took to make a conversational AI bot, we’ll quickly cover how to get started with Telnyx. If you’re a first-time user, you can access our quickstart guide to set up your developer environment. The guide covers the following steps:
- Signing up for a Telnyx account
- Buying a number
- Creating a SIP connection
- Creating an outbound voice profile
- Obtaining your API keys
After that, be sure to follow our developer setup guide and install the Telnyx WebRTC Voice SDK in the language of your choice:
- Python
- PHP
- Node
- Java
- .NET
- Ruby
Telephony setup
Once your environment is set up, you’re ready to start building your conversational AI bot. Here are the simple steps I took to create a Telnyx sales chatbot.
Step 1: Purchase a number
The foundation of any voice application is telephony. So, I started by going to the Telnyx Mission Control Portal and purchasing a phone number.
Step 2: Create a Voice API application
Then, I created a Voice API application and configured it using a webhook URL that could be tied to my local application. I used ngrok to do this, but any cross-platform application that allows you to expose your local web servers to the internet will work.
Step 3: Assign the application to the number
Next, I assigned the Voice API application to the number I purchased, enabling the application to receive webhooks when the number was called.
Once I had the telephony component all set up, it was time to work on the call flow.
Voice API call flow
Step 4: Call initiation and answering
Upon receiving the "call.initiated" webhook, I issued a call answer command.
Step 5: Begin transcription
Then, I received the "call.answered" webhook and began the transcription with the following parameters:
- language: "en”
- transcription_engine: "B" (This is our new transcription engine),
- transcription_tracks: "inbound" (We’re only interested in getting the transcription of the caller.)
Step 6: Text-to-speech (TTS) greeting
I wanted to answer our caller with a friendly greeting using a speak command, so I used, “Welcome to Telnyx. How can I help you today?”
Step 7: Speech-to-text (STT) transcription
At this point, the caller would answer, and the transcription would begin. Since I was the one building and testing this application, the caller on the other line was myself.
I waited for the "call.transcription" webhooks on the backend that contained what our customer (me) was saying in plain text.
Large language model (LLM) integration
Step 8: Generating and sending responses
For each caller transcript, I needed to generate a response and send it back within the call. To do this, I used OpenAI’s Chat Completion API, a chat model that takes a list of messages as input and returns a model-generated message as output. It’s designed to make multi-turn conversations easy.
In this project, I wanted to send a request to OpenAI’s API that looked like this for each transcript:
1. Model: gpt-3.5-turbo-16k
2. Temperature: 0
3. Messages: [ {"role": "system", "content":
"Today you work at Telnyx, try to sell Telnyx
as best as you can, also try to get as much
information about the caller as you can
so we can call them back"},{"role": "user",
"content": <here I will put the first transcript>].
This list of messages will keep growing as the
conversation goes, and for each OpenAI response,
I will add to the list: {"role": "assistant",
"content": <LLM response>}, and then the
next human transcription and so on.
Step 9: Text-to-speech response
After receiving the LLM response, I used the “speak” command to turn the LLM text response into speech and recite it to the caller. From there, the cycle begins again, and the conversation carries on.
As you can see, I built a conversational AI sales bot just like Air.ai’s demo using Telnyx in just a few steps.
Why build with Telnyx?
With many options available to developers, I still love using Telnyx to build conversational AI for multiple reasons:
High-quality voice
Telnyx is built on a private global network, which means voice travels over private gateways rather than the public internet. These private connections ensure crystal-clear voice quality, which is crucial for generating accurate conversational AI responses.
Developer docs and support
Telnyx has a plethora of developer documentation, debugging tools, and reporting, all of which allow me to manage my projects end-to-end in its Mission Control Portal.
Telnyx is at the forefront of AI
We just released our new in-house speech-to-text (STT) engine that’s more accurate and cost-effective than Google’s (our previous default transcription engine). We also recently released a feature to enable HD voice. And later in the fall, we’ll release a feature for improved noise suppression. These three features are crucial for accurate and natural conversational AI interactions.
We’re also working on reducing latency between conversation turns for quick and human-like responses—because no one likes long, awkward pauses.
Partner with Telnyx to build your conversational AI tools
Building a conversational AI bot using Telnyx Voice API and OpenAI's language model was simple and rewarding. From setting up the telephony to implementing real-time transcriptions and automated responses, this project demonstrates the endless possibilities at the intersection of telecommunication and AI.
Contact our team of experts to learn more about how you can build conversational AI with Telnyx.
Sign up for emails of our latest articles and news
Related articles