How I built a conversational AI bot using Telnyx

Inspired by the Air.ai demo that recently went viral on X (formerly Twitter) of an AI bot making a sales call, I set out to build my own conversational AI bot using the Telnyx Voice API.

I’m Enzo Piacenza, a Telnyx software engineer. In this project, I combined Telnyx's telephony capabilities with OpenAI's powerful language models to make an AI expert for inbound callers interested in Telnyx’s offerings.

Here's a step-by-step guide on how I brought this vision to life.

Getting started and developer setup

Before diving into the steps I took to make a conversational AI bot, we’ll quickly cover how to get started with Telnyx. If you’re a first-time user, you can access our quickstart guide to set up your developer environment. The guide covers the following steps:

1. Signing up for a Telnyx account
2. Buying a number
3. Creating a SIP connection
4. Creating an outbound voice profile
5. Obtaining your API keys

After that, be sure to follow our developer setup guide and install the Telnyx WebRTC Voice SDK in the language of your choice:

• Python
• PHP
• Node
• Java
• .NET
• Ruby

Telephony setup

Once your environment is set up, you’re ready to start building your conversational AI bot. Here are the simple steps I took to create a Telnyx sales chatbot.

Step 1: Purchase a number

The foundation of any voice application is telephony. So, I started by going to the Telnyx Mission Control Portal and purchasing a phone number.

Step 2: Create a Voice API application

Then, I created a Voice API application and configured it using a webhook URL that could be tied to my local application. I used ngrok to do this, but any cross-platform application that allows you to expose your local web servers to the internet will work.

Step 3: Assign the application to the number

Next, I assigned the Voice API application to the number I purchased, enabling the application to receive webhooks when the number was called.

Once I had the telephony component all set up, it was time to work on the call flow.

Voice API call flow

Step 4: Call initiation and answering

Upon receiving the "call.initiated" webhook, I issued a call answer command.

Step 5: Begin transcription

Then, I received the "call.answered" webhook and began the transcription with the following parameters:

• language: "en”
• transcription_engine: "B" (This is our new transcription engine),
• transcription_tracks: "inbound" (We’re only interested in getting the transcription of the caller.)

Step 6: Text-to-speech (TTS) greeting

I wanted to answer our caller with a friendly greeting using a speak command, so I used, “Welcome to Telnyx. How can I help you today?”

Step 7: Speech-to-text (STT) transcription

At this point, the caller would answer, and the transcription would begin. Since I was the one building and testing this application, the caller on the other line was myself.

I waited for the "call.transcription" webhooks on the backend that contained what our customer (me) was saying in plain text.

Large language model (LLM) integration

Step 8: Generating and sending responses

For each caller transcript, I needed to generate a response and send it back within the call. To do this, I used OpenAI’s Chat Completion API, a chat model that takes a list of messages as input and returns a model-generated message as output. It’s designed to make multi-turn conversations easy.

In this project, I wanted to send a request to OpenAI’s API that looked like this for each transcript:

1. Model: gpt-3.5-turbo-16k
2. Temperature: 0
3. Messages: [ {"role": "system", "content": 
"Today you work at Telnyx, try to sell Telnyx 
as best as you can, also try to get as much 
information about the caller as you can 
so we can call them back"},{"role": "user", 
"content": <here I will put the first transcript>]. 
This list of messages will keep growing as the 
conversation goes, and for each OpenAI response, 
I will add to the list: {"role": "assistant", 
"content": <LLM response>}, and then the 
next human transcription and so on.

Step 9: Text-to-speech response

After receiving the LLM response, I used the “speak” command to turn the LLM text response into speech and recite it to the caller. From there, the cycle begins again, and the conversation carries on.

As you can see, I built a conversational AI sales bot just like Air.ai’s demo using Telnyx in just a few steps.

Why build with Telnyx?

With many options available to developers, I still love using Telnyx to build conversational AI for multiple reasons:

High-quality voice

Telnyx is built on a private global network, which means voice travels over private gateways rather than the public internet. These private connections ensure crystal-clear voice quality, which is crucial for generating accurate conversational AI responses.

Developer docs and support

Telnyx has a plethora of developer documentation, debugging tools, and reporting, all of which allow me to manage my projects end-to-end in its Mission Control Portal.

Telnyx is at the forefront of AI

We just released our new in-house speech-to-text (STT) engine that’s more accurate and cost-effective than Google’s (our previous default transcription engine). We also recently released a feature to enable HD voice. And later in the fall, we’ll release a feature for improved noise suppression. These three features are crucial for accurate and natural conversational AI interactions.

We’re also working on reducing latency between conversation turns for quick and human-like responses—because no one likes long, awkward pauses.

Partner with Telnyx to build your conversational AI tools

Building a conversational AI bot using Telnyx Voice API and OpenAI's language model was simple and rewarding. From setting up the telephony to implementing real-time transcriptions and automated responses, this project demonstrates the endless possibilities at the intersection of telecommunication and AI.

Contact our team of experts to learn more about how you can build conversational AI with Telnyx.