On this page
Agents generated from natural language descriptions
HIPAA compliant with BAA
90% of inbound calls are repetitive targets for AI
Ultradoc is a modern clinical workflow platform for physicians: real-time patient rounding, AI-powered billing, voice dictation, fax analysis, and EHR integration, all unified in a single source of truth. When the team set out to add a Voice AI phone agent as a new module, they evaluated several of the most visible names in the space before landing on Telnyx, and built a HIPAA-compliant AI Agent that turns inbound patient calls into structured, patient-matched tasks inside Ultradoc.
For Ultradoc, adding a phone agent wasn't a standalone bet. The platform already captured rich, continuously-updated patient data, from progress notes to CPT-coded encounters, and the natural next step was to let patients interact with that data directly over the phone. A prescription refill, an , or a status update could be handled by an AI agent and automatically attached to the correct patient record, saving clinical staff from hours of repetitive call handling.
Ultradoc discovered Telnyx almost by accident, while investigating ElevenLabs' supported phone providers. Digging deeper, Sajkowski found that Telnyx wasn't just a phone number provider but a full Voice AI stack, with inference hosted on the same infrastructure as the telephony layer.
"I saw you had your own voice AI implementation, and that you host the models locally, everything on one infrastructure. I tried that and was like, wow, this thing responds so fast. That was one of the main things that sold me."
The difference was structural, not incremental. With every other option Sajkowski had tested, a single turn in a conversation had to hop between a telephony vendor, a speech-to-text vendor, an LLM vendor, and a text-to-speech vendor, each adding its own latency. Telnyx collapsed that stack onto one platform. The result was conversational speed that actually felt conversational.
Beyond latency, the commercial fit was compelling. Telnyx's phone numbers were , and critically for a healthcare technology company, Telnyx was willing to execute the Business Associate Agreement Ultradoc needed to operate under HIPAA. "Telnyx was able to work out a BAA with us, which was really nice," Sajkowski said. "That allowed us to be HIPAA compliant."
As a developer-led organization, Ultradoc cared as much about the build experience as the runtime performance. Here, too, Telnyx delivered.
"It was pretty easy. The documentation is great," Sajkowski said. "You have a Node.js package, which is really awesome. That made the implementation even easier. I didn't have to make any custom calls. The webhooks are very simple and comprehensive."
The Node.js SDK was clean enough that Sajkowski was able to hand large portions of the integration work off to an AI coding assistant. "Claude Code was able to build the Telnyx integration super easily because of your Node package. That simplified it."
The phone agent Ultradoc built on Telnyx isn't a generic voice bot bolted onto the product. It's a module that sits inside the Ultradoc platform and gets immediate access to the same patient records, tasks, and clinical summaries that clinicians work with in the app. A patient calls in, authenticates, and the AI can act on their record in real time, creating tasks, submitting refill requests, or updating notes, all of which surface instantly in the clinical team's dashboard.
Ultradoc's first production phone agent was a prescription refill assistant. "It asks for name, date of birth, things like that, and then where you get your prescription filled," Sajkowski explained. "It's so easy to throw in tool calls. Do you want to send it to the pharmacy you last used? You pull up these things so fast. That's all through the tool calls within the phone agent, and then it calls a webhook to our server, which returns the data in real time."
Healthcare is one of the most regulated environments in software, and Voice AI raises the stakes: a poorly-designed phone agent can leak protected health information with a single careless response. Ultradoc addressed this with a deliberately restrictive authentication and data-handling model built on top of Telnyx.
"We built an authentication system where if the Voice AI tries to make a patient tool call and they're not authenticated, it'll say, 'please authenticate the patient,'" Sajkowski said. "The patient provides their first name, last name, and date of birth. It looks it up, makes sure the patient actually exists in the database, and if it does, all the tool calls are unlocked."
Even after authentication, the agent is designed to be write-heavy rather than read-heavy, so it doesn't casually recite protected information back to the caller. "The AI is never reading patient data back out to the user. So if someone tried to impersonate that user, it's not going to say, 'okay, when you did this.' It's mostly going to write data. And if it is going to read something out, it's something that's not confidential at all."
The result is a phone agent that can take high-value actions on a patient's behalf, without becoming a vector for social-engineering attacks on their record.
The clinicians using Ultradoc's AI Agent noticed the same thing Sajkowski did during his evaluation.
"They were mostly impressed by the speed, which I was very happy about," Sajkowski said.
Beyond speed, the business case for deploying phone agents inside Ultradoc is straightforward. Tasks like prescription refills, , and status updates collapse from live human interactions into structured text that shows up in the app, already attached to the correct patient. "It just allows your existing data to flow so much easier, and it frees up tasks for the people in the office. They don't have to do repetitive tasks anymore. A prescription refill shows up as text in the app, completely handled. You don't even have to think about it."
One of the advantages of Telnyx hosting inference directly is that model upgrades flow through to Ultradoc's agents without a re-architecture. As open-source models improve, the phone agent improves with them.
"I'm definitely expecting to see much smarter AI models implemented," Sajkowski said. "As these open-source models get smarter, the Telnyx stack will get smarter because you can just upgrade it."
For healthcare specifically, Sajkowski sees this trajectory ending in a near-complete automation of repetitive call work. "All the repetitive tasks will just be completely automated, which is really awesome, because now you can have humans for tasks that actually require a bunch of thought power, things that are more meticulous than the automated 90% of calls for these places."
Ultradoc's build illustrates a pattern that any organization handling patient communications should pay attention to. The takeaway isn't "add a voice bot." It's more specific than that.
If your Voice AI agent is going to interact with patients in real time, sub-second response time isn't a nice-to-have. It's the difference between a patient who engages and one who hangs up and calls the front desk anyway. Ultradoc evaluated multiple vendors and found that stacked architectures with separate telephony, STT, LLM, and TTS providers consistently added too much latency for live conversation. A unified stack on a single network was the only architecture that met the bar.
In healthcare, a voice agent that can read data out is a liability. Ultradoc's write-heavy, auth-gated approach is a model worth replicating: authenticate before acting, write more than you read, and never volunteer protected information unprompted. HIPAA compliance isn't just about signing a BAA. It's about how your agent is architecturally prevented from leaking data under adversarial conditions.
But that vision had one non-negotiable prerequisite: the agent had to feel fast enough that patients didn't notice it was AI. That was harder to find than it sounds.
After evaluating several providers, Ultradoc found that the existing options couldn't meet their latency requirements. "I started with Retell AI, which was a wrapper on top of ElevenLabs and Twilio. Then I tried using ElevenLabs and Twilio directly," said Vincent Sajkowski, CEO and co-founder of Ultradoc.
The problem showed up immediately. "The phone agents had ridiculously long response times. I would say something and get a response a second later. That's terrible. It should be instant."
For a platform whose whole identity is built around real-time sync across every device a clinician touches, a one-second lag on a phone call was a dealbreaker.
What makes the architecture distinctly Ultradoc is that customers don't pick from a catalog of pre-built agents. They describe what they want in natural language, and Ultradoc's platform generates the phone agent, wires up the right tool calls, writes the prompt, and assigns a Telnyx phone number, automatically.
"You tell an AI that builds the AI," Sajkowski said. "You say, 'I want this phone agent to do this,' and it automatically adds tool calls and builds its prompt. Call the phone agent, test it out, and it just works."
Because Ultradoc maintains the catalog of tool calls the AI can draw from, any capability the platform supports, from appointment booking to refill routing to task creation, can be composed into a custom agent on demand.
That frees clinical staff to spend their time on the cases that actually require human judgment, instead of the 90% of inbound calls that are well-defined, repetitive, and highly automatable.
The business case for Voice AI in healthcare isn't replacing clinicians. It's absorbing the repetitive, well-defined interactions that consume the majority of front-desk time, prescription refills, appointment confirmations, status updates, so that human staff can focus on cases that require judgment, empathy, and nuanced decision-making. That math holds whether you're a multi-location health system or a single-physician practice. And by building on a platform where model upgrades, new voice options, and infrastructure improvements flow through without re-architecture, you ensure the stack compounds over time rather than locking you into a stitched-together multi-vendor setup that can't keep up.
