How to detect an AI-generated voice on live calls

In April 2026, someone called a Charter Communications employee, pretended to be a member of an IT help desk, and got their Microsoft Entra ID credentials. From that call, the extortion group ShinyHunters breached Charter's Salesforce and claimed access to 40 million customer records.

Threat intelligence from EclecticIQ documents that ShinyHunters affiliates used AI-powered voice agents to run these vishing calls at scale infiltrating other large multinational companies including ADT, 7-Eleven, Instructure, Vimeo, Medtronic, and Carnival. The voice agents adjusted their scripts in real time based on how the victim responded but otherwise used the same playbook everytime:vishing call to compromise SSO, pivot to Salesforce, exfiltrate, and extort.

These attacks demonstrate the growing need to detect AI-generated voices on live call, in real time. Doing so requires a multi-faceted approach to detection which we will dive into.

What you need to know

• "AI-generated voice" has three distinct pathways: fixed-voice TTS, voice-cloned TTS, and real-time voice conversion. Each enters a call differently but one model that learns generator-agnostic structure can cover all three rather than needing a separate detector for each.
• Detection runs as a trust stack: a detection model reads the audio, explainability adds context, identity matching confirms a known voice, and watermarking corroborates provenance.
• Three of the five tested PSTN codecs hit 100% accuracy on both real and synthetic audio. The two compressed codecs, G.728 and iLBC, stay above 91%.
• Two deployment patterns work in production: in-call streaming detection (four-second windows, verdict in under 300 ms, act mid-call) and post-call audit (full recording, aggregated windows, fewer false positives).
• Detection requires the same recording-consent posture as recording the call, and it complements rather than replaces STIR/SHAKEN, which verifies caller identity but not whether the voice is real.

What "AI-generated voice" actually means on a call

Not all synthetic voices work the same way so, detection splits into three categories, and each one enters a phone call differently.

Fixed-voice TTS. A text prompt goes in and generated speech comes out using a stock or default voice, no human speaker in the loop. This is the category most people think of first, and it is the one where TTS models have improved fastest. ElevenLabs, OpenAI, Azure, AWS Polly, and dozens of open-source engines can produce speech that passes for human on a phone call, especially when the audio is compressed to 8 kHz PSTN quality where the telltale high-frequency artifacts disappear. Because the voice belongs to no real person, its spectral signature can be fingerprinted, which makes it the most straightforward of the three to catch.

Voice-cloned TTS. The input is still text, but the output impersonates a target voice from just a short reference sample without fine-tuning or model training. An attacker supplies a few seconds of someone's voice and the system produces new speech that sounds like that person. It runs on the same underlying technology as fixed-voice TTS, but detecting it is harder because the output carries the spectral and prosodic fingerprint of a real person. This is what the Charter and ADT attacks used in practice.

Real-time voice conversion. Unlike the first two categories, the input here is not text. It is a live human voice. A model takes that voice and transforms it to sound like someone else as the person speaks. Because a real person is generating the audio, everything downstream of the spectral transformation is genuinely human including the prosody, the pacing, the breathing, and the way they respond in conversation. Only the spectral envelope is synthetic which is why detection models have the least public benchmark coverage here for exactly that reason.

These three categories sit on a spectrum at the signal level. A detector that only memorizes the fingerprints of known generators tends to be bound to the first. Tune a classifier for fixed-voice TTS and it can miss a clone that carries real-human spectral characteristics. Either of those can miss voice conversion where the prosody is genuinely human.

Getting past that limit means not relying on memorized generator fingerprints alone. Resemble's DETECT-World learns the structural patterns generative architectures leave in the frequency and temporal structure of audio rather than just the phonetic content. That is what makes it language-agnostic and lets one model generalize across all three categories, including generators it has never seen, instead of needing a separate detector per category.

How detection works

Detection tells you whether the voice on the call was generated or altered by AI, and it answers with a confidence score. It does not read intent or confirm consent. "Is this voice AI?" is the first question, and the answer tells you which question to ask next: block the call, route it to a human, or log it for review.

Therefore, a reliable answer on a live call takes more than one technique. Resemble runs detection as part of a larger trust stack, where each layer contributes a different kind of evidence.

Detection. Resemble Detect reads the waveform directly and returns an independent score for whether the audio was AI-generated or altered, without relying on caller claims or metadata.

Explainability. Resemble Intelligence turns a detection score into a forensic report. It identifies the artifacts behind the verdict, fraud type, liveness status, and speaker context such as language and dialect. That context gives agents and fraud teams a basis for action and creates an audit trail for review.

Identity. Where a voice has been enrolled with Resemble Identity, an inbound voice can be matched against a known likeness. This speeds up the verdict and confirms whether the real person consented to that voice being used.

Watermarking. Resemble's PerTh multimodal watermarker embeds an imperceptible neural watermark at generation, optimized to survive PSTN compression to 8 kHz, G.723.1, and multiple carrier hops. A present watermark is the strongest provenance signal available. The reverse does not hold: the absence of a mark never proves a voice is human, because most generators do not watermark. Detection carries the load, and watermarking corroborates it.

Pattern scoring. Resemble Signal flags call content that matches a known fraud pattern and explains the match in plain language. It runs without transcription, so no spoken content or PII materializes as text at any stage. Teams can customize the pattern library with the scripts they actually see.

The detection layer is powered by DETECT-World, Resemble's third-generation model benchmarked against 250+ generators and spanning 54 languages. It learns the statistical fingerprints generative architectures leave in the frequency and temporal structure of audio, not the phonetic content alone, which is what makes it language-agnostic and lets it reach zero-day coverage more quickly. Podonos, a third party benchmark which scores every system itself on a fixed clip set with privately held labels, ranked DETECT-World first of the 18 systems it has tested, at 99.47% accuracy and the only system with both false-positive and false-negative rates under 1%. What a phone call does to those numbers is the subject of the next section, and it is why detection on a live line is tuned differently.

The transport reality of a real phone call

A detection model that hits near-perfect accuracy on clean lab audio often degrades on real phone audio. The reasons are physical and architectural, not model-quality problems. Any honest article about detecting AI voice on live calls has to start with what a phone call actually does to the audio.

The codec ladder. Most PSTN calls use G.711 (mu-law in North America, a-law in Europe), which encodes audio at 8 kHz with 8-bit samples. The entire spectral content above 3.4 kHz is gone. G.722, marketed as HD Voice, extends to 16 kHz but is rare in carrier-to-carrier connections. WebRTC and mobile calls use Opus, which adapts its bitrate to available bandwidth, sometimes dropping as low as 6 kbps. Cellular networks use AMR and AMR-WB, which apply their own lossy compression. Every codec is a lossy transform. A detection signal that lives in the high frequencies does not survive a standard PSTN call. For more on how codecs affect voice quality, the principles are the same for detection.

The transcoding chain. A single call between a caller and a callee can be encoded, decoded, and re-encoded two to four times. Each leg between SIP proxies, media gateways, and SBCs adds quantization noise and removes micro-detail. The audio the detector sees at the end of the chain is not the audio the caller's microphone produced.

Mobile network compression. Cellular carriers run rate adaptation, discontinuous transmission (DTX), and comfort-noise generation. DTX silences "silent" frames entirely and replaces them with synthetic background noise. Network processing changes the micro-silences and breathing patterns a forensic detector tries to measure.

Jitter buffer and packet loss. Real-time transport buffers 20 to 60 ms of audio, reorders packets, and inserts fake audio to mask lost packets. Frame-precise timing analysis on a real call is harder than on a clean file.

Echo cancellation and AGC. Automatic gain control and echo cancellation modify spectral content and amplitude before the audio reaches any analysis endpoint. Detection features that depend on raw amplitude or spectral envelope get shifted.

Resemble has tested their detection model across the PSTN codec family. Their internal evaluation, run on the Resemble AI Codec Testbench v3, shows the following accuracy:

G.711 a-law, G.722, and EVS all hold 100% accuracy on both real and synthetic audio. G.728 is the tightest case, at 91.9% on real audio and 97.9% on synthetic. iLBC sits at 95.8% real and 98.7% synthetic. The two codecs where accuracy dips are also the two with the most aggressive compression, which strips out the micro-detail detection relies on. For those cases, real-time deployments apply a higher detection threshold tuned for precision, and the post-call pattern aggregates multiple windows to reduce single-decision risk.

Where detection sits in a real call pipeline

Detection on a live call has to fit inside the call's media path. There are two deployment patterns, and they serve different operational needs.

Pattern A: In-call streaming detection

The call arrives at a Telnyx number. Telnyx Call Control forks the media stream via WebSocket to the Resemble Detect endpoint using the streaming_start command. Telnyx Media Streaming duplicates the call audio and delivers it to a customer-provided WebSocket URL without touching the live call's audio quality. The stream carries base64-encoded RTP payloads in the codec of the call: PCMU, PCMA, G.722, Opus, AMR-WB, or L16.

Resemble Detect analyzes the audio stream and returns a detection score. Resemble Intelligence turns that score into an intelligence report identifying the artifacts behind the verdict, fraud type, liveness status, and speaker context. Four seconds of audio is sufficient for a verdict, with a result returned in under 300 milliseconds.

Telnyx Call Control acts on the signal. Any Call Control command can be issued mid-call using the call_control_id: hang up the call, transfer to a human agent, play a prompt, or log the result. The decision is yours. The detection endpoint returns a score. Your application decides what to do with it.

For real-time deployments, Resemble Detect applies a higher detection threshold than their offline benchmark configuration. This tunes for precision: when something is flagged as synthetic, confidence is high, and false alarms on real customer calls are reduced. The tradeoff is that some synthetic calls near the threshold boundary will be missed. Resemble is continuing to refine this threshold per customer environment, accounting for factors like call quality and background noise.

Pattern B: Post-call audit

Telnyx records the call via SIPREC or Call Control recording. SIPREC sessions land at siprec.telnyx.com, Telnyx's SIPREC Recording Server, which fires call.initiated webhooks with the recording metadata. Recordings land in object storage. Resemble Detect runs as a batch job against the recording file. Output goes to a compliance pipeline, audit log, or fraud review queue.

This pattern trades latency for accuracy. Post-call, you have the full recording, not a four-second window. You can run detection on multiple segments and aggregate the results, which reduces the false-positive risk that comes with single-window real-time decisions.

Resemble Detect supports both streaming and batch API modes. The streaming endpoint is designed for real-time telephony and meeting-bot deployments where each four-second window requires an instantaneous response. The batch endpoint is designed for post-call and post-upload analysis where multiple segments can be processed and aggregated. For live call deployments, the recommendation is to set a high confidence threshold for any "act" decision (hang up, transfer) and a lower threshold for "log" decisions (flag for review, add to audit trail). This split prevents expensive false positives on real customer calls while still capturing borderline cases for human review.

Both patterns have different latency, cost, and regulatory profiles. The right choice depends on whether the use case requires acting on the call while it is still live (fraud prevention) or whether post-call evidence is sufficient (compliance audit, fraud investigation).

Where detection works best

Not every call needs real-time AI voice detection. The use cases where it earns its cost fall into a few categories.

• High-value transaction calls: Wire transfers, payment authorizations, account changes, credential resets. If the action the caller is asking for moves money or changes access, the cost of a wrong call is high enough to justify detection. A bank that drops a fraudulent wire-transfer call before it reaches an agent saves more than the cost of the detection API. A bank that drops a real customer's wire-transfer call loses the customer.
• Inbound calls to regulated industries: Banking, healthcare, insurance, utilities. These industries already record calls for compliance. Adding detection to the recording pipeline is a marginal cost. The detection signal lands in the same audit trail as the recording, the transcript, and the call metadata. For these deployments, the post-call pattern is often sufficient.
• Outbound AI agent calls at scale: If you are running outbound voice AI campaigns, you are already on the other side of this problem. Detection can verify that your AI voice is not being cloned and replayed back at you. Watermarking can prove that the audio that left your system is the audio that arrived at the callee's phone. This is the Resemble Watermarker use case: encode the watermark at generation, verify it on the receiving end.
• Contact center fraud monitoring: Contact centers that handle customer authentication are the highest-value target for voice cloning. An attacker who clones a customer's voice and calls the contact center to reset a password or authorize a transfer is using the phone channel to bypass the entire authentication stack. Detection on the inbound leg, before the agent picks up, is the defense. The four-second detection window means the system can flag a synthetic caller before the agent has finished the greeting.

Compliance and operational hooks

Detection output joins the regulatory and operational systems that already govern the call.

STIR/SHAKEN. STIR/SHAKEN establishes who initiated the call at the network level. A signed call with full attestation can still carry a synthetic voice. Detection assesses whether the audio is AI-generated or altered. Teams need both signals when a caller's identity and the voice on the call matter.

As of August 2, 2026, the EU AI Act requires machine-readable marking of AI-generated content, including audio. Audio watermarking is the compliance mechanism. Resemble's PerTh multimodal watermarker embeds a neural watermark optimized for the 8 kHz bandwidth constraint of PSTN calls, surviving G.723.1 compression and multiple carrier hops. For deployments serving EU customers, watermarking at generation and detection at the receiving end is the compliance path. One of our previous webinars on Voice AI in Europe with Zohaib Ahmed (Resemble AI CEO) and David Casem (Telnyx CEO), covers this in depth.

Detecting on a call requires the same recording-consent posture as recording the call. One-party and two-party consent state rules apply. If you are already recording, detection adds no new consent burden. If you are adding detection to a call you were not previously recording, consult your legal team.

Detection output is most useful when it lands in an evidence-grade log: timestamps, call ID, confidence scores, and action taken. Fraud teams and regulators can assess those records alongside call recordings, speech analytics (CallMiner, Observe.AI, Verint), and fraud scoring tools. Detection adds audio-authenticity evidence to the case record.

Where we are both investing next

On the Resemble side, we are extending the world-model reasoning behind DETECT-World across every modality a live call can carry. The aim is a detector that reasons about whether a voice is physically plausible, so coverage holds as synthesis tools evolve faster than any fingerprint library can track. We are also hardening detection on degraded phone audio, where the codec ladder does the most damage today.

On the Telnyx side, this is part of a broader bet on agent primitives. We have been rolling out the building blocks an AI agent needs to act in the real world: Voice API for making and receiving calls, an Email API for sending and receiving mail, a Browser Agent for navigating the web, a Search API for retrieving information. Call Control media streaming is the primitive that lets a detection layer like Resemble sit inside a live call without owning the call. The next step is a native detection hook in Call Control: a pre-configured webhook that receives detection signals from a partner like Resemble and triggers call actions without the customer building an intermediary application server. The goal is reducing the integration surface from "build a WebSocket server, handle frames, call back to Call Control" to "enable detection on this number, set your threshold, pick your action."

The larger direction is making Telnyx the single stop for everything an agent needs to communicate and do things across the internet. Voice, messaging, email, browsing, search, and now detection. Together they are the infrastructure layer for agents that need to reach people and systems outside their own application.

Where to learn more

• Resemble Detect
• Resemble Multimodal Watermarker
• Resemble Intelligence
• Resemble Signal
• PerTh open source audio watermarker
• Telnyx Call Control
• Telnyx Media Streaming
• Telnyx SIPREC
• May 2026 webinar: "Voice AI in Europe: Deepfakes, Data Sovereignty, and the August 2026 Compliance Cliff" with Zohaib Ahmed (Resemble AI) and David Casem (Telnyx)
• Learn more about AI voice fraud and the telecom trust layer
• Read about deepfake detection risks and defenses