Speaker-Activated Backdoor Hijacks Speech AI Agents
A Backdoor That Listens for One Voice
Researchers have taught speech recognition systems to recognize a specific person by voice. At that moment, the system secretly changes the person’s words. In the most dangerous attack scenario, the model quietly added a request to transfer $5,000 to the transcript. A connected AI agent then made the payment in a test banking environment.
The victim does not need to speak a trigger phrase. The attacker does not need to mix noise or ultrasound into the recording. The trigger is the person’s own voice. The team calls the technique a Speaker-Activated Backdoor, since it fires when the chosen speaker appears.
How the Attack Is Prepared
First, the attacker fine-tunes the model to tell voices apart. Next, the attacker makes it react to the target in a preset way.
In the main setup, the authors needed only two minutes of real audio from the target. They produced another 18 minutes with voice cloning. They then tested the attack on other real recordings that the model had never heard.
The Results Varied
Success depended heavily on the model and the voice.
- Whisper-medium: it performed the planted action in 90.2 percent of cases on average. It wrongly reacted to other people in 1.7 percent of recordings.
- NVIDIA Parakeet: it reached 97 percent, with no false triggers in the matching test.
- Other voices: with other chosen voices, Whisper-medium dropped to roughly 50 to 68 percent. The attack is therefore not guaranteed.
The Payment Test
The most telling test went beyond simple text swaps. The infected Whisper-medium added an instruction to send $5,000 whenever the chosen person spoke. The command appeared in 93.2 percent of test recordings.
The text then went to GPT-4o and to Claude models that can use external tools. Depending on the agent model, the transfer happened in 47.9 to 82.3 percent of cases. With Qwen2.5-Omni, the rate for GPT-4o reached 94.8 percent. All payments stayed inside an isolated test environment with fake money.
Why AI Agents Raise the Stakes
The danger grows when an AI agent can call bank services, APIs, and other external tools on its own. The agent sees a finished text command. It may take that command for the user’s words, although the person never said it.
Checking the transcript alone does not help here. The voice feature that fires the backdoor never appears in the text.
Limits of the Method
The method has serious limits. The authors tested only five voices and one voice cloning tool. The attacker also needs access to the model’s fine-tuning process.
Ordinary extra training of Whisper-medium on clean recordings almost wiped out the backdoor. The trigger rate fell to 2.3 percent at most, and to zero in some tests. Qwen2.5-Omni kept the harmful behavior far better, so the researchers found no universal defense yet.
Advice for Builders
For systems that can move money or take other sensitive actions, the authors advise against trusting a single voice command. Confirm risky operations through an independent channel. Vet speech recognition models before deployment, and where possible, train them further on your own clean data.
Support Our Threat Intelligence
If you find our technology report and cybersecurity news helpful, consider supporting our work.
Buy Me a Coffee
PayPal
