AI Customer Service Bots Vulnerable to Prompt Injection and Email Spoofing Attacks
New research reveals AI-powered customer service bots can be manipulated through prompt injection and email spoofing to leak sensitive data or execute unauthorized actions.
AI-powered customer service bots are increasingly integrated into business operations, handling sensitive customer data, account modifications, and refund processes. However, recent research indicates that attackers may not need traditional exploits to compromise these systems. Instead, they can leverage vulnerabilities in how AI agents process and interpret data, leading to significant security risks.
One primary attack vector involves prompt injection, where malicious text is embedded within chat sessions. When a user requests a transcript of the conversation, the injected text can be used to craft phishing emails that appear to originate from a trusted company support address. This tactic exploits user trust, making recipients more likely to fall for the fraudulent communication.
Email spoofing vulnerabilities further exacerbate these risks. Some AI agents authenticate users by examining the visible 'From' header of incoming emails. Attackers can exploit discrepancies between this header and the actual sender validated by email delivery systems. By sending an email from an attacker-controlled address that passes authentication, they can trick the AI into associating the message with a legitimate victim's account, potentially leading to unauthorized access or data retrieval.
In a specific attack scenario, an AI agent could be tricked into retrieving sensitive customer information, such as billing or profile details, in response to a seemingly legitimate request. If the attacker has manipulated the 'CC' or 'reply-to' fields, the AI might inadvertently send this confidential data directly to the attacker.
Multi-factor authentication (MFA) is also not immune. While some bots require one-time passcodes for sensitive actions, weak email normalization can allow attackers to bypass rate limits. By subtly altering email address formats (e.g., using aliases or different formatting) while still pointing to the same mailbox, an attacker might trick one system into recognizing their address while another backend service retrieves the victim's account details, especially if raw user input is directly inserted into API requests.
Furthermore, AI agents connected to support inboxes can be exploited to leak third-party account codes, such as one-time passcodes for password resets. An attacker can instruct the bot to forward or leak such codes to attacker-controlled infrastructure after triggering a legitimate password reset email from another service to the company's support inbox.
Even human oversight doesn't guarantee security. Attackers can craft multipart messages, use hidden HTML, CSS styling, or specially formatted attachments to present a benign message to a human operator while delivering malicious instructions specifically to the AI system. Knowledge-base poisoning is another emerging threat, where attackers inject false information into company documentation or untrusted pages, causing the AI to treat this malicious content as legitimate.
To mitigate these risks, organizations must implement robust security measures. This includes strictly separating untrusted customer content from system instructions, employing verified session-bound identity controls, consistently normalizing email addresses, validating all tool requests server-side, and preventing bots from sending secrets to unverified recipients. AI agents should also operate with limited permissions, treating them as privileged automation systems rather than mere conversational interfaces.