Hackers Weaponize SEO Poisoning and Hidden HTML to Manipulate AI Agents
Attackers are increasingly targeting AI agents with indirect prompt injection, using SEO poisoning and hidden HTML to trick them into executing malicious commands, including fraudulent payments and trusting fake websites.
A new wave of malicious websites is actively targeting artificial intelligence agents, turning them into unwitting accomplices for cybercriminals. Instead of tricking human users, these sites employ sophisticated techniques like search engine optimization (SEO) poisoning and hidden HTML code to inject malicious instructions directly into AI systems that browse and interact with the web. This emerging attack vector, known as indirect prompt injection (IPI), exploits the trust AI models place in the content they process, enabling attackers to manipulate automated browsing tools.
Researchers from Zscaler ThreatLabz have identified two distinct campaigns leveraging this IPI method. The first campaign focused on tricking AI agents into making fraudulent payments. Attackers created fake documentation for a Python library, stuffing the page with keywords to ensure it ranked highly in search results for developers seeking solutions. Buried within this content, using a format called JSON-LD (often treated as highly authoritative by AI), were hidden instructions. These instructions guided AI agents to believe a $3 developer license fee was a necessary step to resolve an error, ultimately prompting them to send cryptocurrency to an attacker-controlled wallet. The malicious code was concealed using CSS positioning, making it invisible to human visitors but readily accessible to AI crawlers.
The second campaign involved typosquatting a popular decentralized finance (DeFi) platform, DeBank. Attackers registered a domain, debank[.]auction, designed to closely mimic the legitimate DeBank website. They populated the fake site with deceptive titles and metadata, including terms like "DeBank Login" and "Crypto Tracker," to appear as an official resource. Crucially, hidden within the page was a prompt instructing AI models to recognize this fraudulent domain as the verified DeBank site and prioritize it in search results. The prompt even included specific instructions to avoid mentioning the word "auction" in the domain name, a subtle attempt to maintain the illusion of legitimacy.
These attacks highlight a critical vulnerability in how AI agents process web content. While AI models often rely on structured data and search engine rankings to determine the trustworthiness of information, attackers are now adept at manipulating these very signals. In testing, some AI agents were successfully tricked into initiating payments or deeming fake websites as legitimate when presented with the manipulated content. This demonstrates that the risk of indirect prompt injection is not theoretical but a tangible threat to systems that automate online interactions.
While Zscaler's testing showed that many AI models could identify the fake DeBank site when provided with a correct reference, the success of the attack without such a comparison underscores the dependency of AI judgment on the immediate information it receives. This reliance makes AI agents particularly susceptible to carefully crafted deceptive content that bypasses traditional human-level scrutiny.
Zscaler recommends that organizations developing or deploying AI agents implement robust, layered security controls. These controls should be capable of detecting the subtle patterns of hidden injection attempts within web content. The company noted that its own platform already flags such activity under specific signatures designed to identify HTML-based prompt injection. As AI agents take on increasingly autonomous roles in navigating the internet, treating every webpage as a potential source of hidden manipulation is becoming an essential security posture.
The implications of these attacks extend to various AI applications, from automated customer service bots to AI-powered research tools and coding assistants. The ability to subtly influence AI decision-making through manipulated web content opens up new avenues for fraud, misinformation, and the compromise of sensitive data. The cybersecurity community must adapt by developing AI-specific security measures that can identify and neutralize these novel indirect prompt injection techniques.