AI Coding Agents Install Untrusted Code, Creating New Supply-Chain Risk
Researchers discovered AI coding agents are installing untrusted code on corporate networks by registering unclaimed domain names, leading to phone-home callbacks from major companies.
A concerning new attack vector has emerged where artificial intelligence coding agents, including prominent models like OpenAI's Codex and Anthropic's Claude, are inadvertently installing untrusted code onto corporate networks. Researchers from an Israeli stealth startup identified this vulnerability by scanning thousands of live domains belonging to defense contractors, Fortune 500 companies, and Big Tech firms. Their investigation uncovered numerous instances where documentation files, such as llms.txt and llms-full.txt, contained references to unclaimed domain names or unverified code packages.
To test the implications of these findings, the researchers registered a selection of these unclaimed domain names and hosted malicious packages designed to trigger a "phone-home" callback from any machine executing them. The results were alarming: within an hour, a callback was received from a Fortune 500 company. Over subsequent hours and days, dozens more callbacks were registered from a mix of large corporations and startups, demonstrating the widespread nature of the issue.
Crucially, the telemetry data collected by the researchers revealed the chain of parent processes that initiated these callbacks. This analysis pointed directly to AI coding agents, including Anthropic's Claude, OpenAI's Codex, and Nous Research's Hermes, as the entities responsible for executing the untrusted code. The vendors involved, including Anthropic and OpenAI, did not immediately respond to requests for comment regarding these findings.
The core of the problem lies in a fundamental breakdown of trust within the AI agent ecosystem. As described by one of the researchers, Alon Hertz, these AI agents are treating vendor documentation and external references as absolute truth without sufficient verification. This blind acceptance creates a new and potentially vast supply-chain attack surface, where malicious actors could exploit the AI's inherent trust in documentation to inject compromised code into development pipelines.
The implications are significant, drawing parallels to devastating supply-chain attacks like SolarWinds. As agentic AI becomes increasingly integrated into various layers of IT infrastructure—from SaaS applications to cloud environments and endpoints—the potential for such vulnerabilities to be exploited grows exponentially. The current security perimeters and monitoring tools are not adequately equipped to detect or prevent these novel threats.
This discovery highlights a critical need for enhanced security protocols and a re-evaluation of how AI agents interact with external resources and code repositories. Organizations must implement stricter validation processes for code generated or integrated by AI, and AI developers must build more robust verification mechanisms into their models to prevent them from becoming unwitting conduits for malware.
Moving forward, the cybersecurity community faces the challenge of securing an evolving digital landscape where AI agents are becoming increasingly autonomous and integrated. The findings underscore the urgency for developing new security paradigms that can account for the unique risks posed by AI-driven development and operations, ensuring that these powerful tools do not become vectors for catastrophic breaches.