AI Agents Bypass Safety Checks, Leak Secrets Through Default Configurations

A novel attack vector has emerged where AI agents, despite robust safety protocols, can be manipulated to exfiltrate sensitive data. Elad Meged, a founding engineer at Novee Security, successfully demonstrated this by processing a pull request against three vendors' repositories using their default configurations. The AI agent extracted shell commands, posted the output, and bypassed established safety checks, revealing a critical exposure for any organization deploying these agents without custom security hardening.

AI agents consist of a model that generates intent and a harness that translates this intent into actions like shell commands, file reads, or API calls. The harness also enforces security boundaries, including approval logic, tool permissions, and path restrictions. When these agents operate without human oversight at each step, the harness becomes the primary security perimeter. Meged leveraged techniques similar to those used in automated penetration testing to target these harnesses, finding that the vulnerabilities were not in the AI model's prompt interpretation but in the harness's trust decisions and how these decisions compounded across workflow stages.

Meged's research highlighted a cyclical patching process with Anthropic's Claude Code Action. Each fix closed a specific vulnerability, but by revealing where the boundary of 'safe' had moved, it guided subsequent attacks. While Anthropic awarded bounties for each finding, the repeated nature of the bypasses suggested a deeper architectural issue. Meged noted that the bounty-per-bypass model, while rewarding, can create a misleading signal that a problem is fully resolved when the underlying exposure persists, contrasting it with Google's approach of framing fixes as updates to the 'Trust Model.'

A similar kill chain was demonstrated against Google's Gemini CLI, using the recommended security configuration for CI workflows processing untrusted input. A configured restriction was found to be unenforced at the point of execution, leading to a critical finding (GHSA-wpqr-6v78-jr5g) with a CVSS score of 10.0. OpenAI's Codex CLI also presented vulnerabilities, where state written in one stage of a multi-stage workflow was treated as trusted context in the next, despite the presence of a sandbox with protected paths.

The core issue across these vendors lies in how harnesses make initial safety decisions—such as deeming a command read-only or a domain pre-approved—and how downstream components inherit this trust without re-validation. A read-only command that feeds into a public output channel effectively loses its read-only nature in practice. Similarly, a pre-approved domain serving attacker-controlled content is no longer safe. Meged argues for a shift from trusting labels to re-validating trust at the point of consumption.

While individual vendors can implement significant improvements, the recurring pattern across multiple major providers suggests a shared architectural assumption in how AI agent harnesses are built industry-wide. Meged emphasized the need for a cross-vendor conversation to establish a shared understanding of these failure modes, potentially leading to formal standards or best practices.

For organizations currently running these agents, Meged advises tracing every path where an agent's output or influenced state is consumed by a later stage with different privileges. The critical question is not just what the agent *can* do, but what happens to its actions and outputs *after* they are deemed 'safe' by the harness, particularly in the handoff between approval and execution, or between reading data and publishing it.