Researchers Build AI Agent That Exploits Security Rules, Highlighting Need for New Defenses
An AI agent designed to bypass core security principles like unique identities and least privilege access has been built by researchers, revealing critical gaps in current AI security measures.
Researchers at Token have successfully developed an AI agent, named Rehearsal, capable of circumventing fundamental security controls, including unique identities, least privilege access, and separate traceability. This exercise, conducted internally by the company that develops the agent and the platform it operates within, underscores a growing need for specialized security measures tailored to the unique challenges posed by autonomous AI agents.
The Rehearsal agent was designed to perform tasks within the Token platform by interacting with web pages, filling forms, and filtering data. Operating as a typical UI agent within a user's authenticated browser session, it inherently inherited the user's session cookie. This presented a significant security challenge because, from the server's perspective, the agent's actions were indistinguishable from those of the human user, lacking any specific identifier for the agent itself.
By controlling both the agent and the application, Token was able to implement internal safeguards. These included visually indicating when the agent was in control, dimming and locking the application interface, and meticulously recording all agent actions. Crucially, the agent's reach was intentionally limited to read-only dashboards, preventing it from making unauthorized changes without explicit user approval. It was also programmed to avoid filling sensitive fields and to recognize accessibility labels for interface controls.
However, the researchers emphasize that these internal controls are not sufficient for agents operating in third-party environments. When an agent interacts with external applications like a CRM or cloud console, the application often lacks the context to differentiate agent activity from human activity. The agent itself may also lack the understanding to identify sensitive fields or critical functions within the application's interface.
The proposed solution involves creating distinct 'lanes' for AI agents, separate from human users. Emerging web standards are beginning to address this, with browsers capable of marking agent-driven sessions and restricting agent access to specific sites. Applications could also expose their functions as callable tools, clearly flagging them as read-only or consequential. Furthermore, agents could digitally sign their requests to verify their identity.
A critical gap remains in connecting these disparate security mechanisms. While browsers might know an agent is active, they don't always communicate this effectively to the application. Similarly, application tools might operate within a user's session, negating the benefit of narrowed scope, and agent authentication drafts often omit the crucial context of who the agent is acting on behalf of.
To bridge this gap, a unified layer is needed that passes essential context: the agent's identity, a verifiable link to the human user it represents, and a clearly defined task scope narrower than the user's permissions. Without this comprehensive context, applications cannot make informed authorization decisions, potentially leading to privilege misuse.
Until these standards are fully realized and adopted, security teams must take proactive steps. This includes discovering and inventorying all browser agents and extensions, tracing their usage, reviewing their access privileges, and identifying potential attack paths. Implementing step-up authentication for sensitive actions and questioning SaaS vendors about their audit capabilities for agent activity are also crucial measures.