Google's Zero-Trust AI Agent Architecture Mitigates Risks in Customer Support
Google's open-source Customer Support & Returns Agent demonstrates a zero-trust approach for AI agents, using external safeguards to prevent manipulation and limit actions.
Google has unveiled an open-source Customer Support & Returns Agent, built with its Agent Development Kit (ADK) and Gemini, which applies zero-trust security principles to AI agents designed to interact with sensitive systems and perform real-world tasks. This innovative project operates under the assumption that an AI agent could be compromised or manipulated, implementing robust external security controls to strictly limit its capabilities and verify its actions.
The core of Google's approach treats the AI model as a component that could be tricked or jailbroken, acknowledging that simple instructions within a system prompt are insufficient against sophisticated attacks like prompt injection. To counter these threats, the architecture incorporates three distinct security layers situated outside the AI model itself. These layers include cryptographic signatures for all database changes, the use of isolated environments for executing AI-generated code, and a Semantic Gateway that rigorously checks both inputs and actions against deterministic rules before they are processed.
A compelling demonstration of these security measures involves a simulated attack scenario. In a normal transaction, the agent would process a customer's return request, calculate deductions, and record a refund. However, the test highlights a malicious scenario where a customer instructs the agent to issue a $10,000 refund for a $149 order and to execute Python code designed to expose environment variables. Without the implemented safeguards, such an agent could easily authorize fraudulent refunds, leak sensitive API keys, or even compromise the host server.
The first layer of defense focuses on database integrity. Each state-changing database write is cryptographically signed by the agent responsible for it. The database then verifies this signature before committing the transaction, providing clear attribution and making any subsequent tampering detectable. For production environments on Google Cloud, this involves assigning each agent a unique service account with signing permissions on an asymmetric key stored in Cloud Key Management Service (KMS), ideally backed by a Hardware Security Module (HSM) to ensure the private key never leaves a secure environment.
Executing AI-generated code presents another significant security risk. Autonomous agents might produce Python scripts for various tasks, but prompt injection could redirect this capability towards malicious ends, such as extracting credentials or establishing connections to attacker-controlled servers. Google's reference architecture addresses this by executing generated code within a gVisor user-space sandbox. This sandbox isolates the code from the host system, disables network egress, and imposes strict limits on resource usage and execution time, with a five-second timeout to prevent runaway processes.
The third critical security layer is the Semantic Gateway, which acts as a gatekeeper between the AI agent and the systems it can interact with. This gateway applies deterministic checks to all incoming prompts and outgoing tool calls. It can identify sensitive information like credit card numbers, detect known jailbreak patterns, and enforce transaction limits, effectively blocking dangerous actions before they occur. These policies are treated as software contracts, with automated tests ensuring their continued efficacy through prompt changes or model migrations.
For enhanced security in production, services can be further protected within a VPC Service Controls perimeter, creating an additional boundary to prevent data exfiltration even if an agent workload is compromised. Google emphasizes that building autonomous agents does not necessitate accepting unconstrained risk. By implementing security boundaries through hardware-backed identity, sandboxing, and deterministic validation, developers can allow AI models to perform dynamic reasoning while the underlying infrastructure enforces strict operational limits.
The open-source reference implementation is readily available for local testing, complete with attack scenarios designed to exercise the security controls. Developers can also explore a browser-based Live Attack Playground and utilize Google's ADK documentation to begin building their own secure AI agent tooling and sessions, fostering a more secure future for AI-driven automation.