Anthropic Details Claude Duet's Security Safeguards Against AI Misuse

Anthropic has provided a comprehensive overview of the security and privacy features built into Claude Duet, its advanced AI agent designed to proactively monitor connected accounts and execute tasks on behalf of users. The safeguards are crucial for building trust and ensuring the responsible deployment of AI agents capable of interacting with sensitive systems.

Central to Duet's security architecture are user approval requirements for critical actions, automated checks for sensitive information, and robust infrastructure designed to isolate user credentials from the AI model itself. Duet operates via a cloud-based system and browser, enabling it to perform tasks such as preparing briefings and drafting messages for connected accounts. Currently, it is available through a research preview waitlist for Pro and Max subscribers in the US, with usage limits shared across the Claude ecosystem.

Anthropic emphasizes that Duet will prompt users for explicit approval before undertaking actions like sending potentially sensitive messages, initiating payments, or deleting information. This approval process includes a detailed summary of the proposed action, its destination, and the content involved. Furthermore, a separate automated check, operating independently of the AI model performing the task, verifies that the action aligns with the user's intent, thereby mitigating risks from malicious instructions embedded within web pages.

While Duet can automate many tasks, certain high-risk activities such as accessing another person's account, trading securities, or transferring funds between bank accounts remain strictly prohibited, even with user approval. Specific actions, including payments and account creation, require re-approval each time they are initiated. The company acknowledges the potential for "approval fatigue" and plans to continuously calibrate these requirements to balance user convenience with security.

To prevent data leakage, Duet employs a dedicated classifier to detect sensitive information in outgoing messages and browser uploads. The AI is explicitly instructed never to share credentials, full payment card or bank account numbers, or government identification numbers, even when directly prompted. Disclosure of other private information, such as health or financial details, requires explicit user consent, which can be granted through a direct instruction to send the data. Anthropic notes that these checks are still in a research preview phase and may occasionally flag legitimate messages or miss sensitive content.

Anthropic has implemented stringent measures to protect user credentials and browser security. Duet itself cannot access user passwords, browser cookies, or sign-in tokens. Instead, users enter passwords through an encrypted form within the Claude app, which securely transmits them to the cloud browser for website login. Future integration with password managers like 1Password aims to further streamline this process without exposing credentials to the AI model. For connected applications, a separate service manages sign-in tokens, keeping them outside the agent's direct workspace.

The company has subjected Duet to rigorous security evaluations, including internal red team exercises and external penetration testing. These evaluations simulate various attack scenarios, such as information leakage, unauthorized actions, and prompt injection attempts. Anthropic also encourages external researchers to report vulnerabilities through its public bug bounty program hosted on HackerOne, further bolstering the security posture of the AI agent.

These detailed safeguards highlight Anthropic's commitment to addressing the inherent security challenges posed by increasingly capable AI agents. By implementing multi-layered controls, user oversight, and robust credential protection, the company aims to mitigate risks associated with AI-driven automation and ensure that tools like Claude Duet can be used safely and effectively.