Researcher Tricks Claude AI into Revealing Internal Binary and Google Cloud Token
A security researcher successfully extracted an internal Anthropic binary and a memory dump containing Google Cloud token fragments by prompting Claude AI, highlighting potential risks of AI models exposing sensitive system information.
A security researcher has demonstrated how a large language model like Anthropic's Claude AI can be manipulated through conversational prompts to reveal sensitive internal system information, including an executable binary and fragments of a Google Cloud token. The findings, detailed by Bishop Fox consultant Chris Davis, showcase the potential for AI models to inadvertently expose their operational environment and credentials.
The investigation began with simple questions about Claude's sandbox environment. Claude, running on an Enterprise account, disclosed that it operated within a gVisor container on Ubuntu 24.04.4 LTS and listed its mounted folders and available tools. Further probing revealed the existence of a program at /process_api, identified as a Rust-built control daemon responsible for managing processes, file mounts, and host communication.
By exploiting Claude's file download feature, which accepted absolute paths, the researcher was able to retrieve the /process_api binary. This 3,179,528-byte file was confirmed through reverse engineering to be Anthropic's internal sandbox orchestration binary, validating Claude's earlier descriptions. The researcher took steps to ensure these were not fabricated responses by repeating tests in unrelated chat sessions.
In a subsequent step, the researcher prompted Claude to dump the memory of process ID 1, framing the request as a comparison of human and computer memory. This yielded a memory dump containing fragmented JSON Web Tokens. This technique echoes previous discoveries where Claude's sandbox bypassed security measures to expose credentials, but this instance involved the sandbox itself leaking secrets.
Reconstructing a usable Google Cloud service account token from the memory dump proved possible. The dump contained incomplete signature bytes for EdDSA tokens, including those generated by Anthropic's systems and Google Cloud service account ID tokens. A Python script was developed to test candidate values against Google's public OAuth keys, successfully verifying a signature and reconstructing a token that Google's token-info API accepted as valid. Approximately one in three memory requests produced usable token material.
The extracted binary also revealed network restrictions, specifically a --block-local-connections flag that prevented connections originating from within the container. However, by providing a subtle hint about IP_FREEBIND, the researcher was able to guide Claude into bypassing these restrictions. Claude established a virtual network address, allowing the service to perceive the connection as non-local, and a /mount_root endpoint responded with a 101 Switching Protocols reply, confirming the bypass.
While Anthropic stated that these findings did not breach their threat model and marked some findings as informational or duplicate through their bug bounty program, the incident underscores the evolving security challenges posed by AI systems. The researcher suggested several mitigations, including restricting model access to infrastructure memory, promptly wiping credentials, limiting download API access, and removing unnecessary network permissions.
This research highlights a critical lesson for defenders: AI assistants, when prompted carefully, can reveal their internal workings and sensitive data. It emphasizes the need for organizations to meticulously manage secrets and sensitive information that might be accessible to any process an AI tool can interact with, especially as AI models become more integrated into enterprise workflows.