Perplexity AI Launches Numbat to Prevent Rogue AI Agent Actions
Perplexity AI has released Numbat, an open-source tool designed to act as a security intermediary for AI agents, preventing them from violating enterprise policies.
The increasing autonomy and capabilities of artificial intelligence agents are creating new security challenges for enterprises, as these agents can potentially deviate from established policies and execute unauthorized actions. Recognizing this growing risk, startup Perplexity AI has developed and released Numbat, an open-source tool aimed at providing a robust layer of security for AI agents.
Numbat functions as an intermediary, sitting between an AI agent's decision-making process and its execution of actions. This strategic placement allows the tool to monitor and evaluate proposed agent behaviors before they are carried out. By doing so, Numbat can detect and block any actions that would violate an organization's predefined security policies, ensuring that AI agents operate within defined safety parameters and preventing potential security breaches or policy violations.
Perplexity CISO Kyle Polley highlighted that the need for such a tool became apparent as enterprises, including Perplexity itself, grappled with the inherent security risks associated with autonomous AI agents. "Folks, especially due to recent events, are now waking up to the idea that agents can go rogue and they might not follow policies even if you add it to the system prompt," Polley stated. The development of Numbat began months before high-profile incidents, driven by the foresight that agents would gain more autonomy and access, thereby increasing the risk of them acting outside of established controls.
One of Numbat's key advantages is its integration capabilities with widely used agent harnesses. This broad compatibility means that security teams do not need to develop custom monitoring and safeguarding tools for each individual AI agent they deploy. Numbat offers live monitoring and policy enforcement, and in the event of an issue, provides tools for forensic reconstruction, allowing security teams to analyze past agent activities.
The tool's design ensures it does not learn from its experiences, maintaining "hard guardrails at its core." All of Numbat's decision-making logic is derived from the policies explicitly configured by the enterprise, ensuring consistent and accurate judgment of agent actions against security protocols. This deterministic approach is crucial for maintaining a predictable security posture.
Numbat employs three primary methods for enforcing security policies. Firstly, it plugs into the hook subsystem common in most coding agents, acting as a pre-action hook that pauses and evaluates an agent's planned action against security policies before execution. Secondly, it accesses and analyzes session artifacts from agent harnesses, normalizing them into NDJSON format for detailed forensic reconstruction and local processing by security teams. This provides a machine-readable log with a consistent schema, unlike often unstructured plaintext transcripts.
Finally, Numbat can run a local OpenTelemetry receiver, allowing coding agents to send telemetry data for downstream analysis while keeping sensitive information on-premises unless explicitly moved. Perplexity uses Numbat internally to secure code generated and reviewed by its engineers across various AI platforms. The company has developed 52 rules, categorized into 11 behavior types and multi-step sequences, to detect a range of malicious activities including secret access, exfiltration, privilege escalation, and lateral movement.
The inclusion of sequence correlation skills in Numbat is particularly noteworthy. This feature allows the tool to identify malicious intent not just from individual commands, which might appear innocuous, but from the overall sequence of actions within a session. This advanced capability helps in detecting sophisticated attempts to bypass security rules, such as those related to secrets management, by recognizing patterns that indicate a broader, rule-breaking objective.