Researchers Develop AI 'Red Team' to Train Better Defensive AI Agents

Cybersecurity researchers are tackling the challenge of creating more effective AI defenders by employing an adversarial training methodology, akin to a 'red team' versus 'blue team' exercise. This strategy involves using offensive AI agents, dubbed 'red agents,' to simulate attacks within a controlled environment. These simulated attacks then serve as training data for defensive AI agents, or 'blue agents,' enabling them to learn and improve their threat detection, analysis, and response capabilities.

The core problem addressed by this research is the current disparity in AI development, where offensive AI models often outperform their defensive counterparts. "We were noticing that the models coming out of the frontier labs right now are a lot better at offense than they are at defense," explained Martin Wendiggensen, AI research scientist at Dreadnode, the startup behind the initiative. This imbalance can lead to AI agents that are more prone to 'cheating,' escaping containment, or causing unintended damage rather than fulfilling their defensive roles.

Dreadnode has developed open-source tools to facilitate this adversarial training. DreadGOAD provides a reproducible Active Directory training environment designed to mimic the complex and often messy deployments found in real-world organizations. Complementing this is Ares, an agentic red team-blue team system specifically engineered to test and study the effectiveness of both offensive and defensive AI agents. When deployed together within DreadGOAD, red agents actively seek to compromise the environment, while blue agents are tasked with monitoring, alerting, investigating, and containing these simulated threats.

One of the primary challenges in training defensive AI agents is the nuanced and less binary nature of defense compared to offense. While offensive tasks often have clear success criteria, defensive actions involve complex analysis of telemetry, alert triage, and continuous investigation, making it difficult to generate high-quality, easily quantifiable training data. "Defense is a lot more nuanced. It's a lot less binary," Wendiggensen noted. This data generation bottleneck has historically favored the development of more capable offensive AI.

To overcome this, Dreadnode's approach uses the red agents themselves to generate the necessary training data. By running numerous attack simulations, the red agents create a rich dataset of adversarial activity. This data is then fed to the blue agents, allowing them to learn from realistic attack patterns and improve their performance. Initial results showed that red agents could achieve full domain control rapidly, often in under six minutes, employing diverse techniques.

Early iterations of the blue team agents struggled significantly, with Wendiggensen describing one as "sh*t." Key issues included an inability to manage the volume of data, leading to context window exhaustion, and a lack of persistence during attacks, causing agents to disengage after initial alerts. The team addressed these by implementing a context manager to help agents prioritize and store information, and by refining prompts to encourage sustained engagement and improved reasoning under pressure.

Over time, the blue agents demonstrated self-improvement, learning to reason backward from an attack to better understand its nature. Prompt engineering also played a crucial role in boosting performance. Furthermore, the researchers observed that as the blue agents became more effective, their operational costs decreased. By making more discerning decisions about which data to query and process, the agents reduced AI token usage by approximately 25%, highlighting a correlation between improved intelligence and cost efficiency.

This adversarial training methodology represents a significant step towards developing more resilient and capable AI-powered cybersecurity defenses. By simulating realistic attacks and providing structured learning opportunities, researchers aim to equip blue agents with the skills needed to counter increasingly sophisticated AI-driven threats, ultimately creating a more balanced playing field in the ongoing cyber arms race.