Infosec Pros Lose Faith in Automated Pentesting Tools Amid Rising AI Vulnerabilities
A significant majority of cybersecurity professionals report critical false negatives from automated pentesting tools, leading to a sharp decline in confidence and adoption of fully autonomous approaches.
A growing disillusionment with fully automated penetration testing tools is sweeping through the cybersecurity industry, according to a recent report by offensive security firm Cobalt. The firm's 2026 State of Pentesting report reveals that a staggering 78% of surveyed security practitioners have encountered critical false negatives from these automated tools, indicating a fundamental failure to detect significant security flaws.
This widespread disappointment is particularly acute concerning vulnerabilities introduced by artificial intelligence (AI) technologies, such as prompt injection. Automated scanners, while adept at identifying known, signature-based vulnerabilities, are proving woefully inadequate against the more complex, logic-based flaws that AI systems can introduce. Cobalt highlights that these advanced exploits require creative, multi-turn interactions and an understanding of adversarial psychology, elements entirely invisible to tools relying on simple, single-shot automated queries.
The consequence of this ineffectiveness is a dramatic drop in organizations considering purely automated security scanning. This year, only 9% of respondents expressed openness to such approaches, a steep decline from 29% in the previous year. This shift underscores a growing demand for human expertise and oversight in the crucial area of security assurance, moving away from the promise of complete automation.
While the survey sample size of 450 individuals is relatively small, the trend is clear and concerning for vendors of fully automated pentesting solutions. Cobalt views this decline in reliance on automation as a positive development, suggesting that practitioners are becoming more discerning, prioritizing genuine security assurance over mere coverage metrics often touted by vendors.
The challenge is compounded by the increasing prevalence of vulnerabilities introduced by AI and large language model (LLM) environments. Cobalt's data indicates that while traditional environments see about 12% of vulnerabilities classified as high or critical, AI and LLM environments exhibit a much higher rate of 32%, a figure that has remained consistent for two years. This suggests that AI is a significant source of new and severe security risks.
Cobalt advocates for a hybrid security model, where automated tools handle routine scanning of most systems, but critical infrastructure and complex AI-driven environments are managed and protected by human experts. This approach acknowledges the strengths of both automation and human intelligence in the face of evolving threats.
This sentiment is echoed by other industry players. Application security firm Veracode previously reported that AI-assisted software development is generating more vulnerabilities than security teams can effectively manage, leading to an increase in unresolved issues. Their findings indicate that 82% of companies leave known vulnerabilities unresolved for over a year, with the proportion of high-risk flaws on the rise.
However, not all industry leaders share this skepticism. Amazon's Chief Security Officer, CJ Moses, has noted that AI pentesting tools have improved his team's efficiency by 40%. Despite these gains, Moses emphasizes the continued necessity of human oversight, stating that while AI excels at data analysis and providing broad views, it lacks the critical decision-making capabilities required for complete security reliance.