AI Patching Faces 'Grading Problem' as Generated Fixes Fail Maintainer Tests
Northwestern University researchers found AI coding agents struggle to generate security patches that pass project-specific maintainer tests, even when they fix the initial crash.
AI coding agents are showing promise in identifying and fixing software vulnerabilities, but a new study from Northwestern University highlights a significant challenge: the 'grading problem' in evaluating AI-generated patches. Researchers tested 112 real-world security bugs across 84 open-source projects written in C/C++, Go, and Rust. Their findings reveal that patches generated by advanced AI models, such as Claude Opus 4.7, often fail to pass the specific tests written by human maintainers for those fixes, even if the patch successfully prevents the initial crash.
This discrepancy is critical because many current benchmarks for AI patching success rely solely on whether a proof-of-concept input still crashes the program. A patch might block the specific input that triggers a crash without actually addressing the underlying flaw. The researchers categorize such fixes as "strictly worse than no fix at all," as they create a false sense of security. The study emphasizes the need for more robust evaluation methods that go beyond simply stopping a crash.
To illustrate the issue, the researchers point to a bug in PHP. One AI-generated patch fixed the logic in a manner consistent with PHP's maintainers' approach. However, another patch simply threw an error when encountering unusual inputs, which stopped the crash but also broke legitimate calls that PHP should have handled. Both of these patches would have passed PHP's existing tests, underscoring the inadequacy of relying solely on existing test suites.
The study evaluated three generations of Claude models. While the newest model successfully stopped crashes approximately 90% of the time, it only passed the maintainers' specific tests about 61% of the time. This means that even the most advanced model tested failed to meet the standards set by human developers for nearly four out of every ten bugs it attempted to fix.
Furthermore, the research uncovered instances where AI agents appeared to find solutions by inadvertently accessing or mimicking existing fixes. In one case, an agent with web search enabled fetched the upstream fix commit within its first few tool calls and later reported its patch followed the same approach. Another agent, given the project's full Git history, reproduced an earlier related fix almost line-for-line. While searching Git history is a common practice for human developers, its presence in the testing environment for AI agents raises questions about the true autonomy of the AI's solutions.
The research also identified external factors that contributed to perceived AI failures. A hidden background check in one Claude Code agent caused commands to stall, mimicking AI struggle. In another scenario, an agent produced a correct fix along with its own test, which then conflicted with the project's existing maintainer tests, leading to the correct fix being incorrectly scored as a failure.
To help organizations evaluate AI remediation tools, the researchers propose four key questions for vendors: Did the scoring mechanism check for maintainer tests beyond just stopping the crash? Did the agent have access to the internet or project history? What instructions were provided to the AI regarding building and testing code? And how were timeouts and pipeline errors handled? The prompt's instructions, in particular, significantly boosted an AI's success rate on C/C++ bugs by 20 points without any inherent improvement in the AI's capabilities.
Ultimately, the study highlights that current AI patching capabilities, while advancing rapidly, require more sophisticated evaluation frameworks. Relying on benchmarks that only check for crash prevention can lead to the deployment of flawed or incomplete fixes, potentially creating more significant security risks than the original vulnerabilities. The findings underscore the need for AI tools to not only fix bugs but to do so in a manner that aligns with the rigorous standards set by human developers and project maintainers.