GitHub Secret Scanning Cuts False Positives by 75% Using LLM-Based Contextual Verification
GitHub announced a 75.76% reduction in false positive secret scanning alerts by integrating LLM-based contextual reasoning that analyzes how detected values are used in code.
GitHub has announced a major improvement to its secret scanning system that reduces false positive alerts by 75.76%, exceeding its target of 65%. The enhancement, developed in collaboration with Microsoft Security & AI's Agents Offense team, applies contextual reasoning from the Agentic Secret Finder system to analyze how detected values are used in code—such as being passed to API requests, authentication headers, or cloud SDK calls—rather than simply matching patterns.
Secret scanning is a critical tool for protecting developers and organizations by catching exposed credentials early. At GitHub's scale, processing billions of pushes across millions of repositories, even small inefficiencies create significant friction. Too many false positives make alerts harder to trust, causing developers to spend more time triaging and less time fixing real issues. This collaboration focused on bringing the precision of AI-detected secrets closer to the same high standard as provider-pattern detections.
The new verification step uses LLM-based contextual reasoning to evaluate detection candidates. Instead of passing entire files or repositories—which introduces noise and increases cost and latency—the system extracts a small set of high-signal information that explains how the value is used. For example, it looks for cases where a value is assigned to a variable and later passed into an API request, authentication header, database client, or cloud SDK call. Pattern matching can tell that a value looks like a secret, but it cannot tell whether the value is actually being used as one.
GitHub already has industry-leading precision for provider-pattern secret detection at massive scale. As it expanded into AI-powered generic secret detection, the next challenge was bringing the precision of AI-detected secrets closer to the same high standard. The new approach builds directly on the existing detection pipeline, generating candidates and then evaluating them with more context-awareness. This results in higher precision without changing upstream detection logic or reducing coverage.
The evaluation was conducted on hundreds of customer-confirmed false positive alerts. The result was a 75.76% reduction, significantly exceeding the 65% target while maintaining strong detection performance. In practice, this means significantly less noise and a higher proportion of alerts that require action. With fewer irrelevant alerts, developers can trust what they see, spend less time triaging noise, and prioritize and fix real issues faster.
GitHub is continuing to evaluate this approach on larger datasets and live traffic, while improving how context is extracted and used for verification. Reducing false positives has been a consistent need at scale, and this work focuses on improving signal quality where it matters most. The goal is simple: fewer distractions, clearer signals, and faster action on real risks.
This announcement underscores the growing role of AI and LLMs in improving developer security tooling. By applying focused contextual reasoning rather than brute-force analysis, GitHub has demonstrated a practical path to making secret scanning alerts more trustworthy at massive scale. The approach could serve as a model for other platforms seeking to balance detection coverage with alert quality.