SANS Analysis Reveals 99.7% of Web Traffic is Bot-Driven
A recent study by the SANS Internet Storm Center reveals that 99.7% of traffic on a monitored website originated from automated bots, underscoring the critical role of CAPTCHA technology in modern web defense.
A recent analysis conducted by the SANS Internet Storm Center has highlighted the overwhelming volume of automated bot traffic targeting web infrastructure, revealing that 99.7% of requests to a monitored site were generated by non-human actors SANS Internet Storm Center. The study, which utilized Cloudflare’s Turnstile CAPTCHA service over several months, found that out of approximately 300 requests, only a single one was verified as coming from a legitimate user SANS Internet Storm Center.
The technical implementation of the CAPTCHA served as a filter to distinguish between human visitors and automated scrapers or malicious bots. During the testing period, the researcher noted that while the system was highly effective at blocking bots, it also identified minor usability issues, such as false positives where users attempted to submit login forms before the CAPTCHA challenge had fully initialized SANS Internet Storm Center. This specific issue was mitigated by implementing a JavaScript-based check that disables the "Submit" button until the verification process is complete SANS Internet Storm Center.
The data collected during this period identified several persistent sources of automated traffic. Notable offenders included IP addresses originating from AWS infrastructure, cloud providers in Hong Kong, and specific ranges associated with Facebook Ireland SANS Internet Storm Center. Additionally, the researcher identified a static IP address in Japan that appeared to be operating as a persistent web spider SANS Internet Storm Center.
The choice of Cloudflare’s Turnstile was driven by a desire to balance security with user privacy and site performance. Unlike traditional CAPTCHA solutions, such as Google reCAPTCHA, Turnstile was selected for its perceived lower impact on user experience and its alignment with existing CDN infrastructure SANS Internet Storm Center. The researcher emphasized that while no CAPTCHA is entirely immune to bypass attempts, the goal is to increase the cost and effort required for an attacker to the point where the targeted data is no longer worth the investment SANS Internet Storm Center.
This analysis underscores the pervasive nature of automated scanning and scraping activities that modern web administrators face daily. By deploying automated verification tools, site owners can significantly reduce the load on their servers and protect sensitive data from unauthorized collection. As bot sophistication continues to evolve, the reliance on robust, privacy-conscious verification mechanisms remains a critical component of a comprehensive web security strategy SANS Internet Storm Center.