Researchers Achieve Code Execution Within Windows SmartScreen Enclave
Security researchers have successfully executed code within a protected Windows SmartScreen enclave, a feat achieved by exploiting a four-byte write vulnerability and a timing side-channel.
Security researchers have demonstrated the ability to run their own code within a highly protected memory region of Windows known as a VBS (Virtualization-Based Security) enclave. This specific enclave is part of the Windows SmartScreen feature, designed to safeguard sensitive data related to anti-phishing protections. The proof-of-concept attack, conducted in a laboratory setting, successfully bypassed the enclave's isolation mechanisms by chaining together a four-byte write vulnerability and a timing side-channel exploit. While this research has not been observed in real-world attacks, it highlights a significant potential avenue for compromising the integrity of Windows' security features.
The vulnerability, tracked as CVE-2026-83498, was rated as Critical by Microsoft with a CVSS score of 7.8. The company stated that the flaw was not publicly known before its fix and had not been exploited in the wild. Microsoft's internal security teams also identified the issue. The research, published by the security firm Doyensec, came out over a month after Microsoft released a patch for the vulnerability in its September 8, 2026 update.
VBS enclaves are designed to create an isolated slice of memory that even the operating system's kernel cannot directly access. This isolation is maintained by the Windows hypervisor, which places the enclave's memory in a more trusted execution environment (VTL1 or Isolated User Mode), separate from the normal operating environment (VTL0). The intention is to protect sensitive data and code within the enclave, even if an attacker has already gained control of the host system's kernel or is operating with SYSTEM privileges.
The specific enclave targeted by Doyensec's research is SFAPE.dll, which is integral to Microsoft Defender SmartScreen's Enhanced Phishing Protection. This feature monitors user input, such as typed or pasted text, and alerts users if they enter their Windows sign-in password into an unsafe location, like a plain text editor or a phishing website. To perform this check without exposing the password in plain text, Windows intercepts the password during login and stores a hashed version within the enclave for later comparison.
On a Windows 11 system, the process responsible for loading this enclave is lsass.exe, the crucial process that handles sign-in credentials and operates with SYSTEM privileges. The attack vector requires an attacker to already have code running on the machine that can interact with the enclave, specifically by calling the LSASS process. Microsoft's own vulnerability assessment confirms this, classifying the attack vector as local, meaning an attacker needs low-level access and cannot exploit it remotely or without prior system compromise.
Doyensec's exploit begins with a subtle flaw in the enclave's initialization process. Calling the enclave's Initialize function a second time triggers an error that causes it to write a fixed four-byte value to an address controlled by the calling program. Crucially, this second call bypasses an address check that the first call performs. While this four-byte write itself is limited, when aimed at a specific memory location, it can corrupt a critical value the enclave uses to distinguish its own memory from the host's memory, thereby disabling its internal safety checks.
Following the corruption of the memory boundary checks, a timing bug in another enclave function is exploited. This timing issue leads to a controlled memory wipe that redirects how the enclave looks up stored objects. The resulting read and write operations allow researchers to overwrite a table responsible for determining which code the enclave is permitted to execute. This manipulation ultimately enables the execution of attacker-chosen code directly within the enclave's protected environment, effectively breaking its isolation.
Microsoft addressed this vulnerability by releasing a cumulative update (KB5124008) on September 8, 2026, for Windows 11 versions 24H2 and 25H2. The patch includes modifications to the enclave's code, specifically adding the address check before the initial write and hardening the function exploited by the timing bug. Additionally, the update introduces a new policy that prevents the older, vulnerable version of the enclave from loading altogether, ensuring that patched systems cannot run the compromised code. There are no separate workarounds; users must install the update to be protected.