Network Traffic Analysis Detects Ransomware Before File Encryption
Researchers developed a novel method to detect ransomware on shared file servers by analyzing network traffic patterns, offering earlier detection than endpoint-only solutions.
Corporate networks often centralize sensitive files on shared servers, making them attractive targets for ransomware. When a ransomware attack originates from a compromised endpoint, it can spread to these central storage systems. Traditional endpoint detection tools, however, face limitations as they primarily monitor the machine they are installed on. The actual encryption process on a remote file server is often masked by ordinary file-sharing protocols, leaving a blind spot between the compromised client and the server.
A team at La Trobe University in Melbourne has introduced a new detection framework designed to operate directly on network traffic. This method analyzes Server Message Block (SMB) traffic, the protocol used by Windows for file sharing, to identify ransomware by the unique patterns of operations it generates. By examining the 'rhythm' of the protocol itself, the system can flag malicious activity without needing agents on endpoints or servers.
The core innovation lies in how the framework segments network traffic. Instead of relying on fixed time windows, it leverages a characteristic of SMB: control packets, such as those used for directory enumeration, maintain a consistent size regardless of network conditions. The researchers define a 'Region of Interest' as the traffic between two such consistent-sized packets. This approach allows for a more accurate reconstruction of client-server interactions.
Furthermore, the framework utilizes the consistent packet sizes associated with specific SMB actions. For instance, responses for creating new files, reading files, renaming, or closing files all have predictable byte counts. By analyzing these sizes, the system can infer the operations being performed on the files, effectively reconstructing the activity without inspecting packet contents or installing any software on the affected machines.
The detection process involves three stages. The initial stages compare traffic against known indicators of compromise, such as the characteristic size of ransom notes dropped by specific ransomware families. Traffic that bypasses these signature-based checks is then fed into a machine learning model, specifically a Random Committee classifier, trained to identify more sophisticated or unknown threats. This model achieved approximately 99.6% accuracy on test data, successfully identifying all ransomware samples in the study with a low rate of false positives.
Crucially, the framework aims for early detection. The researchers found that even a small window of activity at the beginning of an attack yielded high accuracy (99.44%), nearing the performance of analyzing complete traffic captures. Some ransomware families, like LockBit, were identifiable after only a third of their encryption process had completed.
A notable aspect of the research is the inclusion of benign traffic that mimics ransomware behavior. The team deliberately used network encryption tools, bulk file copy utilities, and compression software with encryption enabled in their benign dataset. This rigorous approach ensures the detection method is robust against legitimate, high-volume file operations that can resemble ransomware activity.
Despite its promising results, the framework has limitations. The initial data was gathered from a single testbed, and its performance on diverse production networks requires further validation. The current indicators are primarily for plaintext SMBv2 traffic, and the method's effectiveness with encrypted SMBv3 traffic, which masks packet sizes, needs to be addressed. Additionally, the framework is designed for automated attacks targeting mapped drives and does not currently handle manual, hands-on-keyboard intrusion techniques common in larger breaches.