Edinburgh Napier University

  Malicious agents increasingly use encrypted tunnels to communicate with external servers. Communications may contain ransomware keys, stolen banking details, or other confidential information. Rapid discovery of communicated contents through decrypting tunnelled traffic can support effective means of dealing with these malicious activities.

Decrypting communications requires knowledge of cryptographic algorithms and artefacts, such as encryption keys and initialisation vectors. Such artefacts may exist in volatile memory when software applications encrypt. Virtualisation technologies can enable the acquisition of virtual machine memory to support the discovery of these cryptographic artefacts.

A framework is constructed to investigate the decryption of potentially malicious communications using novel approaches to identify candidate initialisation vectors, and use these to discover candidate keys. The framework focuses on communications that use the Secure Shell and Transport Layer Security protocols in virtualised environments for different operating systems, protocols, encryption algorithms, and software implementations. The framework minimises virtual machine impact, and functions at an elevated level to make detection by virtual machine software difficult.

The framework analyses Windows and Linux memory and validates decrypts for both protocols when the Advanced Encryption Standard symmetric block or ChaCha20 symmetric stream algorithms are used for encryption. It also investigates communications originating from malware clients, such as bot and ransomware, that use Windows cryptographic libraries.

The framework correctly decrypted tunnelled traffic with near certainty in almost all experiments. The analysis durations ranged from sub-second to less than a minute, demonstrating that decryption of malicious activity before network session completion is possible. This can enable in-line detection of unknown malicious agents, timely discovery of ransomware keys, and knowledge of exfiltrated confidential information.