Hidden OS objects correlated detection technology based on VMM

To evade the detection of security monitoring systems, malware often hides its behavior. Current monitoring systems usually reside in the operating system (OS). Thus, it is hard to detect the existence of malware, especially the kernel rootkits. In this paper, a hidden OS objects detection and correlation approach based on VMM (virtual machine monitor) is proposed, and the corresponding detection system, vDetector, is designed and implemented. Both implicit and explicit information are used to create multiple views of OS objects, and a multi-view comparison mechanism are designed to identify three kinds of hidden OS objects: process, file and connections. The relations among hidden objects are established based on OS semantic information to trace the complete attack path. vDetector is implemented based on KVM virtualization platform and the effectiveness and performance overhead of vDetector are evaluated by comprehensive experiments. The results show that vDetector can successfully detect the existence of hidden OS objects with reasonable performance overhead.