Title: Enabling Detection of Elusive Malware by Going Out of the Box with Semantically Reconstructed View (OBSERV)

There is an alarming trend that elusive malware is armed with techniques that detect, evade, and subvert malware detection facilities of the victim. On the defensive side, a fundamental limitation of traditional host-based anti-malware systems is that they run inside the very hosts they are protecting, making them vulnerable to malware's counter-detection and subversion. To address this limitation, solutions using virtual machine (VM) technologies advocate placing the malware detection facility outside of the protected VM. However, a dilemma exists between these two approaches: The "out of the box" approach gains tamper resistance at the cost of losing the native, semantic view of the host enjoyed by the "in the box" approach.

To resolve the above dilemma, a new approach called OBSERV ("Out of the Box with SEmantically Reconstructed View") is introduced to achieve the advantages of both camps by reconstructing the semantic internal view of a VM from external, low-level observations. OBSERV enables three exciting malware defense opportunities: (1) malware detection by view comparison; (2) "out-of-the-box" execution of unmodified, off-the-shelf anti-malware software; and (3) non-intrusive VM monitoring, including the logging of guest system calls.

People

  • Xuxian Jiang (PI)
  • Dongyan Xu (PI)
  • Zhi Wang (PhD student)
  • Talks/Posters

  • Enabling Detection of Elusive Malware by Going Out of the Box with Semantically Reconstructed View (NSF Cyber Trust PI Meeting 08) (ppt)
  • Stealthy Malware Detection Through VMM-Based "Out-of-the-Box" Semantic View Reconstruction (CCS 07) (ppt)
  • "Out-of-the-Box" Monitoring of VM-Based High-Interaction Honeypots (RAID 07) (ppt)
  • Publications

  • Xuxian Jiang, Xinyuan Wang, Dongyan Xu, "Stealthy Malware Detection Through VMM-Based 'Out-of-the-Box' Semantic View Reconstruction", Proceedings of the 14th ACM Conference on Computer and Communications Security (CCS 2007), Alexandria, VA, November 2007 (18.0%).
  • Xuxian Jiang, Xinyuan Wang, "'Out-of-the-Box' Monitoring of VM-Based High-Interaction Honeypots", Proceedings of the 10th International Symposium on Recent Advances in Intrusion Detection (RAID 2007), Queensland, Australia, September 2007(18.0%).
  • Acknowledgement

    This project is supported by the National Science Foundation under Cyber-Trust Grant CNS-0716376 and CNS-0716444. Its conclusions and findings are those of the authors and do not necessarily reflect the views of the National Science Foundation.