Windows malware detector using convolutional neural network based on visualization images

Abstract

The evolution of malware is continuing at an alarming rate, despite the efforts made towards detecting and mitigating them. Malware analysis is needed to defend against its sophisticated behaviour. However, the manual heuristic inspection is no longer effective or efficient. To cope with these critical issues, behaviour-based malware detection approaches with machine learning techniques have been widely adopted as a solution. It involves supervised classifiers to appraise their predictive performance on gaining the most relevant features from the original features' set and the trade-off between high detection rate and low computation overhead. Though machine learning-based malware detection techniques have exhibited success in detecting malware, their shallow learning architecture is still deficient in identifying sophisticated malware. Therefore, in this paper, a Convolutional Neural Network (CNN) based Windows malware detector has been proposed that uses the execution time behavioural features of the Portable Executable (PE) files to detect and classify obscure malware. The 10-fold cross-validation tests were conducted to assess the proficiency of the proposed approach. The experimental results showed that the proposed approach was effective in uncovering malware PE files by utilizing significant behavioural features suggested by the Relief Feature Selection Technique. It attained detection accuracy of 97.968 percent. © 2013 IEEE.