Search

Driven by the high profit, Portable Executable (PE) malware has been consistently evolving in terms of both volume and sophistication. PE malware family classification has gained great attention and a large number of approaches have been proposed. With the rapid development of machine learning techniques and the exciting results they achieved on various tasks, machine learning algorithms have also gained popularity in the PE malware family classification task. Three mainstream approaches that use learning based algorithms, as categorized by the input format the methods take, are image-based, binary-based and disassembly-based approaches. Although a large number of approaches are published, there is no consistent comparisons on those approaches, especially from the practical industry adoption perspective. Moreover, there is no comparison in the scenario of concept drift, which is a fact for the malware classification task due to the fast evolving nature of malware. In this work, we conduct a thorough empirical study on learning-based PE malware classification approaches on 4 different datasets and consistent experiment settings. Based on the experiment results and an interview with our industry partners, we find that (1) there is no individual class of methods that significantly outperforms the others; (2) All classes of methods show performance degradation on concept drift (by an average F1-score of 32.23%); and (3) the prediction time and high memory consumption hinder existing approaches from being adopted for industry usage., Comment: 12 pages, 3 figures, conference

It is crucial to understand how malware variants are generated to bypass malware detection systems and understand their characteristics to improve the detectors' performances. To achieve this goal, we propose an evolutionary-based framework named FUMVar to generate Fully-working and Unseen Malware Variants. In particular, we applied FUMVar on portable executable (PE) files that have been used extensively to infect Windows operating systems. Compared to the state-of-the-art approach named AIMED, our experimental results show that FUMVar generated 25% more evasive malware variants while reducing the time taken to generate them by 23%. Furthermore, FUMVar generated malware variants that bypassed commercial anti-malware engines, such as TrendMicro, with an alarming rate of up to 73% false-negative rate. To improve the detection techniques, we evaluate how different perturbations enhance the evasiveness and how different malware categories are affected by those perturbations. The results show that perturbations' effectiveness varies significantly by up to 6 times (e.g., section add v.s. unpack), and more suitable perturbations can be selected for different malware categories due to their varying applications. This information can then be used to develop more robust malware detection systems to detect unseen malware variants more effectively.

Due to voluminous malware attacks in the cyberspace, machine learning has become popular for automating malware detection and classification. In this work we play devil's advocate by investigating a new type of threats aimed at deceiving multi-class Portable Executable (PE) malware classifiers into targeted misclassification with practical adversarial samples. Using a malware dataset with tens of thousands of samples, we construct three types of PE malware classifiers, the first one based on frequencies of opcodes in the disassembled malware code (opcode classifier), the second one the list of API functions imported by each PE sample (API classifier), and the third one the list of system calls observed in dynamic execution (system call classifier). We develop a genetic algorithm augmented with different support functions to deceive these classifiers into misclassifying a PE sample into any target family. Using an Rbot malware sample whose source code is publicly available, we are able to create practical adversarial samples that can deceive the opcode classifier into targeted misclassification with a successful rate of 75%, the API classifier with a successful rate of 83.3%, and the system call classifier with a successful rate of 91.7%.

One of the main challenges in security today is defending against unknown malware attacks which have the potential to harm a computer or network. Hence, detecting malware has become one of the most important challenges for the security of computer systems. The known malware detection methods based on the appearance of opcode sequences has to construct a matrix from programs of different architectures to extract high-level features. In order to resolve high dimensional inputs vector and differences assembly instruction, this paper proposes a novel method for detecting static characteristics of 32-bit and 64-bit malicious Portable Executable (PE) Windows files by opcode sequences analysis. By compute the frequency of occurrence of each opcode sequence and distinguishing different types of 32-bit and 64-bit PE files, the proposed method shows promising results with less complexity in comparison with previous studies, which is beneficial to train machine learning model such as k-nearest neighbor (KNN) and back-propagation neural network (BP). Our method is evaluated on more than 20,000 samples, and experimental results show that our system can effectively detect and classify unknown malware.

This paper presents the analysis of using data mining to detect unseen viruses. The supervised learning of ensemble classifiers, random forest and extreme gradient boosting, are being explored. The features consist of n-gram bytecode, n-gram opcode and basic-block opcode. Each feature is selected from top 8,192 attributes using term frequency-inverse document frequency. Half of the attributes are recursively removed by recursive feature elimination with 4-fold cross-validation. Every step of attribute removal is evaluated by 10-fold cross-validation. F1-score is used as a measurement for virus detection. Our data set contains 6,319 portable executable files. The result shows that data with more than 128 attributes achieve more than 0.936 F1-score for every model. Also, extreme gradient boosting is 66.33% faster using 256 attributes and 164.8% faster using 128 attributes compared to random forest. Moreover, trained model of extreme gradient boosting is smaller than random forest. However, random forest learns faster than extreme gradient boosting at that same number and types of features.

Many efforts have been made to use various forms of domain knowledge in malware detection. Currently there exist two common approaches to malware detection without domain knowledge, namely byte n-grams and strings. In this work we explore the feasibility of applying neural networks to malware detection and feature learning. We do this by restricting ourselves to a minimal amount of domain knowledge in order to extract a portion of the Portable Executable (PE) header. By doing this we show that neural networks can learn from raw bytes without explicit feature construction, and perform even better than a domain knowledge approach that parses the PE header into explicit features.

These days, Windows Operating System is widely used and it is dominant than other commercial operating systems. Besides, most of the commercial software vendors provide a number of software mainly for Windows OS. Therefore, software similarity analysis systems for Windows OS are highly useful for software plagiarism detection and copyright protection. However, mostly the software similarity analysis systems and techniques work on source codes, not executables, even though the majority of commercial software barely makes public its source codes. In this paper, we propose a novel scheme for software plagiarism detection for Windows executables. Our technique exploits both extended basic blocks' opcode sequences and imported API functions information in executable files. We implemented our scheme and measured its performance through several kinds of experiments using code samples and different versions of commercial software.

Behavior-based malware detection techniques remain one of the most efficient protections against malicious programs. Such techniques are based on constructing models representing malicious and legitimate behaviors by analyzing the set of APIs (Application Programming Interface) called by a program. Most of the existing behavior-based methods that provide a multiclass categorization of malware manually generate the models for each malware category. Due to the large number of existing malware types such techniques can be compute intensive and resource consuming. In this paper, we propose a new method based on the MCA (Multiple Correspondence Analysis) for identifying the set of most likely used APIs by PE (Portable Executable) malware. Our method will provide a clustering of likely used APIs according to different malware types.

Present day malware shows stealthy and dynamic capability and avails administrative rights to control the victim computers. Malware writers depend on evasion techniques like code obfuscation, packing, compression, encryption or polymorphism to avoid detection by Anti-Virus (AV) scanners as AV primarily use syntactic signature to detect a known malware. Our approach is based on semantic aspect of PE exectable that analyses API Call-grams to detect unknown malicious code. As in--exact source code is analysed, the machine is not infected by the executable. Moreover, static analysis covers all the paths of code which is not possible with dynamic behavioural methods as latter does not gurantee the execution of sample being analysed. Modern malicious samples also detect controlled virtual and emulated environments and stop the functioning. Semantic invariant approach is important as signature of known samples are changed by code obfuscation tools. Static analysis is performed by generating an API Call graph from control flow of an executable, then mining the Call graph as API Call-gram to detect malicious files.

Present day malware shows stealthy and dynamic capability to avail administrative rights and control the victim computer [10]. Malware writers depend on evasion techniques like code obfuscation, packing, compression, encryption or polymorphism to avoid detection by Anti-Virus (AV) scanners as AV primarily use signature based detection. According to the FireEye Threat report second half of 2011 [15], top 50 malware have generated 80% infections. Malware like Zues, Conficker, Koobface have become more stealthy by use of pay per install toolkits like Blackhole [15]. Pay per install toolkits make the samples dynamic in nature. This has led to exponential increase of unknown, zero-day malware [14].To complement the signatured approach, a good behavioral scheme is imminent due to exponential increase in number of encoded malware samples. Behavioural analysis can detect unknown, encrypted, zero day malware, but these methods result in increased false alarm rate. We propose a behaviour model that represents abstraction of a binary by analyzing the Application Programming Interface (API) strings made by Windows Portable Executable (PE) [25] files. Our focus is based on extracting temporal snapshots of malware and benign executables known as API Call-grams, as API strings are primarily written for software development kits to generate sane code. Malcode writers misues the available functionality to keep the code compact and escape being detected by AV software.

Packing is the most popular obfuscation technique used by malware writers' community in present scenario. The traditional signature-based anti-virus software had played a major role in malware detection, until the dawn of the trend of packed malware. Hence to evade detection of the malwares, a malicious writer relies on packers' softwares; which transforms the binary appearance of the programs without affecting its execution semantics. Therefore the biggest challenge today for malware detection techniques is to figure out whether a given binary is packed or not.In this paper, we apply pattern recognition technique for detection of packed malware binaries. The objective of our approach is to take out the best set of features from Windows Portable executable files in order to pass it to our classification model. The classification model works in two phases, in the first phase it classifies the packed and non-packed executables. Once an executable is classified as packed, the second phase of classification concludes whether it is packed benign or packed malware executable. We worked with the UPX packer for this approach and have been able to achieve more than 99.9% accuracy in the first phase of classification. We achieved more than 95% accuracy in the second phase of classification as well.

Financial loss due to malware nearly doubles every two years. For instance in 2006, malware caused near 33.5 Million GBP direct financial losses only to member organizations of banks in UK. Recent malware cannot be detected by traditional signature based anti-malware tools due to their polymorphic and/or metamorphic nature. Malware detection based on its immutable characteristics has been a recent industrial practice. The datasets are not public. Thus the results are not reproducible and conducting research in academic setting is difficult. In this work, we not only have improved a recent method of malware detection based on mining Application Programming Interface (API) calls significantly, but also have created the first public dataset to promote malware research.Our technique first reads API call sets used in a collection of Portable Executable (PE) files, then generates a set of discriminative and domain interpretable features. These features are then used to train a classifier to detect unseen malware. We have achieved detection rate of 99.7% while keeping accuracy as high as 98.3%. Our method improved state of the art technology in several aspects: accuracy by 5.24%, detection rate by 2.51% and false alarm rate was decreased from 19.86% to 1.51%. This project's data and source code can be found at http://home.shirazu.ac.ir/~sami/malware.

Software researchers have already developed static code security checkers to parse through and scan source code files, looking for security vulnerabilities [8, 9]. What about executable files? Can these files also be statically checked for security weaknesses such as buffer overflows? We have created a methodology that uses information located in the headers, sections, and tables of a Windows NT/XP executable file, along with information derived from the overall contents of the file, as a means to detect specific anomalies and software security vulnerabilities without having to disassemble the code. In addition, we have instantiated this methodology in a software utility program called findssv that automatically performs this static analysis.We tested findssv on six categories of files: executable installation files, software development files, Windows XP operating system files, Microsoft application files, security-centric applications files, and miscellaneous application files. Through the test results on over 2700 files, we show that findssv can detect 1) inconsistent table sizes, 2) large zero-filled regions of bytes, 3) unknown regions of bytes, 4) compressed files placed in a file, 5) sections that are both writable and executable, and 6) the use of functions susceptible to buffer overflow attacks. We also identify key vulnerability findings about the software in the six categories.