Search

The explosive growth in network traffic in recent times has resulted in increased processing pressure on network intrusion detection systems. In addition, there is a lack of reliable methods for preprocessing network traffic generated by benign applications that do not steal users’ data from their devices. To alleviate these problems, this study analyzed the differences between benign and malicious traffic produced by benign applications and malware, respectively. To fully express these differences, this study proposed a new set of statistical features for training a clustering model. Furthermore, to mine the communication channels generated by benign applications in batches, a semisupervised clustering method was adopted. Using a small number of labeled samples, our method aggregated historical network traffic into two types of clusters. The cluster that did not contain labeled malicious samples was regarded as a benign traffic cluster. The experimental results were compared using four types of clustering algorithms. The density-based spatial clustering of applications with noise (DBSCAN) clustering algorithm was selected to mine benign communication channels. We also compared our method with two other methods, and the results demonstrated that the benign channels mined through our method were more reliable. Finally, using our method, 1,811 benign transport layer security (TLS) channels were mined from 18,357 TLS communication channels. The number of flows carried by these benign channels comprised 65.37% of the entire network flows, and no malicious flow was included in our results, which proves the effectiveness of our method.

The popularity of encryption method brings a great challenge to malware traffic identification. Traditional classes defined by expert experience are usually classified based on the host behaviors of malware, such as banking malware or ransomware, which are often irrelevant to its communication traffic behaviors. It leads to the fact that the boundaries of traffic feature dataset of different malware classes are fuzzy and make these traditional classes unhelpful for classification based on traffic features. Meanwhile, traditional machine learning-based encrypted malware traffic identification methods, such as using the multi-classification supervised learning model, are inefficient both in model training and detection, and its detection accuracy cannot meet the demand. In this paper, we propose a distance-based method, which utilizes unsupervised learning algorithm Gaussian mixture model (GMM) and ordering points to identify the clustering structure (OPTICS) to calculate the Distance between malwares and make use of the Distance to define the new malware class called FClass. Then, a set of models are trained by XGBoost algorithm to build an identification framework based on the FClass. The performance of the proposed method has been evaluated by comparing it with the other four methods. The results show that the proposed distance-based method is more efficient and accurate.

For highly camouflaged Command and Control (C&C) communications, especially those using the Transport Security Layer (TLS) protocol, traditional classifiers which only based on statistical features or TLS handshake features gradually fail to detect such behavior. In this context, exploring features of other dimensions to build a more targeted recognition model is one of the ways to alleviate this problem. This paper proposed a new method of detecting malicious TLS traffic by using the communication channel as the detection unit, and a new set of modeling features for the communication channel was designed, including distribution features, the consistency features and statistical features of TLS communication channel. Experiments show that compared with other two types of features, the consistency features contribute most, and combining these three types of features together can train a better classifier which the precision reaches 92.57%. Comparative experiments show that the proposed method is more advantageous when faced with highly camouflaged TLS flows because the proposed method also achieved highest F1 score, and the accuracy is about 2% higher than the classifier based on the TLS handshake features, and 12% higher than the clustering model based on the statistical features of flow.

Pornographic and gambling websites become increasingly stubborn via disguising, misleading, blocking, and bypassing, which hinder the construction of a safe and healthy network environment. However, most traditional approaches conduct the detection process through a single aspect of these sites, which would fail to handle the more intricate and challenging situations. To alleviate this problem, this study proposed an automatic detection system for porn and gambling websites based on visual and textual content using a decision mechanism (PG-VTDM). This system can be applied to the intelligent wireless router at home or school to realize the identification, blocking, and warning of ill-suited websites. First, Doc2Vec was employed to learn the textual features that can be used to represent the textual content in the hypertext markup language (HTML) source code of the websites. In addition, the traditional bag-of-visual-words (BoVW) was improved by introducing local spatial relationships of feature points for better representing the visual features of the website screenshot. Then, based on these two types of features, a text classifier and an image classifier were both trained. In the decision mechanism, a data fusion algorithm based on logistic regression (LR) was designed to obtain the final prediction result by measuring the contribution of the two classification results to the final category prediction. The efficiency of this proposed approach was substantiated via comparison experiments using gambling and porn website datasets crawled from the Internet. The proposed approach outperformed the approach based on a single feature and some state-of-the-art approaches, with accuracy, precision, and F-measure all over 99%.

We present a novel method for detecting malicious TLS traffic based on communication channels that can detect deeply camouflaged malicious traffic. Moreover, we designed and extracted three types of channel features, namely, distribution features, consistency features of the Transport Layer Security (TLS) handshake field, and statistical features. Simultaneously, an efficacy feature selection method comprising a genetic algorithm is presented to obtain a global optimal feature subset, which reduces feature dimensions by 64% and increases accuracy by 1.5%. Comparison experiment results show that the proposed method possesses a more stable detection efficacy on different datasets with an accuracy of 97.65% and a much higher F1-score compared with other state-of-the-art classification methods.

Domain Name System (DNS) tunnels, established between the controlled host and master server disguised as the authoritative domain name server, can be used as a secret data communication channel for malicious activities. Owing to the ready evasion of the DNS traffic to bypass the network security mechanism, DNS tunnelling can cause severe damage. Thus, an in-depth and comprehensive understanding of the various detection technologies is of considerable importance when facing this type of threat. However, most of the existing reviews focus on a single aspect of the DNS tunnel detection technologies, such as methods based on machine learning, traffic, and payload analysis. In addition, few studies have conducted comprehensive investigation that includes a sequentially integrated range of literature in this researchfield, or have analysed the latest literature on DNS tunnels. This paper reviews these detection technologies from a novel perspective of rule-based and model-based methods with descriptions and analyses of the DNS-based tools and their corresponding features. To the best of our knowledge, this is the first study to comprehensively discuss and analyse DNS tunnel detection in a novel and specific classification fashion from various aspects in detail, covering almost all the detection methods developed from 2006 to 2020. Furthermore, a comparative analysis of detection methods and several suggestions for future research directions are presented.

The transport layer security (TLS) protocol is widely adopted by apps as well as malware. With the geometric growth of TLS traffic, accurate and efficient detection of malicious TLS flows is becoming an imperative. However, current studies focus on either detection accuracy or detection efficiency, and few studies take into account both indicators. In this paper, we propose a two-layer detection framework composed of a filtering model (FM) and a malware family classification model (MFCM). In the first layer, a new set of TLS handshake features is presented to train the FM, which is devised to filter out a majority of benign TLS flows. For identifying malware families, both TLS handshake features and statistical features are applied to construct the MFCM in the second layer. Comprehensive experiments are conducted to substantiate the high accuracy and efficiency of the proposed two-layer framework. A total of 96.32% of benign TLS flows can be filtered out by the FM with few malicious TLS flows being discarded provided the threshold of the FM is set to 0.01. Moreover, a multiclassifier is selected to construct the MFCM to provide better performance than a set of binary classifiers under the same feature set. In addition, when the ratio of benign and malicious TLS flows is set to 10:1, the detection efficiency of the two-layer framework is 188% faster than that of the single-layer framework, while the average detection accuracy reaches 99.45%.