Search

An analysis of a host behavior is an essential key for modern network management and security. A robust behavior profile enables the network managers to detect anomalies with high accuracy, predict the host behavior, or group host to clusters for better management. Hence, host profiling methods attract the interest of many researchers, and novel methods for host profiling are being introduced. However, these methods are frequently developed on preprocessed and small datasets. Therefore, they do not reflect the real-world artifacts of the host profiling, such as missing observations, temporal patterns, or variability in the profile characteristics in time. To provide the needed insight into the artifacts of host profiling in real-world settings, we present a study of the host behavior in a network conducted on a one-year-long real-world network dataset. In the study, we inspect the availability of the data for host profiling, identify the temporal patterns in host behavior, introduce a method for stable labeling of the hosts, and assess the variability of the host characteristics in the course of the year using the coefficient of variance. Moreover, we make the one-year dataset containing nine characteristics used for host behavior analysis available for public use and further research, including selected use cases representing host profiling caveats. We also share the record of analysis presented in the paper.

Cyber situation awareness has been recognized as a vital requirement for effective cyber defense. Cyber situation awareness allows cybersecurity operators to identify, understand, and anticipate incoming threats. Achieving and maintaining the cyber situation awareness is a challenging task given the continuous evolution of the computer networks, increasing volume and speeds of the data in a network, and rising number of threats to network security. Our work contributes to the continuous evolution of cyber situation awareness by the research of novel approaches to the perception and comprehension of a computer network. We concentrate our research efforts on the domain of IP flow network monitoring. We propose improvements to the IP flow monitoring techniques that enable the enhanced perception of a computer network. Further, we conduct detailed analyses of network traffic, which allows for an in-depth understanding of host behavior in a computer network. Last but not least, we propose a novel approach to IP flow network monitoring that enables real-time cyber situation awareness.

Cyber situational awareness is an essential part of cyber defense that allows the cybersecurity operators to cope with the complexity of today's networks and threat landscape. Perceiving and comprehending the situation allow the operator to project upcoming events and make strategic decisions. In this paper, we recapitulate the fundamentals of cyber situational awareness and highlight its unique characteristics in comparison to generic situational awareness known from other fields. Subsequently, we provide an overview of existing research and trends in publishing on the topic, introduce front research groups, and highlight the impact of cyber situational awareness research. Further, we propose an updated taxonomy and enumeration of the components used for achieving cyber situational awareness. The updated taxonomy conforms to the widely-accepted three-level definition of cyber situational awareness and newly includes the projection level. Finally, we identify and discuss contemporary research and operational challenges, such as the need to cope with rising volume, velocity, and variety of cybersecurity data and the need to provide cybersecurity operators with the right data at the right time and increase their value through visualization.

Flow monitoring has become an essential source of information for intrusion detection systems and various forms of network data analytics. However, the attention of researchers is focused primarily on the utilisation of the flow data, and the process of flow data creation is often neglected. This lack of consideration negatively affects the results of data analytics. Either the results are suboptimal due to the low quality of the flow data, or a description of the configuration of the flow monitoring system is missing, which leads to irreproducible results. The goal of this paper is to demonstrate how the configuration of the flow monitoring system affects the resulting data. The most basic flow monitoring configuration variables are the flow expiration timeouts. We analyse their effect on the number of created flow records to show their importance. Moreover, we demonstrate that the choice of the flow expiration timeouts can have a severe impact on the network data analytics. The use-case of Slowloris attack detection is used as an example to illustrate this fact.

Network segmentation is a powerful tool for network defense. In contemporary complex, dynamic, and multilayer networks, network segmentation suffers from lack of visibility into processes in the network, which results in less strict segment definition and loosen network security. Moreover, the dynamics of the networks makes the manual identification of network segments nearly impossible. In this paper, we inspect the possibilities of the behavior-aware network segmentation using IP flows and machine learning approaches that would enable to identify segments automatically, even in a complex network. We evaluate the suitability of clustering algorithms for identification of behavior-consistent segments in a network. We show that the clustering algorithms can identify relevant behavior-consistent clusters that overlap with those identified manually by experts. Apart from the segment identification, we investigate the other essential task of network segmentation process: assignment of an unknown host to an existing segment. We evaluate the performance of four different classification mechanisms on a real-world dataset. We show that it is possible to assign an unknown host to an appropriate network segment with up to 92% precision. Moreover, we release the whole dataset and experiment steps available for public use. Network segmentation is a powerful tool for network defense. In contemporary complex, dynamic, and multilayer networks, network segmentation suffers from lack of visibility into processes in the network, which results in less strict segment definition and loosen network security. Moreover, the dynamics of the networks makes the manual identification of network segments nearly impossible. In this paper, we inspect the possibilities of the behavior-aware network segmentation using IP flows and machine learning approaches that would enable to identify segments automatically, even in a complex network. We evaluate the suitability of clustering algorithms for identification of behavior-consistent segments in a network. We show that the clustering algorithms can identify relevant behavior-consistent clusters that overlap with those identified manually by experts. Apart from the segment identification, we investigate the other essential task of network segmentation process: assignment of an unknown host to an existing segment. We evaluate the performance of four different classification mechanisms on a real-world dataset. We show that it is possible to assign an unknown host to an appropriate network segment with up to 92% precision. Moreover, we release the whole dataset and experiment steps available for public use.

Research in network traffic measurement and analysis is a long-lasting field with growing interest from both scientists and the industry. However, even after so many years, results replication, criticism, and review are still rare. We face not only a lack of research standards, but also inaccessibility of appropriate datasets that can be used for methods development and evaluation. Therefore, a lot of potentially high-quality research cannot be verified and is not adopted by the industry or the community. The aim of this paper is to overcome this controversy with a unique solution based on a combination of distinct approaches proposed by other research works. Unlike these studies, we focus on the whole issue covering all areas of data anonymization, authenticity, recency, publicity, and their usage for research provability. We believe that these challenges can be solved by utilization of semi-labeled datasets composed of real-world network traffic and annotated units with interest-related packet traces only. In this paper, we outline the basic ideas of the methodology from unit trace collection and semi-labeled dataset creation to its usage for research evaluation. We strive for this proposal to start a discussion of the approach and help to overcome some of the challenges the research faces today.

Operating system fingerprinting methods are well-known in the domain of static networks and managed environments. Yet few studies tackled this challenge in real networks, where users can bring and connect any device. We evaluate the performance of three OS fingerprinting methods on a large dataset collected from university wireless network. Our results show that method based on HTTP User-agents is the most accurate but can identify only low portion of the traffic. TCP/IP parameters method proved to be the opposite with high identification rate but low accuracy. We also implemented a new method based on detection of communication to OS-specific domains and its performance is comparable to the two established ones. After that, we discuss the impacts of traffic encryption and embracing new protocols such as IPv6 or HTTP/2.0 on OS fingerprinting. Our findings suggest that OS identification based on specific domain detection is viable and corresponds to the current directions of network traffic evolution, while methods based on TCP/IP parameters and User-agents will become ineffective in the future.

In today's complex computer networks, we are constantly facing a risk of data loss, system compromise, or intellectual property theft. The complexity of the networks hinders their effective defense. A Network-wide Cyber Situational Awareness (NwCSA) has been introduced to assist a network security administrator with network security. The concept, how-ever, faces several challenges that hinder an efficient application of the NwCSA in a real-world environment. The challenges include the overload of raw data, low speed of reaction, and a lack of context and unified view on a network. In this paper, we present a novel framework that faces above mentioned challenges. The framework leverages a distributed data stream processing system and methods for real-time big data processing. The framework is evaluated with respect to stated requirements on systems for NwCSA. Moreover, we present a prototype framework implementation and provide lessons learned from its real-world deployment.

In this paper, we present Stream4Flow, a framework for cyber situational awareness based on Apache Spark Streaming. We demonstrate utilization of Stream4Flow for real-time IP flow host monitoring in a large campus network. Contemporary IP flow analysis systems are not designed for the continuous host monitoring. Gaining the detailed overview of each host is not straightforward with these systems due to connection-based paradigm and performance challenges. We show that distributed stream processing is a natural solution for detailed IP flow host monitoring. Moreover, we describe a real-time host monitoring workflow in data streams in detail and present advantages of flow-based host monitoring in Apache Spark including real-time host profiling, dynamic level of detail and granularity.

Analyzing IP flows is an essential part of traffic measurement for cyber security. Based on information from IP flows, it is possible to discover the majority of concurrent cyber threats in highspeed, large-scale networks. Some major prevailing challenges for IP flow analysis include, but are not limited to, analysis over a large volume of IP flows, scalability issues, and detecting cyber threats in real time. In this article, we discuss the transformation of present IP flow analysis into a stream-based approach to face current challenges in IP flow analysis. We examine the possible positive and negative impacts of the transformation and present examples of real-world applications, along with our recommendations. Our ongoing results show that stream-based IP flow analysis successfully meets the above-mentioned challenges and is suitable for achieving real-time network security analysis and situational awareness.

The encryption of network traffic complicates legitimate network monitoring, traffic analysis, and network forensics. In this paper, we present real-time lightweight identification of HTTPS clients based on network monitoring and SSL/TLS fingerprinting. Our experiment shows that it is possible to estimate the User-Agent of a client in HTTPS communication via the analysis of the SSL/TLS handshake. The fingerprints of SSL/TLS handshakes, including a list of supported cipher suites, differ among clients and correlate to User-Agent values from a HTTP header. We built up a dictionary of SSL/TLS cipher suite lists and HTTP User-Agents and assigned the User-Agents to the observed SSL/TLS connections to identify communicating clients. The dictionary was used to classify live HTTPS network traffic. We were able to retrieve client types from 95.4 % of HTTPS network traffic. Further, we discussed host-based and network-based methods of dictionary retrieval and estimated the quality of the data.

—In this paper, we present a framework for the realtime generation of network traffic statistics on Apache Spark Streaming, a modern distributed stream processing system. Our previous results showed that stream processing systems provide enough throughput to process a large volume of NetFlow data and hence they are suitable for network traffic monitoring. This paper describes the integration of Apache Spark Streaming into a current network monitoring architecture. We prove that it is possible to implement the same basic methods for NetFlow data analysis in the stream processing framework as in the traditional ones. Moreover, our stream processing implementation discovers new information which is not available when using traditional network monitoring approaches.

In the era of Internet of Things (IoT), the volume of the monitored data from IoT network is enormous. However, not all data provide sufficient or relevant information. Since the analysis of big data is both resource and time exhausting, only relevant information should be analysed. In this paper, we scrutinize the widely used Top N statistics and evaluate its information value with respect to gathering information about individual hosts in the network. All theoretical discussions are evaluated on the real-world data. Moreover, we provide an assessment of statistic's suitability for identifying a host in network traffic. The results of the paper should assist data analyst of IoT network data.

Part 6: Monitoring Methods for Quality-of-Service and Security; International audience; The rapid development of network technologies entails an increase in traffic volume and attack count. The associated increase in computational complexity for methods of deep packet inspection has driven the development of behavioral detection methods. These methods distinguish attackers from valid users by measuring how closely their behavior resembles known anomalous behavior. In real-life deployment, an attacker is flagged only on very close resemblance to avoid false positives. However, many attacks can then go undetected. We believe that this problem can be solved by using more detection methods and then correlating their results. These methods can be set to higher sensitivity, and false positives are then reduced by accepting only attacks reported from more sources. To this end we propose a novel sketch-based method that can detect attackers using a correlation of particular anomaly detections. This is in contrast with the current use of sketch-based methods that focuses on the detection of heavy hitters and heavy changes. We illustrate the potential of our method by detecting attacks on RDP and SSH authentication by correlating four methods detecting the following anomalies: source network scan, destination network scan, abnormal connection count, and low traffic variance. We evaluate our method in terms of detection capabilities compared to other deployed detection methods, hardware requirements, and the attacker’s ability to evade detection.

Cyber attacks have become ubiquitous and in order to face current threats it is important to understand them. Studying attacks in a real environment however, is not viable and therefore it is necessary to find other methods how to examine the nature of attacks. Gaining detailed knowledge about them facilitates designing of new detection methods as well as understanding their impact. In this paper we present a testbed framework to simulate attacks that enables to study a wide range of security scenarios. The framework provides a notion of real-world arrangements, yet it retains full control over all the activities performed within the simulated infrastructures. Utilizing the sandbox environment, it is possible to simulate various security attacks and evaluate their impacts on real infrastructures. The design of the framework benefits from IaaS clouds. Therefore its deployment does not require dedicated facilities and the testbed can be deployed over miscellaneous contemporary clouds. The viability of the testbed has been verified by a simulation of particular DDoS attack.

Many vulnerabilities are operating system specific. Information about the OS of all hosts in a network represents a valuable asset for network administrators. While OS detection in small networks is an easy task, expanding the same process on a large scale becomes a challenge. The weak performance, high speed traffic and large amount of hosts for OS detection are issues to overcome. In this paper we propose a flow based framework for large scale OS detection. Furthermore, we describe the framework implementation into a flow probe, provide performance comparison and share remarks on deployment in a real world network.

In this paper we present a cloud-based research testbed designed to aid network security managers. The testbed enables operators to emulate various network topologies, services, and to analyze attacks threatening these systems. A possibility to test results of network management measures is desired, since testing these measures in a production environment is always not possible. We demonstrate a testbed use case, which aids to scrutinize network behavior under attack. Our use case is based on a large DDoS attack which targeted network infrastructure and web servers in Czech Republic in March, 2013. Článek popisuje cloudové testovací prostředí navržené pro využití správci síťové bezpečnosti. Testovací prostředí umožňuje simulovat různé síťové topologie, služby a provádět analýzu útoků zaměřených proti provozovaným službám. Protože není vždy možné provádět tuto analýzu na reálné síti, je potřeba mít k dipozici testovací prostředí s výše uvedenými vlastnostmi. V tomto článku je popsána případová studie využití cloudového testovacího k simulaci a analýze chování sítě pod náporem DDoS útoku. Studie je inspirována DDoS útokem na významné síťové infrastruktury v březnu 2013.

The exhaustion of IPv4 address space increases pressure on network operators and content providers to continue the transition to IPv6. The IPv6 transition mechanisms such as Teredo and 6to4 allow IPv4 hosts to connect to IPv6 hosts. On the other hand, they increase network complexity and render ineffective many methods to observe IP traffic. In this paper, we modified our flow-based measurement system to involve transition mechanisms information to provide full IPv6 visibility. Our traffic analysis focuses on IPv6 tunneled traffic and uses data collected over one week in the Czech national research and education network. The results expose various traffic characteristics of native and tunneled IPv6 traffic, among others the TTL and HOP limit distribution, geolocation aspect of the traffic, and list of Teredo servers used in the network. Furthermore, we show how the traffic of IPv6 transition mechanisms has evolved since 2010. Vyčerpání IPv4 adresního prostoru zvyšuje naléhavost přechodu na IPv6 protokol. Přechodové mechanismy jako Teredo a 6to4 umožňují komunikaci mezi IPv4 a IPv6 systémy. Tyto přechodové mechanismy však zvyšují složitost sítě a mnoho nástrojů monitorujících sít není schopno tento typ provozu sledovat. V článku jsme upravili monitorovací systém založený na IP tocích tak, aby bylo možné sledovat i provoz tvořený přechodovými mechanismy. Následná analýza se zaměřila na IPv6 tunelovaný provoz přechodových mechanismů na datech ze sítě CESNET. Výsledky analýzy popisují charakteristiky tunelovaného a nativního IPv6 provozu. Je popsáno rozložení hodnot TTL a HOP limitů, geolokační aspekty provozu a seznam Teredo serverů používaných v měřené síti. Je ukázán vývoj množství provozu přechodovými mechanismů v síti CESNET od roku 2010.

Network connected devices has become inherent part of our lives. These devices have come to be more and more mobile and are target of various malware attacks. An inability to guarantee or check proper security settings of such devices poses a serious risk to network security. In this paper we propose a novel concept of flow based host trustworthiness estimation. The estimated trustworthiness determines a level of the risk to the network security the host posses. This concept enables network operators to identify a potential dangerous host in their network and take an appropriate precautions. Models used for trustworthiness estimation are based on scoring either single events or host characteristics. In order to be able to estimate trustworthiness of a host even in large scale networks, the data used for estimation are reduced only to extended network flows. The research is in its initial phase and will conclude with Ph.D. thesis in three years. Zařízení připojené do sítě se stala neodmyslitelnou součástí našeho života. Tyto zařízení jsou stále mobilnější a stávají se cílem různých druhů škodlivého softwaru. Neschopnost zaručit či prověřit správné nastavení bezpečnostního zajištění těchto zařízení představuje nezanedbatelné riziko pro bezpečnost počítačové sítě. V tomto článku je představen koncept pro odhadování míry důvěryhodnosti daného zařízení. Míra důvěryhodnosti reprezentuje míru rizika, kterou zařízení představuje pro síťovou bezpečnost. Tento koncept umožní správcům sítě identifikovat potenciálně nebezpečné zařízení. Modely pro odhadování důvěryhodnosti jsou založeny na scoringu buď jednotlivých událostí nebo charakteristik zařízení. Aby bylo možné odhadovat důvěryhodnost zařízené dokonce i v rozsáhlých sítích, pro odhad důvěryhodnosti jsou využívána pouze data ze síťových toků. Výzkum je v jeho počáteční fázi a bude v horizontu tří let završen disertační prací.

Performing research on live network traffic requires the traffic to be well documented and described. The results of such research are heavily dependent on the particular network. This paper presents a study of network characteristics, which can be used to describe the behaviour of a network. We propose a number of characteristics that can be collected from the networks and evaluate them on five different networks of Masaryk University. The proposed characteristics cover IP, transport and application layers of the network traffic. Moreover, they reflect strong day-night and weekday patterns that are present in most of the networks. Variation in the characteristics between the networks indicates that they can be used for the description and differentiation of the networks. Furthermore, a weak correlation between the chosen characteristics implies their independence and contribution to network description.