Search

Network monitoring is essential for operators to review and optimize network behaviour, and to troubleshoot any issues that arise. With the adoption of unlicensed band and spectrum-sharing technologies, conventional optimization techniques are no longer feasible. As cellular networks grow in complexity, data-driven solutions are becoming increasingly central to ensuring optimal network performance, reliability, and user experience. Recent cellular research has focused on data-driven analysis and optimization to ensure high Quality of Service in a cellular network. However, the biggest challenge for such work is data collection at scale from sophisticated and reliable network monitoring tools. This work bridges a gap in cellular network literature with a thorough review of cellular network monitoring tools. We rigorously test and review eleven applications that are popularly used to collect information on cellular networks. We understand, analyze, and critique their architecture to assess their reliability. We share insights on their performance and ease of use. Most importantly, we analyze the ability of applications to collect accurate cellular network data at scale. We also review recent literature on cellular network monitoring and analysis after carefully selecting 32 papers relevant to the topic.

Darknets are probes listening to traffic reaching IP addresses that host no services. Traffic reaching a darknet results from the actions of internet scanners, botnets, and possibly misconfigured hosts. Such peculiar nature of the darknet traffic makes darknets a valuable instrument to discover malicious online activities, e.g., identifying coordinated actions performed by bots or scanners. However, the massive amount of packets and sources that darknets observe makes it hard to extract meaningful insights, calling for scalable tools to automatically identify and group sources that share similar behaviour. We here present i-DarkVec, a methodology to learn meaningful representations of Darknet traffic. i-DarkVec leverages Natural Language Processing techniques (e.g., Word2Vec) to capture the co-occurrence patterns that emerge when scanners or bots launch coordinated actions. As in NLP problems, the embeddings learned with i-DarkVec enable several new machine learning tasks on the darknet traffic, such as identifying clusters of senders engaged in similar activities. We extensively test i-DarkVec and explore its design space in a case study using real darknets.We showthat with a proper definition of services, the learned embeddings can be used to (i) solve the classification problem to associate unknown sources' IP addresses to the correct classes of coordinated actors and (ii) automatically identify clusters of previously unknown sources performing similar attacks and scans, easing the security analyst's job. i-DarkVec leverages a novel incremental embedding learning approach that is scalable and robust to traffic changes, making it applicable to dynamic and large-scale scenarios. [ABSTRACT FROM AUTHOR]

Abstract Researchers have paid a lot of attention to complex networks in recent decades. Due to their rapid evolution, they turn into a major scientific and innovative field. Several studies on complex networks are carried out, and other subjects are evolving every day such as the challenge of detecting influential nodes. In this study, we provide a brief overview of complex networks, as well as several concepts key related to measurements, the structure of complex network and social influence, an important state of the art on complex networks including basic metrics on complex networks, the evolution of their topology over the years as well as the dynamic of networks. A detailed literature about influential finding approaches is also provided to indicate their strength and shortcomings. We aim that our contribution of literature can be an interesting base of information for beginners’ scientists in this field. At the end of this paper, some conclusions are drawn and some future perspectives are mentioned to be studied as new directions in the future. More detailed references are provided to go further and deep in this area.

Content recommendation systems, also known as recommenders, are pervasive and have significant impact on user demands over the Internet. Platforms such as YouTube and Netflix constantly seek to improve their recommendation systems, to provide better quality of experience (QoE) for their users. QoE, in turn, depends on a multitude of factors, including the quality of recommendation (QoR), e.g., based on users histories and content categories, and the quality of service (QoS), e.g., measured by network delay and throughput. Even though QoS is key in a best-effort Internet, existing recommendation systems overlook it, resulting in recommendations that are suboptimal in terms of QoE. In this study, our goal is to devise a QoS-aware, QoE-friendly, content recommendation system and indicate its feasibility in the wild. For this purpose, we conducted an experiment with real users driven by the following question: When should recommenders account for low QoS? Each user is requested to evaluate pairs of videos, that vary in their contents and QoS levels. We experimentally determined category-dependent thresholds that determine the sensitivity of users with respect to QoS and QoR. Given the collected insights on QoS-aware recommendations, we considered our second research question: Can recommenders compensate for low QoS? We conducted experiments over the Internet, relying on YouTube API and network measurements tools, and report our findings on (i) the characterization of QoS and (ii) the compensation for low QoS. Our measurements suggest that content far from the trends tends to be far from the user. We quantified the extent to which unpopular content tends to be served with a lower QoS and established a methodology to determine the relationship between content popularity and its physical proximity to users. Then, we verified that making requests a bit trendier can hit much closer content.

The application of machine learning (ML) to mitigate network-related problems poses significant challenges for researchers and operators alike. For one, there is a general lack of labeled training data in networking, and labeling techniques popular in other domains are ill-suited due to the scarcity of operators’ domain expertise. Second, network problems are typically multi-tasked in nature, requiring multiple ML models (one per task) and resulting in multiplicative increases in training times as the number of tasks increases. Third, the adoption of ML by network operators hinges on the models’ ability to provide basic reasoning about their decision-making procedures. To address these challenges, we propose ARISE, a multi-task weak supervision framework for network measurements. ARISE uses weak supervision-based data programming to label network data at scale and applies learning paradigms such as multi-task learning (MTL) and meta-learning to facilitate information sharing between tasks as well as reduce overall training time. Using community datasets, we show that ARISE can generate MTL models with improved classification accuracy compared to multiple single-task learning (STL) models. We also report findings that show the promise of MTL models for providing a means for reasoning about their decision-making process, at least at the level of individual tasks. [ABSTRACT FROM AUTHOR]

Cloud gaming is a class of services that promises to revolutionize the videogame market. It allows the user to play a videogame with essential equipment while using a remote server for the actual execution. The multimedia content is streamed through the network from the server to the user. Hence, this service requires low latency and a large bandwidth to work properly with low response time and high-definition video. Three of the leading tech companies (Google, Sony, and NVIDIA) entered this market with their products, and others, like Microsoft and Amazon, are also launching their platforms. However, these companies have released little information about their cloud gaming operation and how they utilize the network. In this work, we study cloud gaming services from the network point of view. We collect more than 200 packet traces under different application settings and network conditions from a broadband network to poor mobile network conditions, for 3 cloud gaming services, namely Stadia from Google, GeForce Now from NVIDIA and PS Now from Sony. We analyze the employed protocols and the workload that they impose on the network. We find that GeForce Now and Stadia use the RTP protocol to stream the multimedia content, with the latter relying on the standard WebRTC APIs. Depending on the network and video quality, they result in bandwidth-hungry services consuming up to 45 Mbit/s. PS Now instead uses only undocumented protocols and never exceeds 13 Mbit/s. 4G mobile networks can often sustain these loads, while traditional 3G connections struggle. The systems quickly react to deteriorated network conditions, and packet losses up to 5% do not cause a reduction in resolution.

Joint caching and recommendation has been recently proposed as a new paradigm for increasing the efficiency of mobile edge caching. Early findings demonstrate significant gains for the network performance. However, previous works evaluated the proposed schemes exclusively on simulation environments. Hence, it still remains uncertain whether the claimed benefits would change in real settings. In this paper, we propose a methodology that enables to evaluate joint network and recommendation schemes in real content services by only using publicly available information. We apply our methodology to the YouTube service, and conduct extensive measurements to investigate the potential performance gains. Our results show that significant gains can be achieved in practice; e.g., 8 to 10 times increase in the cache hit ratio from cache-aware recommendations. Finally, we build an experimental testbed and conduct experiments with real users; we make available our code and datasets to facilitate further research. To our best knowledge, this is the first realistic evaluation (over a real service, with real measurements and user experiments) of the joint caching and recommendations paradigm. Our findings provide experimental evidence for the feasibility and benefits of this paradigm, validate assumptions of previous works, and provide insights that can drive future research. [ABSTRACT FROM AUTHOR]

With the increasing number of geographically distributed scientific collaborations and the growing sizes of scientific data, it has become challenging for users to achieve the best possible network performance on a shared network. We have developed a model to forecast expected bandwidth utilization on high-bandwidth wide area networks. The forecast model can improve the efficiency of the resource utilization and scheduling of data movements on high-bandwidth networks to accommodate ever increasing data volume for large-scale scientific data applications. A univariate time-series forecast model is developed with the Seasonal decomposition of Time series by Loess (STL) and the AutoRegressive Integrated Moving Average (ARIMA) on Simple Network Management Protocol (SNMP) path utilization measurement data. Compared with the traditional approach such as Box-Jenkins methodology to train the ARIMA model, our forecast model reduces computation time up to 92.6 %. It also shows resilience against abrupt network usage changes. Our forecast model conducts the large number of multi-step forecast, and the forecast errors are within the mean absolute deviation (MAD) of the monitored measurements.

Long-range wide-area network (LoRaWAN) is an energy-efficient and inexpensive networking technology that is rapidly being adopted for many Internet-of-Things applications. In this study, we perform extensive measurements on a new LoRaWAN deployment to characterise the spatio-temporal properties of the LoRaWAN channel. Our experiments reveal that LoRaWAN frames are mostly lost due to the channel effects, which are adverse when the end-devices are mobile. The frame losses are up to 70 percent, which can be bursty for both mobile and stationary scenarios. Frame losses result in data losses since the frames are transmitted only once in the basic configuration. To reduce data losses in LoRaWAN, we design a novel coding scheme for data recovery called DaRe that works on the application layer. DaRe combines techniques from convolutional and fountain codes. By implementing DaRe, we show that 99 percent of the data can be recovered with a code rate of 1/2 when the frame loss is up to 40 percent. Compared to the repetition coding scheme, DaRe provides 21 percent higher data recovery and can save up to 42 percent of the energy consumed on a transmission for 10-byte data units. We also show that DaRe provides better resilience to bursty frame losses. [ABSTRACT FROM AUTHOR]

5G is the new generation of 3GPP-based cellular communications that provides remarkable connectivity capabilities and extreme network performance to mobile network operators and cellular users worldwide. The rollout process of a new capacity layer (cell) on top of the existing previous cellular technologies is a complex process that requires time and manual effort from radio planning-engineering teams and parameter optimization teams. When it comes to optimum configuration of the 5G gNB cell parameters, the maximization of achieved coverage (RSRP) and quality (SINR) of the served mobile terminals are of high importance for achieving the very high data transmission rates expected in 5G. This process strongly relies on network measurements that can be even more insightful when mobile terminal localization information is present. This information can be generated by modern algorithmic techniques that act on the cellular network signaling measurements. Configuration algorithms can then use these measurements combined with location information to optimize various cell deployment parameters such as cell azimuth. Furthermore, data-driven approaches are shown in the literature to outperform traditional, model-based algorithms as they can automate the optimization of parameters while specializing in the characteristics of each individual geographical zone. In the context of the above, in this paper, we tested the automated network reconfiguration schemes based on unsupervised learning and applied statistics for cell azimuth steering. We compared network metric clustering and geospatial clustering to be used as our baseline algorithms that are based on K-means with the proposed scheme—hybrid network and spatial clustering based on hierarchical DBSCAN. Each of these algorithms used data generated by an initial scenario to produce cell re-configuration actions and their performance was then evaluated on a validated simulation platform to capture the impact of each set of gNB reconfiguration actions. Our performance evaluation methodology was based on statistical distribution analysis for RSRP and SINR metrics for the reference scenario as well as for each reconfiguration scheme. It is shown that while both baseline algorithms improved the overall performance of the network, the proposed hybrid network–spatial scheme greatly outperformed them in all statistical criteria that were evaluated, making it a better candidate for the optimization of 5G capacity layers in modern urban environments. [ABSTRACT FROM AUTHOR]

The increasing success of Machine Learning (ML) and Deep Learning (DL) has recently re-sparked interest towards traffic classification. While supervised techniques provide satisfactory performance when classifying known traffic, the detection of zero-day (i.e., unknown) traffic is a more challenging task. At the same time, zero-day detection, generally tackled with unsupervised techniques such as clustering, received less coverage by the traffic classification literature which focuses more on deriving DL models via supervised techniques. Moreover, the combination of supervised and unsupervised techniques poses challenges not fully covered by the traffic classification literature. In this paper, we share our experience on a commercial-grade DL traffic classification engine that combines supervised and unsupervised techniques to identify known and zero-day traffic. In particular, we rely on a dataset with hundreds of very fine grained application labels, and perform a thorough assessment of two state of the art traffic classifiers in commercial-grade settings. This pushes the boundaries of traffic classifiers evaluation beyond the few tens of classes typically used in the literature. Our main contribution is the design and evaluation of GradBP, a novel technique for zero-day applications detection. Based on gradient backpropagation and tailored for DL models, GradBP yields superior performance with respect to state of the art alternatives, in both accuracy and computational cost. Overall, while ML and DL models are both equally able to provide excellent performance for the classification of known traffic, the non-linear feature extraction process of DL models backbone provides sizable advantages for the detection of unknown classes over classical ML models. [ABSTRACT FROM AUTHOR]

Network programmability also sneaked into the mobile world leading to the emergence of Software-Defined Radio Access Network (SD-RAN) architectures. Interestingly, while only a small number of prototype architectures exist for SD-RAN, their performance evaluations are unfortunately also limited. Recent evaluations are carried out for small network dimensions of up to 50 devices, while emerging 5G/6G networks envision numbers of devices beyond 5000. Although 5G/6G applications are more stringent with respect to latency guarantees, performance evaluations of such low scale remain questionable. To fill this void, this paper presents MARC: a novel benchmarking tool for SD-RAN architectures and their controllers. We use MARC to measure, analyze and identify performance implications for two state-of-the-art open source SD-RAN solutions: FlexRAN and 5G-EmPOWER. We perceive results for monitoring application scenarios considering fully centralized control. For this setting, our findings show that the proposed architectures with a single SD-RAN controller are not scalable and can even lead to unpredictable network operations. Using our tool and based on our insights, we provide and implement design guidelines for the internal working behavior of the existing controllers. [ABSTRACT FROM AUTHOR]

The reliability of air quality simulations has a strong dependence on the input emissions inventories, which are associated with various sources of uncertainties, particularly in regions undergoing rapid emission changes where inventories can be 'out of date' almost as soon as they are compiled. This work provides a new methodology for updating emissions inventories by source sector using air quality ensemble simulations and observations from a dense monitoring network. It is adopted to determine the short-term trends in carbon monoxide (CO) emissions, an important pollutant and precursor to tropospheric ozone, in a study area centred around Beijing following the implementation of clean air policies. We sample the uncertainties associated with using an a priori emissions inventory for the year 2013 in air quality simulations of 2016, using an atmospheric dispersion model combined with a perturbed emissions ensemble (PEE), which is constructed based on expert-elicited uncertainty ranges for individual source sectors in the inventory. By comparing the simulation outputs with observational constraints, we are able to constrain the emissions of key source sectors relative to those in the a priori emissions inventory. From 2013 to 2016, we find a 44–88% reduction in the transport sector emissions (0.92–4.4×105 Mg in 2016) and a minimum 61% decrease in residential sector emissions (<3.5×105 Mg in 2016) within the study area. We also provide evidence that the night-time fraction of traffic sources in 2016 was higher than that in the 2013 emissions inventory. This study shows the applicability of PEEs and high-resolution observations in providing timely updates of emission estimates by source sector. [ABSTRACT FROM AUTHOR]

In this article, we present a method to detect and identify mobile network signals. The method relies on features extracted from the power spectral density of the signals. The performance of the proposed method is evaluated using over-the-air measurements acquired in various cellular bands, in different geographical areas, and in rural and urban settings. Results verify the high detection and identification accuracy of the method. [ABSTRACT FROM AUTHOR]

Traffic prediction is significantly important for performance analysis and network planning in Software Defined Networking (SDN). However, to effectively predict network traffic in current networks is very difficult and nearly prohibitive. As a new cutting-edge network technology, SDN decouples the control and data planes of network switch devices to enable the flexibility of network measurements and managements. The SDN architecture of the flow-based forwarding idea brings forth a promising of network traffic capture and prediction. We propose a lightweight traffic prediction algorithm for SDN applications. Firstly, different from traditional network traffic measurements, our method uses the flow-based forwarding idea in SDN to extract traffic statistic from data plane. The statistical variable describes network flow information forwarded in SDN and enables more accurate measurements of flow traffic via a direct and low-overhead way compared with traditional traffic measurements. Secondly, based on the temporal nature of traffic, the time-correlation theory is utilized to model flow traffic, where the time-series analysis theory and regressive modeling approach are used to characterize network traffic in SDN. A fully new method is proposed to perform traffic prediction. Thirdly, we propose the flow-based forwarding traffic prediction algorithm to forecast to SDN traffic. The detailed algorithm process is discussed and analyze. Finally, sufficient experiments are presented and designed to validate the proposed method. Simulation results show that our method is feasible and effective. [ABSTRACT FROM AUTHOR]

Network dimensioning is a critical task for cellular operators to avoid degraded user experience and unnecessary upgrades of network resources with changing mobile traffic patterns. For this purpose, smart network planning tools require accurate cell and user capacity estimates. In these tools, throughput is often used as a capacity metric due to its close relationship with user satisfaction. In this work, a comprehensive analysis is carried out to compare different well-known Supervised Learning (SL) algorithms for estimating cell and user throughput in the DownLink in busy hours from radio measurements collected on a cell basis in the Operation Support System (OSS). The considered SL approaches include random forest, shallow multi-layer perceptron, support vector regression and k-nearest neighbors. Such algorithms are compared with classical multiple linear regression and deep learning approaches considered in previous works. All these algorithms are tested in two radio access technologies: High Speed DownLink Packet Access (HSDPA) and Long Term Evolution (LTE). To this end, two datasets with the most relevant performance indicators per technology are collected from live cellular networks. Results show that non-deep SL algorithms are the most appropriate option for applications with storage constraints, such as network planning tools, since they provide a higher accuracy with reduced datasets. [ABSTRACT FROM AUTHOR]

Democratic principles, from the freedom of speech, to fair business practices, rely on Net neutrality, i.e. equal access to communication infrastructure and services. While a number of national and international regulations stipulate Net neutrality, the actual enforcement is challenging as regulators have to collect and analyze a large amount of network measurements, and pinpoint cases of neutrality violations. Through a large-scale distributed crowdsourced measurements campaign, the Agency for Communication Networks and Services of the Republic of Slovenia (AKOS) has acquired a massive dataset of Internet performance measurements in Slovenia. In this work we analyze about one million multi-dimensional data records gathered by the AKOS Test Net measurement system and identify the practices, such as port blocking, that might violate Net neutrality principles. We then chart the limitations of the employed measurement approach and propose a holistic multi-stakeholder approach ensuring high quality measurement data upon which reliable Net neutrality violation inferences should be based. [ABSTRACT FROM AUTHOR]

Sketches are probabilistic data structures designed for recording frequencies of items in a multi-set. They are widely used in various fields, especially for gathering Internet statistics from distributed data streams in network measurements. In a distributed streaming application with high data rates, a sketch in each monitoring node “fills up” very quickly and then its content is transferred to a remote collector responsible for answering queries. Thus, the size of the contents transferred must be kept as small as possible while meeting the desired accuracy requirement. To obtain significantly higher accuracy while keeping the same update and query speed as the best prior sketches, in this article, we propose a new sketch – the Slim-Fat (SF) sketch. The key idea behind the SF-sketch is to maintain two separate sketches: a larger sketch, the Fat-subsketch, and a smaller sketch, the Slim-subsketch. The Fat-subsketch is used for updating and periodically producing the Slim-subsketch, which is then transferred to the remote collector for answering queries quickly and accurately. We also present the error bound as well as an accurate model of the correct rate of the SF-sketch, and verify their correctness through experiments. We implemented and extensively evaluated the SF-sketch along with several prior sketches. Our results show that when the size of our Slim-subsketch and of the widely used Count-Min (CM) sketch are kept the same, our SF-sketch outperforms the CM-sketch by up to 33.1 times in terms of accuracy (when the ratio of the sizes of the Fat-subsketch and the Slim-subsketch is 16:1). We have made all source codes publicly available at Github [“Source code of SF sketches,” [Online]. Available: https://github.com/paper2017/SF-sketch ]. [ABSTRACT FROM AUTHOR]

The Web is one of the most successful Internet applications. Yet, the quality of Web users’ experience is still largely impenetrable. Whereas Web performance is typically studied with controlled experiments, in this work we perform a large-scale study of a real site, Wikipedia, explicitly asking (a small fraction of its) users for feedback on the browsing experience. The analysis of the collected feedback reveals that 85% of users are satisfied, along with both expected (e.g., the impact of browser and network connectivity) and surprising findings (e.g., absence of day/night, weekday/weekend seasonality) that we detail in this paper. Also, we leverage user responses to build supervised data-driven models to predict user satisfaction which, despite including state-of-the art quality of experience metrics, are still far from achieving accurate results (0.62 recall of negative answers). Finally, we make our dataset publicly available, hopefully contributing in enriching and refining the scientific community knowledge on Web users’ QoE. [ABSTRACT FROM AUTHOR]

5G is the new generation of 3GPP-based cellular communications that provides remarkable connectivity capabilities and extreme network performance to mobile network operators and cellular users worldwide. The rollout process of a new capacity layer (cell) on top of the existing previous cellular technologies is a complex process that requires time and manual effort from radio planning-engineering teams and parameter optimization teams. When it comes to optimum configuration of the 5G gNB cell parameters, the maximization of achieved coverage (RSRP) and quality (SINR) of the served mobile terminals are of high importance for achieving the very high data transmission rates expected in 5G. This process strongly relies on network measurements that can be even more insightful when mobile terminal localization information is present. This information can be generated by modern algorithmic techniques that act on the cellular network signaling measurements. Configuration algorithms can then use these measurements combined with location information to optimize various cell deployment parameters such as cell azimuth. Furthermore, data-driven approaches are shown in the literature to outperform traditional, model-based algorithms as they can automate the optimization of parameters while specializing in the characteristics of each individual geographical zone. In the context of the above, in this paper, we tested the automated network reconfiguration schemes based on unsupervised learning and applied statistics for cell azimuth steering. We compared network metric clustering and geospatial clustering to be used as our baseline algorithms that are based on K-means with the proposed scheme—hybrid network and spatial clustering based on hierarchical DBSCAN. Each of these algorithms used data generated by an initial scenario to produce cell re-configuration actions and their performance was then evaluated on a validated simulation platform to capture the impact of each set of gNB reconfiguration actions. Our performance evaluation methodology was based on statistical distribution analysis for RSRP and SINR metrics for the reference scenario as well as for each reconfiguration scheme. It is shown that while both baseline algorithms improved the overall performance of the network, the proposed hybrid network–spatial scheme greatly outperformed them in all statistical criteria that were evaluated, making it a better candidate for the optimization of 5G capacity layers in modern urban environments.

We propose a monochromatic low-cost automatic sun photometer (LoCo-ASP) to perform distributed aerosol optical depth (AOD) measurements at the city scale. This kind of network could fill the gap between current automatic ground instruments—with good temporal resolution and accuracy, but few devices per city and satellite products—with global coverage, but lower temporal resolution and accuracy-. As a first approach, we consider a single equivalent wavelength around 408 nm. The cost of materials for the instrument is around 220 dollars. Moreover, we propose a calibration transfer for a pattern instrument, and estimate the uncertainties for several units and due to the internal differences and the calibration process. We achieve a max MAE of 0.026 for 38 sensors at 408 nm compared with AERONET Cimel; a mean standard deviation of 0.0062 among our entire sensor for measurement and a calibration uncertainty of 0.01. Finally, we perform city-scale measurements to show the dynamics of AOD. Our instrument can measure unsupervised, with an expected error for AOD between 0.02 and 0.03.

When network is undergoing problems such as congestion, scan attack, DDoS attack, etc, measurements are much more important than usual. In this case, traffic characteristics including available bandwidth, packet rate, and flow size distribution vary drastically, significantly degrading the performance of measurements. To address this issue, we propose the Elastic sketch. It is adaptive to currently traffic characteristics. Besides, it is generic to measurement tasks and platforms. We implement the Elastic sketch on six platforms: P4, FPGA, GPU, CPU, multi-core CPU, and OVS, to process six typical measurement tasks. Experimental results and theoretical analysis show that the Elastic sketch can adapt well to traffic characteristics. Compared to the state-of-the-art, the Elastic sketch achieves 44.6 ~ 45.2 times faster speed and 2.0 ~ 273.7 smaller error rate. [ABSTRACT FROM AUTHOR]

Finding top- $k$ elephant flows is a critical task in network traffic measurement, with many applications in congestion control, anomaly detection and traffic engineering. As the line rates keep increasing in today’s networks, designing accurate and fast algorithms for online identification of elephant flows becomes more and more challenging. The prior algorithms are seriously limited in achieving accuracy under the constraints of heavy traffic and small on-chip memory in use. We observe that the basic strategies adopted by these algorithms either require significant space overhead to measure the sizes of all flows or incur significant inaccuracy when deciding which flows to keep track of. In this paper, we adopt a new strategy, called count-with-exponential-decay, to achieve space-accuracy balance by actively removing small flows through decaying, while minimizing the impact on large flows, so as to achieve high precision in finding top- $k$ elephant flows. Moreover, the proposed algorithm called HeavyKeeper incurs small, constant processing overhead per packet and thus supports high line rates. Experimental results show that HeavyKeeper algorithm achieves 99.99% precision with a small memory size, and reduces the error by around 3 orders of magnitude on average compared to the state-of-the-art. [ABSTRACT FROM AUTHOR]

Network Traffic Monitoring and Analysis (NTMA) represents a key component for network management, especially to guarantee the correct operation of large-scale networks such as the Internet. As the complexity of Internet services and the volume of traffic continue to increase, it becomes difficult to design scalable NTMA applications. Applications such as traffic classification and policing require real-time and scalable approaches. Anomaly detection and security mechanisms require to quickly identify and react to unpredictable events while processing millions of heterogeneous events. At last, the system has to collect, store, and process massive sets of historical data for post-mortem analysis. Those are precisely the challenges faced by general big data approaches: Volume, Velocity, Variety, and Veracity. This survey brings together NTMA and big data. We catalog previous work on NTMA that adopt big data approaches to understand to what extent the potential of big data is being explored in NTMA. This survey mainly focuses on approaches and technologies to manage the big NTMA data, additionally briefly discussing big data analytics (e.g., machine learning) for the sake of NTMA. Finally, we provide guidelines for future work, discussing lessons learned, and research directions. [ABSTRACT FROM AUTHOR]

Understanding the user-perceived quality of experience (QoE) of HTTP-based video has become critical for content providers, distributors, and network operators. For network operators, monitoring QoE is challenging due to lack of access to video streaming applications, user devices, or servers. Thus, network operators need to rely on the network traffic to infer key metrics that influence video QoE. Furthermore, with content providers increasingly encrypting the network traffic, the task of QoE inference from passive measurements has become even more challenging. In this paper, we present a methodology called eMIMIC that uses passive network measurements to estimate key video QoE metrics for encrypted HTTP-based adaptive streaming (HAS) sessions. eMIMIC uses packet headers from network traffic to model an HAS session and estimate video QoE metrics, such as average bitrate and re-buffering ratio. We evaluate our methodology using network traces from a variety of realistic conditions and ground truth collected using a lab testbed for video sessions from three popular services, two video on demand (VoD) and one Live. eMIMIC estimates re-buffering ratio within 1% point of ground truth for up to 75% sessions in VoD (80% in Live) and average bitrate with error under 100 Kb/s for up to 80% sessions in VoD (70% in Live). We also compare eMIMIC with recently proposed machine learning-based QoE estimation methodology. We show that eMIMIC can predict average bitrate with 2.8%–3.2% higher accuracy and re-buffering ratio with 9.8%–24.8% higher accuracy without requiring any training on ground truth QoE metrics. Finally, we show that eMIMIC can estimate real-time QoE metrics with at least 89.6% accuracy in identifying buffer occupancy state and at least 85.7% accuracy in identifying average bitrate class of recently downloaded chunks. [ABSTRACT FROM AUTHOR]

Identifying congested links accurately to ensure the Service Level Agreements is an important but challenging task, since it is costly or even practically unfeasible to monitor massive interior links directly for large networks. Network tomography has been proposed to overcome this problem by using end-to-end (path) measurements. However, most of existing tomographic methods only focus on the loss performance degradation, while paying much less attention the fact that network congestion will also greatly worsen the delay performance. Nevertheless, most of them normally work under single-path routing, which may also get violated in today's Internet as multipath routing is increasingly common. In this paper, we consider the problem of using end-to-end measurements to identify congested links when multipath routing is employed in a non-tree network. Firstly, we use both link delay variances and link loss rates to model the system constraints between end- to-end paths and the interior links, and transfer the issue of congested link identification as an optimization problem. By theoretically demonstrating that the link delay variances are identifiable from the end-to-end delay measurements with certain topology conditions, we further prove that the above optimization problem is a Non-deterministic Polynomial-time hard (NP-hard) problem. Then in order to solve such an NP-hard problem, two greedy algorithms based on bool and additive congestion statuses are proposed. Lastly, simulation studies show that with extra delay constraints, our proposed algorithms are able to achieve better identification performances than existing methods under multipath routing. [ABSTRACT FROM AUTHOR]

Due to its huge impact on the overall quality of service (QoS) of wireless networks, both academic and industrial research have actively focused on analyzing the received signal strength in areas of particular interest. In this paper, we propose the improvement of signal-strength aggregation with a special focus on Mobile Crowdsourcing scenarios by avoiding common issues related to the mishandling of log-scaled signal values, and by the proposal of a novel aggregation method based on interpolation. Our paper presents two clear contributions. First, we discuss the misuse of log-scaled signal-strength values, which is a persistent problem within the mobile computing community. We present the physical and mathematical formalities on how signal-strength values must be handled in a scientific environment. Second, we present a solution to the difficulties of aggregating signal strength in Mobile Crowdsourcing scenarios, as a low number of measurements and nonuniformity in spatial distribution. Our proposed method obtained consistently lower Root Mean Squared Error (RMSE) values than other commonly used methods at estimating the expected value of signal strength over an area. Both contributions of this paper are important for several recent pieces of research that characterize signal strength for an area of interest.

While cellular networks connect over 3.7 billion people worldwide, their availability and quality is not uniform across regions. Under-provisioned and overloaded networks, as are common in rural or post-disaster areas, lead to poor network performance and a poor-quality user experience. To address this problem, we propose HybridCell: a system that leverages locally-owned small-scale cellular networks to augment the operation of overloaded commercial networks. HybridCell is the first system to allow a user with their existing SIM card and mobile phone to seamlessly switch between commercial and local networks in order to maintain continuous connectivity. HybridCell accomplishes this by identifying poorly-performing networks and taking action to provide seamless cellular connectivity to end users. Using traces from commercial cellular networks collected during our visit to the Za'atari refugee camp in Jordan, we demonstrate HybridCell's capability to detect and act upon commercial network overload, offering an alternate communication channel during times of congestion. We show that even in scenarios where provider networks deny calls due to overload, HybridCell is able to accommodate users and facilitate local calling. [ABSTRACT FROM AUTHOR]

In many optical network scenarios, such as Storage Array Network (SAN) replication, keeping latency under control is cornerstone to provide a proper Quality of Service (QoS). Hence, measuring latencies in such optical networks becomes fundamental. However, for low distances, microseconds resolution is required, which, in turn, demands ad-hoc hardware implementation for the measurement device. Alternatively, a more cost-effective solution is that of software-based methods, but up to date they were not precise enough at 10 Gbit/s or above. In this paper, we analyze current high-performance packet engines, such as DPDK, and pinpoint the issues involved when it comes to measure latencies in high-speed optical networks. Based on these findings, we propose the use of a software-based solution to measure latency. Furthermore, we also propose an extension that serves to measure bandwidth as well, with the novel concept of convoy of packet trains. [ABSTRACT FROM AUTHOR]