Search

A distributed firewall is a security application that monitors and controls traffic on an organization's network. While centralized firewalls are used against attacks coming from outside a network, distributed firewalls are considered for inside attacks from internal networks such as wireless access and VPN tunnel. Distributed firewalls use policies, which are stated by rules, to find anomalous packets. However, such static rules may be incomplete. In this case, by monitoring firewall logs, the anomalies can be detected. Such logs become big when networks have high traffic, but their hidden knowledge contains valuable information about existing anomalies. In this paper, to detect the anomalies, we extract patterns from big data logs of distributed firewalls using data mining and machine learning. The proposed method is applied to big logs from distributed firewalls in a real security environment, and results are analyzed. [ABSTRACT FROM AUTHOR]

Traffic flow forecasting (TFF) is critical for constructing intelligent transportation systems and offering real-time traffic applications, and especially accurate flow forecasting based on traffic big data can drive reliable strategies for traffic management and control. To consider the weight of the influence of the spatial correlation among the road segments and capture the nonlinear characteristics of traffic flow, this paper presents a parallel Normal Distribution and Attention Mechanism Weighted Deep Bidirectional Long Short-Term Memory (NAW-DBLSTM) algorithm on Spark. Specifically, we employ the resilient distributed data set (RDD) to preprocess large-scale mobile trajectory data (e.g., taxi GPS trajectory data), and then Kalman Filter (KF) is utilized to smooth the taxi trajectory big data. Next, the parallel NAW-DBLSTM algorithm is put forward on a Spark distributed computing platform to enhance the accuracy and scalability of TFF, combined with the attention mechanism and the normal distribution, and then the time window is used for TFF. Finally, the traffic flow is forecasted successfully on Spark by our NAW-DBLSTM algorithm with the real-world GPS trajectories of taxicabs. The experimental results demonstrate that, compared with LSTM, BiLSTM, DBLSTM, DNN, SVR, KNN, SAEs, BP, CNN, GRU, and ANNs, NAW-DBLSTM can produce better performance with the MAPE value that is 85.1%, 80.1%, 85.8%, 73.1%, 78.2%, 77.9%, 78.8%, 84.6%, 96.4%, 86.2%, and 73.2% lower than that of comparable algorithms, respectively. Particularly, the MAPE value of NAW-DBLSTM is 28.3%, 20.1%, 71.1%, and 79.1% lower than that of LSTM weighted with the normal distribution and the attention mechanism (NAW-LSTM), BiLSTM weighted with the normal distribution and the attention mechanism (NAW-BiLSTM), DBLSTM weighted with the normal distribution (NW-DBLSTM), and NAW-DBLSTM weighted without the time window (NT-NAW-DBLSTM), respectively. [ABSTRACT FROM AUTHOR]

We propose a stochastic approximation (SA) based method with randomization of samples for policy evaluation using the least squares temporal difference (LSTD) algorithm. Our proposed scheme is equivalent to running regular temporal difference learning with linear function approximation, albeit with samples picked uniformly from a given dataset. Our method results in an O(d) improvement in complexity in comparison to LSTD, where d is the dimension of the data. We provide non-asymptotic bounds for our proposed method, both in high probability and in expectation, under the assumption that the matrix underlying the LSTD solution is positive definite. The latter assumption can be easily satisfied for the pathwise LSTD variant proposed by Lazaric (J Mach Learn Res 13:3041–3074, 2012). Moreover, we also establish that using our method in place of LSTD does not impact the rate of convergence of the approximate value function to the true value function. These rate results coupled with the low computational complexity of our method make it attractive for implementation in big data settings, where d is large. A similar low-complexity alternative for least squares regression is well-known as the stochastic gradient descent (SGD) algorithm. We provide finite-time bounds for SGD. We demonstrate the practicality of our method as an efficient alternative for pathwise LSTD empirically by combining it with the least squares policy iteration algorithm in a traffic signal control application. We also conduct another set of experiments that combines the SA-based low-complexity variant for least squares regression with the LinUCB algorithm for contextual bandits, using the large scale news recommendation dataset from Yahoo. [ABSTRACT FROM AUTHOR]

Intelligent transportation brings huge benefits to humans' life and Industrial production in terms of vehicle control and traffic management. Now, the development of edge-cloud computing has once again promoted intelligent transportation into a new era. However, the development of intelligent transportation inevitably produces a large amount of data, which brings new challenges to data privacy protection and security. In this paper, we propose to develop an improved trajectory prediction framework based on the self-adaptive trajectory prediction model (SATP), which could significantly enhance traffic safety in transportation systems. The proposed framework is capable of guaranteeing the accurate trajectory prediction of moving target under different application scenarios. In particular, to reduce the size of original trajectory point data collected by sensors, the angle change and minimum description length (MDL) principle are first combined to remove the redundant points in raw trajectories. The obtained points can then be reduced for model using the two-step clustering method. To further enhance the prediction performance, we add the "self-transfer" to the original model to solve the problems that the state of original SATP model may be discontinuous. Furthermore, we propose to develop a trajectory complementation method based on Bezier curve to improve the prediction accuracy. Finally, by comparing the two-step clustering method with the commonly-used SinglePass and density-based clustering method (DBCM) algorithms, the proposed two-step clustering policy greatly reduce the time cost of clustering. At the same time, by comparing the improved SATP model with the original model, the results show that the improved SATP method can greatly improve the speed of prediction model. [ABSTRACT FROM AUTHOR]

With the progress of society and the acceleration of urbanization, the problem of multimedia assisted urban traffic is becoming increasingly apparent. Intelligent transportation system arises at the historic moment, but the road traffic information data accumulation of intelligent transportation system is quite large, and the information analysis is complex. Usually, there are huge amounts of data stored in the database of the traffic system, thus we need to analyze and manage the data with scientific methods. As the visualized tools, the multimedia based analysis for the on-time data conditions should also be integrated, therefore, how to effectively model the scenario is challenging. In this paper, we introduce and analyze the structure and related model of recurrent neural network, and apply RNN to traffic big data mining model. According to the characteristics of traffic flow, this paper analyzes the causes of the error data in the process of traffic data collection, and puts forward the corresponding processing methods. The proposed model uses the TensorFlow for development, and applies the proposed deep learning model to implement traffic data mining, and we use charts to visually show the prediction results. Experimental results show that proposed algorithm can effectively mine large traffic multimedia data and has good robustness. At the same time, the prediction accuracy has reached 97.36%. [ABSTRACT FROM AUTHOR]

As the development of crowdsourcing technique, acquiring amounts of data in urban cities becomes possible and reliable, which makes it possible to mine useful and significant information from data. Traffic anomaly detection is to find the traffic patterns which are not expected and it can be used to explore traffic problems accurately and efficiently. In this paper, we propose LoTAD to explore anomalous regions with long-term poor traffic situations. Specifically, we process crowdsourced bus data into TS-segments (Temporal and Spatial segments) to model the traffic condition. Later, we explore anomalous TS-segments in each bus line by calculating their AI (Anomaly Index). Then, we combine anomalous TS-segments detected in different lines to mine anomalous regions. The information of anomalous regions provides suggestions for future traffic planning. We conduct experiments with real crowdsourced bus trajectory datasets of October in 2014 and March in 2015 in Hangzhou. We analyze the varieties of the results and explain how they are consistent with the real urban traffic planning or social events happened between the time interval of the two datasets. At last we do a contrast experiment with the most ten congested roads in Hangzhou, which verifies the effectiveness of LoTAD. [ABSTRACT FROM AUTHOR]

Intelligent Traffic Systems (ITS), as integrated systems including control technologies, communication technologies, vehicle sensing and vehicle electronic technologies, have provided valuable solutions to the increasingly serious traffic problems. In the process of construction and operation of ITS, big data, especially multimedia data is produced at a rapid speed, which has made traffic information more and more complicated, causing traffic management facing new challenges. Hence, in order to achieve efficient management of all types of transportation resources and make better use of ITS, it is necessary and significant to study the architecture and performance of ITS in depth. Through dividing the system into different functional modules and assigning these modules to components in Performance Evaluation Process Algebra (PEPA), we can adopt a new method to realize the modeling and evaluating the working process of real-time positioning and route planning in ITS. Meanwhile, the fluid flow approximation is employed to conduct a performance analysis through PEPA models, guaranteeing that the response time, the maximum utilization and the throughput of the system can be achieved and analyzed. [ABSTRACT FROM AUTHOR]

Advancement in technology causes the rise in smart systems. City authorities want to make their cities smarter by making an intelligent decision at real time without the involvement of humans. Monitoring and controlling city traffic is one of the major challenges faced by the authorities. These days, city traffic is monitored by static network cameras deployed on few places of highways. Most of the vehicles are also equipped with cameras to store the videos as a black box. However, monitoring and controlling city traffic by using these thousands of cameras produce an overwhelming amount of high-speed videos, which is challenging to process at real time. Therefore, in this paper, we proposed a system to control city traffic by identifying illegal traffic behaviour, such as illegal U-turn, through continuous monitoring of city traffic. The continuous city traffic is monitored by the network static cameras placed on the road as well as by all the vehicles' cameras. An architecture is proposed to handle high-speed vast volume of real-time videos efficiently. For that, the two-level parallelism is achieved with the combination of Hadoop and graphics processing unit (GPU) while processing each frame using parallel environment of Hadoop and each block of a frame using GPU. MapReduce Hadoop programming paradigm is not suitable for real-time processing. Thus, we proposed a parameter calculation algorithm that is equivalent to MapReduce mechanism for image processing while dividing the images/frames into fixed-size blocks. We analyzed the city road traffic, which is collected by static cameras placed on various roads and also by vehicles' cameras while running on the road. Later, the illegal traffic behaviour are recognized, e.g. illegal U-turn, drunken drive, zig-zag drive, over-speed, etc. Finally, the efficiency of the designed system and algorithms are tested by considering the overall running time and system's throughput with respect to video duration as well as the number of frames. The findings indicate that the proposed architecture with GPU-based algorithm over the Hadoop system perform extraordinary. [ABSTRACT FROM AUTHOR]