Search

Abstract Most of the intrusion detection datasets to research machine learning-based intrusion detection systems (IDSs) are devoted to cyber-only systems, and they typically collect data from one architectural layer. Often the attacks are generated in dedicated attack sessions, without reproducing the realistic alternation and overlap of normal and attack actions. We present a dataset for intrusion detection by performing penetration testing on an embedded cyber-physical system built over Robot Operating System 2 (ROS2). Features are monitored from three architectural layers: the Linux operating system, the network, and the ROS2 services. The dataset is structured as a time series and describes the expected behavior of the system and its response to ROS2-specific attacks: it repeatedly alternates periods of attack-free operation with periods when a specific attack is being performed. This allows measuring the time to detect an attacker and the number of malicious activities performed before detection. Also, it allows training an intrusion detector to minimize both, by taking advantage of the numerous alternating periods of normal and attack operations.

Deep Neural Networks (DNNs) have become an enabling technology for building accurate image classifiers, and are increasingly being applied in many ICT systems such as autonomous vehicles. Unfortunately, classifiers can be deceived by images that are altered due to failures of the visual camera, preventing the proper execution of the classification process. Therefore, it is of utmost importance to build image classifiers that can guarantee accurate classification even in the presence of such camera failures. This study crafts classifiers that are robust to failures of the visual camera by augmenting the training set with artificially altered images that simulate the effects of such failures. Such a data augmentation approach improves classification accuracy with respect to the most common data augmentation approaches, even in the absence of camera failures. To provide experimental evidence for our claims, we exercise three DNN image classifiers on three image datasets, in which we inject the effects of many failures into the visual camera. Finally, we applied eXplainable AI to debate why classifiers trained with the data augmentation approach proposed in this study can tolerate failures of the visual camera.

Failures of the vehicle camera may compromise the correct acquisition of frames, that are subsequently used by autonomous driving tasks. A clear understanding of the behavior of the autonomous driving tasks under such failure conditions, together with strategies to avoid safety is jeopardized, are indeed necessary. This study analyses and improve the performance of Traffic Sign Recognition (TSR) systems for road vehicles under the possible occurrence of camera failures. Our experimental assessment relies on three public datasets, which are commonly used for benchmarking TSR systems. We artificially inject 13 different types of camera failures into the three datasets. Then, we exploit three deep neural networks (DNNs) to classify either a single frame of a traffic sign or a sequence (i.e., a sliding window) of frames. We show that sliding windows significantly improves the robustness of the classifier against altered frames. We confirm our observations through explainable AI, which allows understanding why different classifiers have different performance in case of camera failures.

Abstract Railway systems are composed of a multitude of subsystems, sensors, and actuators that exchange datagrams through safety-critical communication protocols. However, the vast majority of these protocols rely on ad hoc interlacing mechanisms and safety codes which raise the heterogeneity and complexity of the overarching railway system. Therefore, Rete Ferroviaria Italiana, the company who is in charge of managing the Italian railway network, coordinated the definition of the Protocollo Vitale Standard (Standard Vital Protocol). This protocol is inspired to, and compliant with, the communication protocols adopted for the European Train Control System (ETCS) (SUBSET, UNISIG, 037, Euroradio FIS, version 2.3. 0; SUBSET, UNISIG, 098, RBC-RBC safe communication interface, 2007), and it is meant to become the standard layer to enable safe communication between components of the Italian railway system. This paper reports our experience in the design, implementation, verification, and validation of the Protocollo Vitale Standard in compliance with the European safety standards for railway systems. We first defined a safety plan and a verification and validation plan, which guide the design, development, verification, and validation activities as required by safety standards. Guidelines of such plans have been followed strictly until completion of the work, which concludes with the provision of a safety case where all safety evidences are summarized. Noticeably, we (i) selected appropriate safety mechanisms, (ii) verified the software design, (iii) implemented the software in compliance with code metrics and coding rules, (iv) conducted tests to validate the protocol against its functional and performance requirements, and ultimately (v) devised all relevant documentation and a safety case which summarizes the evidences needed for certification.

In the last decade, researchers, practitioners and companies struggled for devising mechanisms to detect cyber-security threats. Among others, those efforts originated rule-based, signature-based or supervised Machine Learning (ML) algorithms that were proven effective for detecting those intrusions that have already been encountered and characterized. Instead, new unknown threats, often referred to as zero-day attacks or zero-days, likely go undetected as they are often misclassified by those techniques. In recent years, unsupervised anomaly detection algorithms showed potential to detect zero-days. However, dedicated support for quantitative analyses of unsupervised anomaly detection algorithms is still scarce and often does not promote meta-learning, which has potential to improve classification performance. To such extent, this paper introduces the problem of zero-days and reviews unsupervised algorithms for their detection. Then, the paper applies a question-answer approach to identify typical issues in conducting quantitative analyses for zero-days detection, and shows how to setup and exercise unsupervised algorithms with appropriate tooling. Using a very recent attack dataset, we debate on i) the impact of features on the detection performance of unsupervised algorithms, ii) the relevant metrics to evaluate intrusion detectors, iii) means to compare multiple unsupervised algorithms, iv) the application of meta-learning to reduce misclassifications. Ultimately, v) we measure detection performance of unsupervised anomaly detection algorithms with respect to zero-days. Overall, the paper exemplifies how to practically orchestrate and apply an appropriate methodology, process and tool, providing even non-experts with means to select appropriate strategies to deal with zero-days.

Deep Neural Networks (DNNs) are the preferred choice for image-based machine learning applications in several domains. However, DNNs are vulnerable to adversarial attacks, that are carefully-crafted perturbations introduced on input images to fool a DNN model. Adversarial attacks may prevent the application of DNNs in security-critical tasks: consequently, relevant research effort is put in securing DNNs. Typical approaches either increase model robustness, or add detection capabilities in the model, or operate on the input data. Instead, in this paper we propose to detect ongoing attacks through monitoring performance indicators of the underlying Graphics Processing Unit (GPU). In fact, adversarial attacks generate images that activate neurons of DNNs in a different way than legitimate images. This also causes an alteration of GPU activities, that can be observed through software monitors and anomaly detectors. This paper presents our monitoring and detection system, and an extensive experimental analysis that includes a total of 14 adversarial attacks, 3 datasets, and 12 models. Results show that, despite limitations on the monitoring resolution, adversarial attacks can be detected in most cases, with peaks of detection accuracy above 90%.

Automatic Traffic Sign Detection and Recognition (TSDR) provides drivers with critical information on traffic signs, and it constitutes an enabling condition for autonomous driving. Misclassifying even a single sign may constitute a severe hazard, which negatively impacts the environment, infrastructures, and human lives. Therefore, a reliable TSDR mechanism is essential to attain a safe circulation of road vehicles. Traffic Sign Recognition (TSR) techniques that use Machine Learning (ML) algorithms have been proposed, but no agreement on a preferred ML algorithm nor perfect classification capabilities were always achieved by any existing solutions. Consequently, our study employs ML-based classifiers to build a TSR system that analyzes a sliding window of frames sampled by sensors on a vehicle. Such TSR processes the most recent frame and past frames sampled by sensors through (i) Long Short-Term Memory (LSTM) networks and (ii) Stacking Meta-Learners, which allow for efficiently combining base-learning classification episodes into a unified and improved meta-level classification. Experimental results by using publicly available datasets show that Stacking Meta-Learners dramatically reduce misclassifications of signs and achieved perfect classification on all three considered datasets. This shows the potential of our novel approach based on sliding windows to be used as an efficient solution for TSR.