Search

Deep learning (DL)-based radiofrequency (RF) fingerprinting identification has shown significantly growing importance in the wireless industry including 5G, IoT and Wireless Sensor Networks (WSN). However, there is little investigation on the robustness of the corresponding DL models against adversarial attacks and adversarial defense. This paper discusses the effects of four cutting-edge adversarial attacks on DL-based RF fingerprinting identification and provides a graphical analysis of RF feature perturbations following adversarial attacks. For the first time we also demonstrate the results of combined physical attack (i.e. tuning the drain currents of the power amplifiers) by tuning the physical features of the device hardware (i.e. drain current), and adversarial attacks on DL-based classifiers. Experimental results from 16 Gallium nitride (GaN) power amplifiers (PA) reveal that adversarial attacks can seriously deteriorate the accuracy of RF identification from about 100.00% to below 10.00% even with only 0.50% adversarial perturbation. In order to deal with the problem of decreased accuracy caused by attacks, we proposed an adversarial defense method based on feature-level interpretability (FIAT), which improved the accuracy of RF identification with interpretability analysis. The results compared with several common defense methods validate that the proposed FIAT can maintain an accuracy of 96.00% even when the adversarial attack perturbation is 5.00% and with physical attack. Moreover, the changes of data features in the process of adversarial attacks and defense are given, providing a new way for the interpretability of adversarial attacks and defense on RF identification tasks.

In this work, we deploy a one-day-ahead prediction algorithm using a deep neural network for a fast-response BESS in an intelligent energy management system (I-EMS) that is called SIEMS. The main role of the SIEMS is to maintain the state of charge at high rates based on the one-day-ahead information about solar power, which depends on meteorological conditions. The remaining power is supplied by the main grid for sustained power streaming between BESS and end-users. Considering the usage of information and communication technology components in the microgrids, the main objective of this paper is focused on the hybrid microgrid performance under cyber-physical security adversarial attacks. Fast gradient sign, basic iterative, and DeepFool methods, which are investigated for the first time in power systems e.g. smart grid and microgrids, in order to produce perturbation for training data.

In recent years, various domains have been influenced by the rapid growth of machine learning. Autonomous driving is an area that has tremendously developed in parallel with the advancement of machine learning. In autonomous vehicles, various machine learning components are used such as traffic lights recognition, traffic sign recognition, limiting speed and pathfinding. For most of these components, computer vision technologies with deep learning such as object detection, semantic segmentation and image classification are used. However, these machine learning models are vulnerable to targeted tensor perturbations called adversarial attacks, which limit the performance of the applications. Therefore, implementing defense models against adversarial attacks has become an increasingly critical research area. The paper aims at summarising the latest adversarial attacks and defense models introduced in the field of autonomous driving with machine learning technologies up until mid-2021.

[EN] Over the last few years, Software Defined Networking (SDN) paradigm has become an emerging architecture to design future networks and to meet new application demands. SDN provides resources for improving network control and management by separating control and data plane, and the logical control is centralized in a controller. However, the centralized control logic can be an ideal target for malicious attacks, mainly Distributed Denial of Service (DDoS) attacks. Recently, Deep Learning has become a powerful technique applied in cybersecurity, and many Network Intrusion Detection (NIDS) have been proposed in recent researches. Some studies have indicated that deep neural networks are sensitive in detecting adversarial attacks. Adversarial attacks are instances with certain perturbations that cause deep neural networks to misclassify. In this paper, we proposed a detection and defense system based on Adversarial training in SDN, which uses Generative Adversarial Network (GAN) framework for detecting DDoS attacks and applies adversarial training to make the system less sensitive to adversarial attacks. The proposed system includes well-defined modules that enable continuous traffic monitoring using IP flow analysis, enabling the anomaly detection system to act in near-real-time. We conducted the experiments on two distinct scenarios, with emulated data and the public dataset CICDDoS 2019. Experimental results demonstrated that the system efficiently detected up-to-date common types of DDoS attacks compared to other approaches., This work has been partially supported by the National Council for Scientific and Technological Development (CNPq) of Brazil under Grant of Project 310668/2019-0 and by SETI, Brazil/Fundacao Araucaria due to the concession of scholarships; by the "Ministerio de Economia y Competitividad, Spain"in the "Programa Estatal de Fomento de la Investigacion Cientifica y Tecnica de Excelencia, Subprograma Estatal de Generacion de Conocimiento"within the project under Grant TIN2017-84802-C2-1-P.

International audience; Deep Neural Networks (DNNs) have been deployed in a wide range of applications, including safety-critical domains, owing to their proven efficiency in solving complex problems. However, these systems have been shown vulnerable to adversarial attacks: carefully crafted perturbations that threaten their integrity and trustworthiness. Several defenses have been recently proposed. However, most of these techniques are costly to deploy since they require retraining and specific fine-tuning procedures. While there are pre-processing defenses that do not require retraining, these were shown to be ineffective against adaptive white-box attacks. In this paper, we propose a model-agnostic defense against adversarial attacks using stochastic pre-processing. Based on a process of down-sampling/up-sampling, we transform the input to a new sample that is: (i) close enough to the initial input to be classified correctly, and (ii) different enough to ignore any potential adversarial noise within it. The proposed defense is generic, easy to deploy and does not require any specific training or fine tuning. We tested our technique comparatively to state-of-the-art defenses under grey-box and strong white-box scenarios. Experimental results show that our defense achieves robustness of up to 94% and 93% against PGD and Cand#x0026;W attacks, respectively, under strong white-box scenario. IEEE

Funding Information: This work is supported in part by the National Natural Science Foundation of China under Grant 62072351 ; in part by the Academy of Finland under Grant 308087 , Grant 335262 , Grant 345072 and Grant 350464 ; in part by the Open research project of ZheJiang Lab under grant 2021PD0AB01 ; in part by the Shaanxi Innovation Team Project under Grant 2018 TD-007; and in part by the 111 Project, China under Grant B16037 . Publisher Copyright: © 2022 Elsevier B.V. Speaker recognition has become very popular in many application scenarios, such as smart homes and smart assistants, due to ease of use for remote control and economic-friendly features. The rapid development of SRSs is inseparable from the advancement of machine learning, especially neural networks. However, previous work has shown that machine learning models are vulnerable to adversarial attacks in the image domain, which inspired researchers to explore adversarial attacks and defenses in Speaker Recognition Systems (SRS). Unfortunately, existing literature lacks a thorough review of this topic. In this paper, we fill this gap by performing a comprehensive survey on adversarial attacks and defenses in SRSs. We first introduce the basics of SRSs and concepts related to adversarial attacks. Then, we propose two sets of criteria to evaluate the performance of attack methods and defense methods in SRSs, respectively. After that, we provide taxonomies of existing attack methods and defense methods, and further review them by employing our proposed criteria. Finally, based on our review, we find some open issues and further specify a number of future directions to motivate the research of SRSs security.

Convolutional neural networks (CNNs) have become one of the most important tools for image classification. However, many models are susceptible to adversarial attacks, and CNNs can perform misclassifications. In previous works, we successfully developed an EA-based black-box attack that creates adversarial images for the target scenario that fulfils two criteria. The CNN should classify the adversarial image in the target category with a confidence ≥ 0.95, and a human should not notice any difference between the adversarial and original images. Thanks to extensive experiments performed with the CNN ${\mathcal{C}}$ = VGG-16 trained on the CIFAR-10 dataset to classify images according to 10 categories, this paper, which substantially enhances most aspects of Chitic et al. (2021), addresses four issues. (1) From a pure EA point of view, we highlight the conceptual originality of our algorithm $\text {EA}_{d}^{\text {target}, {\mathcal{C}}}$ , versus the classical EA approach. The competitive advantage obtained was assessed experimentally during image classification. (2) We then measured the intrinsic performance of the EA-based attack for an extensive series of ancestor images. (3) We challenged the filter resistance of the adversarial images created by the EA for five well-known filters. (4) We proceed to the creation of natively filter-resistant adversarial images that can fool humans, CNNs, and CNNs composed with filters.

International audience; The stability of neural networks with respect to adversarial perturbations has been extensively studied. One of the main strategies consist of quantifying the Lipschitz regularity of neural networks. In this paper, we introduce a multivariate Lipschitz constant-based stability analysis of fully connected neural networks allowing us to capture the influence of each input or group of inputs on the neural network stability. Our approach relies on a suitable re-normalization of the input space, with the objective to perform a more precise analysis than the one provided by a global Lipschitz constant. We investigate the mathematical properties of the proposed multivariate Lipschitz analysis and show its usefulness in better understanding the sensitivity of the neural network with regard to groups of inputs. We display the results of this analysis by a new representation designed for machine learning practitioners and safety engineers termed as a Lipschitz star. The Lipschitz star is a graphical and practical tool to analyze the sensitivity of a neural network model during its development, with regard to different combinations of inputs. By leveraging this tool, we show that it is possible to build robust-by-design models using spectral normalization techniques for controlling the stability of a neural network, given a safety Lipschitz target. Thanks to our multivariate Lipschitz analysis, we can also measure the efficiency of adversarial training in inference tasks. We perform experiments on various open access tabular datasets, and also on a real Thales Air Mobility industrial application subject to certification requirements.

Deep neural networks are vulnerable to adversarial samples, posing potential threats to the applications deployed with deep learning models in practical conditions. A typical example is the fingerprint liveness detection module in fingerprint authentication systems. Inspired by great progress of deep learning, deep networks-based fingerprint liveness detection algorithms spring up and dominate the field. Thus, we investigate the feasibility of deceiving state-of-the-art deep networks-based fingerprint liveness detection schemes by leveraging this property in this paper. Extensive evaluations are made with three existing adversarial methods: FGSM, MI-FGSM, and Deepfool. We also proposed an adversarial attack method that enhances the robustness of adversarial fingerprint images to various transformations like rotations and flip. We demonstrate these outstanding schemes are likely to classify fake fingerprints as live fingerprints by adding tiny perturbations, even without internal details of their used model. The experimental results reveal a big loophole and threats for these schemes from a view of security, and enough attention is urgently needed to be paid on anti-adversarial not only in fingerprint liveness detection but also in all deep learning applications.

Face recognition systems are used in high security applications for identification, authentication and authorization. Therefore they need to be robust not only to people wearing face accessories and masks like in the COVID19 pandemic, they also need to be robust against adversarial attacks. We have identified three inconspicuous facial areas to wear adversarial examples to attack face recognition. These are the mouth-nose section, the forehead and the eye area. In this paper, we will address the question of how much of a face needs to be present for successful identification and whether removing the critical regions is a viable countermeasure against adversarial examples.

Existing literature on adversarial Machine Learning (ML) focuses either on showing attacks that break every ML model, or defenses that withstand most attacks. Unfortunately, little consideration is given to the actual \textit{cost} of the attack or the defense. Moreover, adversarial samples are often crafted in the "feature-space", making the corresponding evaluations of questionable value. Simply put, the current situation does not allow to estimate the actual threat posed by adversarial attacks, leading to a lack of secure ML systems. We aim to clarify such confusion in this paper. By considering the application of ML for Phishing Website Detection (PWD), we formalize the "evasion-space" in which an adversarial perturbation can be introduced to fool a ML-PWD -- demonstrating that even perturbations in the "feature-space" are useful. Then, we propose a realistic threat model describing evasion attacks against ML-PWD that are cheap to stage, and hence intrinsically more attractive for real phishers. Finally, we perform the first statistically validated assessment of state-of-the-art ML-PWD against 12 evasion attacks. Our evaluation shows (i) the true efficacy of evasion attempts that are more likely to occur; and (ii) the impact of perturbations crafted in different evasion-spaces. Our realistic evasion attempts induce a statistically significant degradation (3-10% at $p\!

Cyber-physical systems, built from the integration of cyber and physical components, are being used in multiple domains ranging from manufacturing and healthcare to traffic con- trol and safety. Ensuring the security of cyber-physical systems is crucial because they provide the foundation of the critical infrastructure, and security incidents can result in catastrophic failures. Recent publications report that machine learning models are vul- nerable to adversarial examples, crafted by adding small perturbations to input data. For the past decade, machine learning security has become a growing interest area, with a significant number of systematic reviews and surveys that have been published. Secu- rity of artificial intelligence in cyber-physical systems is more challenging in comparison to machine learning security, because adversaries have a wider possible attack surface, in both cyber and physical domains. However, comprehensive systematic literature re- views in this research field are not available. Therefore, this work presents a systematic literature review of the adversarial attacks on artificial intelligence in cyber-physical sys- tems, examining 45 scientific papers, selected from 134 publications found in the Scopus database. It provides the classification of attack algorithms and defense methods, the sur- vey of evaluation metrics, an overview of the state of the art in methodologies and tools, and, as the main contribution, identifies open problems and research gaps and highlights future research challenges in this area of interest.

This paper outlines intertwined challenges related to three areas: The hype surrounding AI, the consequences of corporate influence on the AI research agenda, and the public sector's uncritical embracement of AI technologies. We argue that AI can backfire if overconfident predictions influence decisions to introduce AI in high-risk domains. Moreover, the corporate colonization of the AI research agenda may cause a decline in societal trust in science, which is highly problematic considering that AI will increasingly power important domains in society.

Regardless of revolutionizing improvements in various domains thanks to recent advancements in the field of Deep Learning (DL), recent studies have demonstrated that DL networks are susceptible to adversarial attacks. Such attacks are crucial in sensitive environments to make critical and life-changing decisions, such as health decision-making. Research efforts on using textual adversaries to attack DL for natural language processing (NLP) have received increasing attention in recent years. Among the available textual adversarial studies, Electronic Health Records (EHR) have gained the least attention. This paper investigates the effectiveness of adversarial attacks on clinical document classification and proposes a defense mechanism to develop a robust convolutional neural network (CNN) model and counteract these attacks. Specifically, we apply various black-box attacks based on concatenation and editing adversaries on unstructured clinical text. Then, we propose a defense technique based on feature selection and filtering to improve the robustness of the models. Experimental results show that a small perturbation to the unstructured text in clinical documents causes a significant drop in performance. Performing the proposed defense mechanism under the same adversarial attacks, on the other hand, avoids such a drop in performance. Therefore, it enhances the robustness of the CNN model for clinical document classification.

International audience; Deep neural networks (DNNs) are shown to be vulnerable to adversarial attacks-- carefully crafted additive noise that undermines DNNs integrity. Previously proposed defenses against these attacks require substantial overheads, making it challenging to deploy these solutions in power and computational resource-constrained devices, such as embedded systems and the Edge. In this paper, we explore the use of voltage overscaling (VOS) as a lightweight defense against adversarial attacks. Specifically, we exploit the stochastic timing violations of VOS to implement a moving-target defense for DNNs. Our experimental results demonstrate that VOS guarantees effective defense against different attack methods, does not require any software/hardware modifications, and offers a by-product reduction in power consumption.

Despite deep neural networks (DNNs) having achieved impressive performance in various domains, it has been revealed that DNNs are vulnerable in the face of adversarial examples, which are maliciously crafted by adding human-imperceptible perturbations to an original sample to cause the wrong output by the DNNs. Encouraged by numerous researches on adversarial examples for computer vision, there has been growing interest in designing adversarial attacks for Natural Language Processing (NLP) tasks. However, the adversarial attacking for NLP is challenging because text is discrete data and a small perturbation can bring a notable shift to the original input. In this paper, we propose a novel method, based on conditional BERT sampling with multiple standards, for generating universal adversarial perturbations: input-agnostic of words that can be concatenated to any input in order to produce a specific prediction. Our universal adversarial attack can create an appearance closer to natural phrases and yet fool sentiment classifiers when added to benign inputs. Based on automatic detection metrics and human evaluations, the adversarial attack we developed dramatically reduces the accuracy of the model on classification tasks, and the trigger is less easily distinguished from natural text. Experimental results demonstrate that our method crafts more high-quality adversarial examples as compared to baseline methods. Further experiments show that our method has high transferability. Our goal is to prove that adversarial attacks are more difficult to detect than previously thought and enable appropriate defenses.

Adversarial attacks are considered a potentially serious security threat for machine learning systems. Medical image analysis (MedIA) systems have recently been argued to be vulnerable to adversarial attacks due to strong financial incentives and the associated technological infrastructure. In this paper, we study previously unexplored factors affecting adversarial attack vulnerability of deep learning MedIA systems in three medical domains: ophthalmology, radiology, and pathology. We focus on adversarial black-box settings, in which the attacker does not have full access to the target model and usually uses another model, commonly referred to as surrogate model, to craft adversarial examples. We consider this to be the most realistic scenario for MedIA systems. Firstly, we study the effect of weight initialization (ImageNet vs. random) on the transferability of adversarial attacks from the surrogate model to the target model. Secondly, we study the influence of differences in development data between target and surrogate models. We further study the interaction of weight initialization and data differences with differences in model architecture. All experiments were done with a perturbation degree tuned to ensure maximal transferability at minimal visual perceptibility of the attacks. Our experiments show that pre-training may dramatically increase the transferability of adversarial examples, even when the target and surrogate's architectures are different: the larger the performance gain using pre-training, the larger the transferability. Differences in the development data between target and surrogate models considerably decrease the performance of the attack; this decrease is further amplified by difference in the model architecture. We believe these factors should be considered when developing security-critical MedIA systems planned to be deployed in clinical practice., Comment: First three authors contributed equally

While machine learning applications are getting mainstream owing to a demonstrated efficiency in solving complex problems, they suffer from inherent vulnerability to adversarial attacks. Adversarial attacks consist of additive noise to an input which can fool a detector. Recently, successful real-world printable adversarial patches were proven efficient against state-of-the-art neural networks. In the transition from digital noise based attacks to real-world physical attacks, the myriad of factors affecting object detection will also affect adversarial patches. Among these factors, view angle is one of the most influential, yet under-explored. In this paper, we study the effect of view angle on the effectiveness of an adversarial patch. To this aim, we propose the first approach that considers a multi-view context by combining existing adversarial patches with a perspective geometric transformation in order to simulate the effect of view angle changes. Our approach has been evaluated on two datasets: the first dataset which contains most real world constraints of a multi-view context, and the second dataset which empirically isolates the effect of view angle. The experiments show that view angle significantly affects the performance of adversarial patches, where in some cases the patch loses most of its effectiveness. We believe that these results motivate taking into account the effect of view angles in future adversarial attacks, and open up new opportunities for adversarial defenses., To appear in the 20th CyberWorlds Conference

Spiking Neural Networks (SNNs) aim at providing energy-efficient learning capabilities when implemented on neuromorphic chips with event-based Dynamic Vision Sensors (DVS). This paper studies the robustness of SNNs against adversarial attacks on such DVS-based systems, and proposes R-SNN, a novel methodology for robustifying SNNs through efficient DVS-noise filtering. We are the first to generate adversarial attacks on DVS signals (i.e., frames of events in the spatio-temporal domain) and to apply noise filters for DVS sensors in the quest for defending against adversarial attacks. Our results show that the noise filters effectively prevent the SNNs from being fooled. The SNNs in our experiments provide more than 90% accuracy on the DVS-Gesture and NMNIST datasets under different adversarial threat models., To appear at the 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021). arXiv admin note: text overlap with arXiv:2107.00415

Despite substantial advances in network architecture performance, the susceptibility of adversarial attacks makes deep learning challenging to implement in safety-critical applications. This paper proposes a data-centric approach to addressing this problem. A nonlocal denoising method with different luminance values has been used to generate adversarial examples from the Modified National Institute of Standards and Technology database (MNIST) and Canadian Institute for Advanced Research (CIFAR-10) data sets. Under perturbation, the method provided absolute accuracy improvements of up to 9.3% in the MNIST data set and 13% in the CIFAR-10 data set. Training using transformed images with higher luminance values increases the robustness of the classifier. We have shown that transfer learning is disadvantageous for adversarial machine learning. The results indicate that simple adversarial examples can improve resilience and make deep learning easier to apply in various applications.

Nowadays, Deep Neural Networks (DNNs) report state-of-the-art results in many machine learning areas, including intrusion detection. Nevertheless, recent studies in computer vision have shown that DNNs can be vulnerable to adversarial attacks that are capable of deceiving them into misclassification by injecting specially crafted data. In security-critical areas, such attacks can cause serious damage; therefore, in this paper, we examine the effect of adversarial attacks on deep learning-based intrusion detection. In addition, we investigate the effectiveness of adversarial training as a defense against such attacks. Experimental results show that with sufficient distortion, adversarial examples are able to mislead the detector and that the use of adversarial training can improve the robustness of intrusion detection., Already published in The Sixteenth International Conference on Systems (ICONS 2021)

Machine learning models are known to be vulnerable to adversarial attacks, namely perturbations of the data that lead to wrong predictions despite being imperceptible. However, the existence of "universal" attacks (i.e., unique perturbations that transfer across different data points) has only been demonstrated for images to date. Part of the reason lies in the lack of a common domain, for geometric data such as graphs, meshes, and point clouds, where a universal perturbation can be defined. In this paper, we offer a change in perspective and demonstrate the existence of universal attacks for geometric data (shapes). We introduce a computational procedure that operates entirely in the spectral domain, where the attacks take the form of small perturbations to short eigenvalue sequences; the resulting geometry is then synthesized via shape-from-spectrum recovery. Our attacks are universal, in that they transfer across different shapes, different representations (meshes and point clouds), and generalize to previously unseen data., Published at CVPR 2021

This paper investigates the theory of robustness against adversarial attacks. We focus on randomized classifiers (i.e. classifiers that output random variables) and provide a thorough analysis of their behavior through the lens of statistical learning theory and information theory. To this aim, we introduce a new notion of robustness for randomized classifiers, enforcing local Lipschitzness using probability metrics. Equipped with this definition, we make two new contributions. The first one consists in devising a new upper bound on the adversarial generalization gap of randomized classifiers. More precisely, we devise bounds on the generalization gap and the adversarial gap i.e. the gap between the risk and the worst-case risk under attack) of randomized classifiers. The second contribution presents a yet simple but efficient noise injection method to design robust randomized classifiers. We show that our results are applicable to a wide range of machine learning models under mild hypotheses. We further corroborate our findings with experimental results using deep neural networks on standard image datasets, namely CIFAR-10 and CIFAR-100. On these tasks, we manage to design robust models that simultaneously achieve state-of-the-art accuracy (over 0.82 clean accuracy on CIFAR-10) and enjoy guaranteed robust accuracy bounds (0.45 against $$\ell _{2}$$ ℓ 2 adversaries with magnitude 0.5 on CIFAR-10).

Transfer learning using pre-trained deep neural networks (DNNs) has been widely used for plant disease identification recently. However, pre-trained DNNs are susceptible to adversarial attacks which generate adversarial samples causing DNN models to make wrong predictions. Successful adversarial attacks on deep learning (DL)-based plant disease identification systems could result in a significant delay of treatments and huge economic losses. This paper is the first attempt to study adversarial attacks and detection on DL-based plant disease identification. Our results show that adversarial attacks with a small number of perturbations can dramatically degrade the performance of DNN models for plant disease identification. We also find that adversarial attacks can be effectively defended by using adversarial sample detection with an appropriate choice of features. Our work will serve as a basis for developing more robust DNN models for plant disease identification and guiding the defense against adversarial attacks.

Deep learned models are now largely adopted in different fields, and they generally provide superior performances with respect to classical signal-based approaches. Notwithstanding this, their actual reliability when working in an unprotected environment is far enough to be proven. In this work, we consider a novel deep neural network architecture, named Neural Ordinary Differential Equations (N-ODE), that is getting particular attention due to an attractive property--a test-time tunable trade-off between accuracy and efficiency. This paper analyzes the robustness of N-ODE image classifiers when faced against a strong adversarial attack and how its effectiveness changes when varying such a tunable trade-off. We show that adversarial robustness is increased when the networks operate in different tolerance regimes during test time and training time. On this basis, we propose a novel adversarial detection strategy for N-ODE nets based on the randomization of the adaptive ODE solver tolerance. Our evaluation performed on standard image classification benchmarks shows that our detection technique provides high rejection of adversarial examples while maintaining most of the original samples under white-box attacks and zero-knowledge adversaries.

With great power comes great responsibility. The success of machine learning, especially deep learning, in research and practice has attracted a great deal of interest, which in turn necessitates increased trust. Sources of mistrust include matters of model genesis ("Is this really the appropriate model?") and interpretability ("Why did the model come to this conclusion?", "Is the model safe from being easily fooled by adversaries?"). In this paper, two partners for the trustworthiness tango are presented: recent advances and ideas, as well as practical applications in industry in (a) Automated machine learning (AutoML), a powerful tool to optimize deep neural network architectures and netune hyperparameters, which promises to build models in a safer and more comprehensive way; (b) Interpretability of neural network outputs, which addresses the vital question regarding the reasoning behind model predictions and provides insights to improve robustness against adversarial attacks.

International audience; For the past few years, deep learning (DL) robustness (i.e. the ability to maintain the same decision when inputs are subject to perturbations) has become a question of paramount importance, in particular in settings where misclassification can have dramatic consequences. To address this question, authors have proposed different approaches, such as adding regularizers or training using noisy examples. In this paper we introduce a regularizer based on the Laplacian of similarity graphs obtained from the representation of training data at each layer of the DL architecture. This regularizer penalizes large changes (across consecutive layers in the architecture) in the distance between examples of different classes, and as such enforces smooth variations of the class boundaries. We provide theoretical justification for this regularizer and demonstrate its effectiveness to improve robustness on classical supervised learning vision datasets for various types of perturbations. We also show it can be combined with existing methods to increase overall robustness.

Today, Machine Learning is being rolled out in a variety of areas. It is a promising field that can offer several assets and can revolutionize several aspects of technology. Nevertheless, despite the advantages of machine learning technologies, learning algorithms can be exploited by attackers to carry out illicit activities. Therefore, the field of security of machine learning is deriving attention in these times so as to meet this challenge and develop secure learning models. In this paper, we overview a taxonomy that will help us understand and analyze the security of machine learning models. In the next sections, we conduct a comparative study of most widespread adversarial attacks then, we analyze common methods that were advanced to protect systems built on Machine learning models from adversaries. Finally, we discuss a proposition of a pattern designed to ensure a security assessment of machine learning models.

Deep Learning (DL) algorithms have gained popularity owing to their practical problem-solving capacity. However, they suffer from a serious integrity threat, i.e., their vulnerability to adversarial attacks. In the quest for DL trustworthiness, recent works claimed the inherent robustness of Spiking Neural Networks (SNNs) to these attacks, without considering the variability in their structural spiking parameters. This paper explores the security enhancement of SNNs through internal structural parameters. Specifically, we investigate the SNNs robustness to adversarial attacks with different values of the neuron's firing voltage thresholds and time window boundaries. We thoroughly study SNNs security under different adversarial attacks in the strong white-box setting, with different noise budgets and under variable spiking parameters. Our results show a significant impact of the structural parameters on the SNNs' security, and promising sweet spots can be reached to design trustworthy SNNs with 85% higher robustness than a traditional non-spiking DL system. To the best of our knowledge, this is the first work that investigates the impact of structural parameters on SNNs robustness to adversarial attacks. The proposed contributions and the experimental framework is available online to the community for reproducible research., Accepted for publication at the 24th Design, Automation and Test in Europe (DATE'21), February, 2021

Neural networks are increasingly important in the developmentof Network Intrusion Detection Systems (NIDS), as they have the potential to achieve high detection accuracy while requiring limited feature engineering. Deep learning-based detectors can be however vulnerable to adversarial examples, by which attackers that maybe oblivious to the precise mechanics of the targeted NIDS add subtle perturbations to malicious traffic features, with the aim of evading detection and disrupting critical systems in a cost-effective manner. Defending against such adversarial attacks is therefore of high importance, but requires to address daunting challenges.In this paper, we introduce Tiki-Taka, a general framework for (i) assessing the robustness of state-of-the-art deep learning-based NIDS against adversarial manipulations, and which (ii) incorporates our proposed defense mechanisms to increase the NIDS’ resistance to attacks employing such evasion techniques. Specifically, we select five different cutting-edge adversarial attack mechanisms to subvert three popular malicious traffic detectors that employ neural networks. We experiment with a publicly available dataset and consider both one-to-all and one-to-one classification scenarios, i.e., discriminating illicit vs benign traffic and respectively identifying specific types of anomalous traffic among many observed. The results obtained reveal that, under realistic constraints, attackers can evade NIDS with up to 35.7% success rates, by only altering time-based features of the traffic generated. To counteract these weaknesses, we propose three defense mechanisms, namely: model voting ensembling, ensembling adversarial training, and query detection. To the best of our knowledge, our work is the first to propose defenses against adversarial attacks targeting NIDS. We demonstrate that when employing the proposed methods, intrusion detection rates can be improved to nearly 100% against most types of malicious traffic, and attacks with potentially catastrophic consequences (e.g., botnet) can be thwarted. This confirms the effectiveness of our solutions and makes the case for their adoption when designing robust and reliable deep anomaly detectors

There is an increasing demand for automated spoken language assessment (SLA) systems, partly driven by the performance improvements that have come from deep learning based approaches. One aspect of deep learning systems is that they do not require expert derived features, operating directly on the original signal such as a speech recognition (ASR) transcript. This, however, increases their potential susceptibility to adversarial attacks as a form of candidate malpractice. In this paper the sensitivity of SLA systems to a universal black-box attack on the ASR text output is explored. The aim is to obtain a single, universal phrase to maximally increase any candidate's score. Four approaches to detect such adversarial attacks are also described. All the systems, and associated detection approaches, are evaluated on a free (spontaneous) speaking section from a Business English test. It is shown that on deep learning based SLA systems the average candidate score can be increased by almost one grade level using a single six word phrase appended to the end of the response hypothesis. Although these large gains can be obtained, they can be easily detected based on detection shifts from the scores of a “traditional” Gaussian Process based grader.

Due to their proven efficiency, machine-learning systems are deployed in a wide range of complex real-life problems. More specifically, Spiking Neural Networks (SNNs) emerged as a promising solution to the accuracy, resource-utilization, and energy-efficiency challenges in machine-learning systems. While these systems are going mainstream, they have inherent security and reliability issues. In this paper, we propose NeuroAttack, a cross-layer attack that threatens the SNNs integrity by exploiting low-level reliability issues through a high-level attack. Particularly, we trigger a fault-injection based sneaky hardware backdoor through a carefully crafted adversarial input noise. Our results on Deep Neural Networks (DNNs) and SNNs show a serious integrity threat to state-of-the art machine-learning techniques., Accepted for publication at the 2020 International Joint Conference on Neural Networks (IJCNN)

Područje računalne sigurnosti je podložno stalnim promjenama i nadogradnjama. Primjena metoda strojnog učenja kao tehnike očuvanja sigurnosti predstavlja pozitivnu promjenu i unaprjeđuje postojeće metode obrane. U radu smo se bazirali na dinamičke napade putem suparničkih primjera. Preciznije, orijentirali smo se na suparničke primjere u detekciji zloćudnih programa na sustavu Android. Početni korak u procesu generiranja suparničkih primjera je konstruiranje polaznog modela binarnog klasifikatora. Metoda generiranja suparničkih primjera koristi polazni model za određivanje smjera promjene koji pospješuje klasifikaciju u željenu klasu. Računamo gradijent funkcije neuronske mreže i na temelju iznosa gradijenta određujemo koje od značajki ulaznog vektora ćemo mijenjati. Svjesni činjenice da je moguće konstruirati smisleni primjer koji će zavarati početni model, primijenili smo suparnički trening kao metodu obrane od ovakvog tipa napada. Početni model smo dodatno trenirali na suparničkim primjerima kako bi ga učinili robusnijim i time manje osjetljivim na moguće napade. Naš suparnički klasifikator je pokazao bolje rezultate u detekciji zloćudnih programa od početnog modela. Istovremeno je zadržana točnost modela u vidu početnih zloćudnih aplikacija i poboljšana sposobnost detekcije primjera koji su specijalno modelirani kako bi zavarali klasifikator. The area of computer security is under constant development. This paper describes the use of adversarial machine learning in the context of computer security. Adversarial learning is used to confront the attackers that dynamically change their behavior over time. Existing models are observed to find weaknesses that could be exploited. We are concentrated on the task of fighting adversarial examples in the context of malicious programs on Android operating systems. The first step of our approach is constructing a standard binary classifier. This model is used to craft adversarial examples. The perturbation algorithm is focused on finding the direction in which input examples can be crafted to mislead the model. The implemented method is based on the values of the network gradient. Features whose absolute value is the greatest are added in an attempt of misleading the classifier. Produced adversarial examples are further used for adversarial training. Adversarial training is a defense mechanism based on adversarial examples. Crafted examples are used in a training set of a new model. This model is known as an adversarial classifier. Adversarial training is found to be an efficient way of fighting with adversarial examples. This model is more robust than the previous one. Model accuracy, false negative, and false positive rates support this claim.

Observed social networks are often considered as proxies for underlying social networks. The analysis of observed networks oftentimes involves the identification of influential nodes via various centrality measures. This paper brings insights from research on adversarial attacks on machine learning systems to the domain of social networks by studying strategies by which an adversary can minimally perturb the observed network structure to achieve their target function of modifying the ranking of a target node according to centrality measures. This can represent the attempt of an adversary to boost or demote the degree to which others perceive individual nodes as influential or powerful. We study the impact of adversarial attacks on targets and victims, and identify metric-based security strategies to mitigate such attacks. We conduct a series of controlled experiments on synthetic network data to identify attacks that allow the adversary to achieve their objective with a single move. We then replicate the experiments with empirical network data. We run our experiments on common network topologies and use common centrality measures. We identify a small set of moves that result in the adversary achieving their objective. This set is smaller for decreasing centrality measures than for increasing them. For both synthetic and empirical networks, we observe that larger networks are less prone to adversarial attacks than smaller ones. Adversarial moves have a higher impact on cellular and small-world networks, while random and scale-free networks are harder to perturb. Also, empirical networks are harder to attack than synthetic networks. Using correlation analysis on our experimental results, we identify how combining measures with low correlation can aid in reducing the effectiveness of adversarial moves. Our results also advance the knowledge about the robustness of centrality measures to network perturbations. The notion of changing social network data to yield adversarial outcomes has practical implications, e.g., for information diffusion on social media, influence and power dynamics in social systems, and developing solutions to improving network security.

Recent studies have revealed that neural network-based policies can be easily fooled by adversarial examples. However, while most prior works analyze the effects of perturbing every pixel of every frame assuming white-box policy access, in this paper we take a more restrictive view towards adversary generation - with the goal of unveiling the limits of a model's vulnerability. In particular, we explore minimalistic attacks by defining \textit{\textbf{three key settings}}: (1) black-box policy access: where the attacker only has access to the input (state) and output (action probability) of an RL policy; (2) fractional-state adversary: where only several pixels are perturbed, with the extreme case being a single-pixel adversary; and (3) tactically-chanced attack: where only significant frames are tactically chosen to be attacked. We formulate the adversarial attack by accommodating the three key settings, and explore their potency on six Atari games by examining four fully trained state-of-the-art policies. In Breakout, for example, we surprisingly find that: (i) all policies showcase significant performance degradation by merely modifying 0.01% of the input state, and (ii) the policy trained by DQN is totally deceived by perturbing only 1% frames. National Research Foundation (NRF) Accepted version This work is funded by the National Research Foundation, Singapore under its AI Singapore programme [Award No.: AISG-RP-2018-004] and the Data Science and Artificial Intelligence Research Center (DSAIR) at Nanyang Technological University. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not reflect the views of National Research Foundation, Singapore.

This paper investigates the vulnerability of urban traffic networks to cyber attacks on traffic lights. We model traffic signal tampering as a bi-objective optimization problem that simultaneously seeks to reduce vehicular throughput in the network over time (maximize impact) while introducing minimal changes to network signal timings (minimize noticeability). We represent the Spatio-temporal traffic dynamics as a static network flow problem on a time-expanded graph. This allows us to reduce the (non-convex) attack problem to a tractable form, which can be solved using traditional techniques used to solve linear network programming problems. We show that minor but objective adjustments in the signal timings over time can severely impact traffic conditions at the network level. We investigate network vulnerability by examining the concavity of the Pareto-optimal frontier obtained by solving the bi-objective attack problem. Numerical experiments are carried to illustrate the types of insights that can be extracted from the Pareto-optimal frontier. For instance, our experiments suggest that the vulnerability of a traffic network to signal tampering is independent of the demand levels.

The openness of Android framework and the enhancement of users trust have gained the attention of malware writers. The momentum of downloaded applications (app for short) from numerous app stores has stimulated the proliferation of mobile malware. Now the threat is due to the sophistication in malware being written to bypass signature-based detectors. In this paper, we investigate system calls to tackle mobile malware on Android operating system. To do so, we first employed machine learning to extract system calls. We then performed the empirical estimation of system calls derived from diverse datasets employing human interaction and random inputs. After accomplishing intensive experiments on synthesized system calls with two feature selection approach, namely Absolute Difference of Weighted System Calls (ADWSC) and Ranked System Calls using Large Population Test (RSLPT), we validated the results on five datasets. All classifiers generated in Area Under Curve of 1.0 with an accuracy exceeding 99.9% suggest the appropriateness and efficacy of the proposed approach. Finally, we evaluated the effectiveness of classifier against adversarial attacks and found that the classifiers are vulnerable to data poisoning and label flipping attacks. Adversarial examples created by poisoning malware samples resulted in the significant drop of classifier performance on perturbing 12–18 prominent attributes. Moreover, we implemented class label poisoning attacks which brought down the classification accuracy by 50% on altering labels of 50 malicious training instances.

International audience; A Security Operations Center (SOC) is a central technical level unit responsible for monitoring, analyzing, assessing, and defending an organization's security posture on an ongoing basis. The SOC staff works closely with incident response teams, security analysts, network engineers and organization managers using sophisticated data processing technologies such as security analytics, threat intelligence, and asset criticality to ensure security issues are detected, analyzed and finally addressed quickly. Those techniques are part of a reactive security strategy because they rely on the human factor, experience and the judgment of security experts, using supplementary technology to evaluate the risk impact and minimize the attack surface. This study suggests an active security strategy that adopts a vigorous method including ingenuity, data analysis, processing and decision-making support to face various cyber hazards. Specifically, the paper introduces a novel intelligence driven cognitive computing SOC that is based exclusively on progressive fully automatic procedures. The proposed λ-Architecture Network Flow Forensics Framework (λ-ΝF3) is an efficient cybersecurity defense framework against adversarial attacks. It implements the Lambda machine learning architecture that can analyze a mixture of batch and streaming data, using two accurate novel computational intelligence algorithms. Specifically, it uses an Extreme Learning Machine neural network with Gaussian Radial Basis Function kernel (ELM/GRBFk) for the batch data analysis and a Self-Adjusting Memory k-Nearest Neighbors classifier (SAM/k-NN) to examine patterns from real-time streams. It is a forensics tool for big data that can enhance the automate defense strategies of SOCs to effectively respond to the threats their environments face.

Interpretability has emerged as a crucial aspect of building trust in machine learning systems, aimed at providing insights into the working of complex neural networks that are otherwise opaque to a user. There are a plethora of existing solutions addressing various aspects of interpretability ranging from identifying prototypical samples in a dataset to explaining image predictions or explaining mis-classifications. While all of these diverse techniques address seemingly different aspects of interpretability, we hypothesize that a large family of interepretability tasks are variants of the same central problem which is identifying \emph{relative} change in a model's prediction. This paper introduces MARGIN, a simple yet general approach to address a large set of interpretability tasks MARGIN exploits ideas rooted in graph signal analysis to determine influential nodes in a graph, which are defined as those nodes that maximally describe a function defined on the graph. By carefully defining task-specific graphs and functions, we demonstrate that MARGIN outperforms existing approaches in a number of disparate interpretability challenges., Technical Report

In computer vision, Defense-GAN is a framework that leverages the capability of generative models to defend neural networks (NNs) against adversarial attacks. This paper investigates how Defense-GAN will behave when it is used to defend RNNs against adversarial texts. It is assumed that little perturbations in textual comments could lead into misclassification, due to its short length. Even though adversarial texts may not be tricky for people, they could be for classifiers.

In the last decade, the use of Machine Learning techniques in anomaly-based intrusion detection systems has seen much success. However, recent studies have shown that Machine learning in general and deep learning specifically are vulnerable to adversarial attacks where the attacker attempts to fool models by supplying deceptive input. Research in computer vision, where this vulnerability was first discovered, has shown that adversarial images designed to fool a specific model can deceive other machine learning models. In this paper, we investigate the transferability of adversarial network traffic against multiple machine learning-based intrusion detection systems. Furthermore, we analyze the robustness of the ensemble intrusion detection system, which is notorious for its better accuracy compared to a single model, against the transferability of adversarial attacks. Finally, we examine Detect & Reject as a defensive mechanism to limit the effect of the transferability property of adversarial network traffic against machine learning-based intrusion detection systems., Advances in Cyber Security. ACeS 2021