Search

Recent studies show that neural networks are vulnerable to backdoor attacks, in which compromised networks behave normally for clean inputs but make mistakes when a pre-defined trigger appears. Although prior studies have designed various invisible triggers to avoid causing visual anomalies, they cannot evade some trigger detectors. In this paper, we consider the stealthiness of backdoor attacks from input space and feature representation space. We propose a novel backdoor attack named sparse backdoor attack, and investigate the minimum required trigger to induce the well-trained networks to make incorrect results. A U-net-based generator is employed to create triggers for each clean image. Considering the stealthiness of the trigger, we restrict the elements of the trigger between −1 and 1. In the aspect of the feature representation domain, we adopt an entanglement cost function to minimize the gap between feature representations of benign and malicious inputs. The inseparability of benign and malicious feature representations contributes to the stealthiness of our attack against various model diagnosis-based defences. We validate the effectiveness and generalization of our method by conducting extensive experiments on multiple datasets and networks. [ABSTRACT FROM AUTHOR]

Deep neural network (DNN) -based object detectors have achieved remarkable success, but recent research has revealed their vulnerability to backdoor attacks. The attacks cause the poisoned model to output results normally on benign images, but outputs results specified by the attacker on images inserted with a trigger. Although backdoor attacks have been extensively investigated on image classification tasks, their exploration in object detection tasks remains limited. With the increasing application of object detectors in safety-sensitive fields such as autonomous driving, backdoor attacks on object detection tasks may have serious consequences. Currently, strategies for object disappearance attack scenarios exhibit certain limitations. First, these strategies typically exhibit a one-to-one correspondence, implying that the insertion of one trigger can only result in the disappearance of one object. Second, these strategies typically necessitate the attacker’s knowledge of the object’s precise location information to achieve its disappearance, thereby rendering real-time insertion of triggers unfeasible. Finally, these strategies exhibit diminished attack success rates when applied to two-stage detectors. The paper presents a global object disappearance attack scenario and proposes a simple, covert, and highly effective attack strategy. Experimental evaluations are conducted on four widely-used object detection models (Yolov5s, Yolov8s, Faster R-CNN, and Libra R-CNN) using two benchmark datasets (PASCAL VOC $07+12$ and MS COCO2017) to validate the effectiveness of the proposed strategy. The results demonstrate that the success rate of this attack strategy exceeds 96%, while the poison rate is only 10%.

The vulnerability of deep-learning-based image classification models to erroneous conclusions in the presence of small perturbations crafted by attackers has prompted attention to the question of the models' robustness level. However, the question of how to comprehensively and fairly measure the adversarial robustness of models with different structures and defenses as well as the performance of different attack methods has never been accurately answered. In this work, we present the design, implementation, and evaluation of Canary, a platform that aims to answer this question. Canary uses a common scoring framework that includes 4 dimensions with 26 (sub)metrics for evaluation. First, Canary generates and selects valid adversarial examples and collects metrics data through a series of tests. Then it uses a two-way evaluation strategy to guide the data organization and finally integrates all the data to give the scores for model robustness and attack effectiveness. In this process, we use Item Response Theory (IRT) for the first time to ensure that all the metrics can be fairly calculated into a score that can visually measure the capability. In order to fully demonstrate the effectiveness of Canary, we conducted large-scale testing of 15 representative models trained on the ImageNet dataset using 12 white-box attacks and 12 black-box attacks and came up with a series of in-depth and interesting findings. This further illustrates the capabilities and strengths of Canary as a benchmarking platform. Our paper provides an open-source framework for model robustness evaluation, allowing researchers to perform comprehensive and rapid evaluations of models or attack/defense algorithms, thus inspiring further improvements and greatly benefiting future work. [ABSTRACT FROM AUTHOR]

Together with impressive advances touching every aspect of our society, AI technology based on Deep Neural Networks (DNN) is bringing increasing security concerns. While attacks operating at test time have monopolised the initial attention of researchers, backdoor attacks, exploiting the possibility of corrupting DNN models by interfering with the training process, represent a further serious threat undermining the dependability of AI techniques. In backdoor attacks, the attacker corrupts the training data to induce an erroneous behaviour at test time. Test-time errors, however, are activated only in the presence of a triggering event. In this way, the corrupted network continues to work as expected for regular inputs, and the malicious behaviour occurs only when the attacker decides to activate the backdoor hidden within the network. Recently, backdoor attacks have been an intense research domain focusing on both the development of new classes of attacks, and the proposal of possible countermeasures. The goal of this overview is to review the works published until now, classifying the different types of attacks and defences proposed so far. The classification guiding the analysis is based on the amount of control that the attacker has on the training process, and the capability of the defender to verify the integrity of the data used for training, and to monitor the operations of the DNN at training and test time. Hence, the proposed analysis is suited to highlight the strengths and weaknesses of both attacks and defences with reference to the application scenarios they are operating in.

Backdoor attacks cause model misbehaving by first implanting backdoors in deep neural networks (DNNs) during training and then activating the backdoor via samples with triggers during inference. The compromised models could pose serious security risks to artificial intelligence systems, such as misidentifying ‘stop’ traffic sign into ‘80km/h’. In this paper, we investigate the connection characteristic between the backdoor and the trigger in DNNs and observe the fact that the backdoor is implanted via establishing a link between a cluster of neurons, representing the backdoor, and the triggers. Based on this observation, we design LinkBreaker, a new generic scheme for defending against backdoor attacks. In particular, LinkBreaker deploys a neuron consistency check mechanism for identifying compromised neuron set related to the trigger. Then, the LinkBreaker regulates the model to make predictions based on benign neuron set only and thus breaks the link between the backdoor and the trigger. Compared to previous defenses, LinkBreaker offers a more general backdoor countermeasure that is not only effective against input-agnostic backdoors but also source-specific backdoors, which the later can not be defeated by majority of state-of-the-arts. Besides, LinkBreaker is robust against adversarial examples, which, to a large extent, provides a holistic defense against adversarial example attacks on DNNs, while almost all current backdoor defenses do not have such consideration and capability. Extensive experimental evaluations on real datasets demonstrate that LinkBreaker is with high efficacy of suppressing trigger inputs while incurring no noticeable accuracy deterioration on benign inputs. [ABSTRACT FROM AUTHOR]

Deep Neural Networks (DNNs) are vulnerable to backdoor attacks, which are external entity model providers that can inject backdoors into the network for illegal purposes. Current attack methods rely on manipulated images with embedded triggers to infiltrate carrier networks, but they suffer from detectable distortions and limited integration into neural networks. To delve further into the potential vulnerabilities of these models, this study introduces an innovative backdoor attack strategy, leveraging deep learning steganography through a Generative Adversarial Network (GAN). Our approach utilizes steganography to create manipulated images, capitalizing on the unique sensitivity of neural networks to minute perturbations. The network is then trained de novo with these manipulated images, creating a backdoor-infused model. Fundamentally, our method harnesses the pronounced sensitivity of DNNs to nuanced alterations. Experimental outcomes substantiate that our backdoor can be effectively integrated into the models, yielding high attack success rates. Notably, it also adeptly circumvents both contemporary state-of-the-art defense mechanisms and human inspection. Our source code is publicly accessible at https://github.com/DLAIResearch/NNSDB. • We employ a novel dynamic backdoor attack against neural networks (NNSDB). • NNSDB produces adversarial perturbations as trigger patterns using Generative Adversarial Networks. • NNSDB achieves satisfactory attack performance and can withstand the most advanced defense mechanisms. • Experimental results further reveal the seriousness of backdoor attacks, which deserves to be alarmed. [ABSTRACT FROM AUTHOR]

• We reveal that the difficulty of clean-label backdoor attacks is mostly due to the antagonistic effects of 'robust features' and verify that DNNs have different learning abilities for different samples. • We revisit the paradigm of existing clean-label backdoor attacks and propose a new complementary paradigm by considering the different learning difficulties of samples. • We empirically verify the effectiveness and the poisoning transferability of our method on benchmark datasets and discuss its intrinsic mechanism. Recent studies demonstrated that deep neural networks (DNNs) are vulnerable to backdoor attacks. The attacked model behaves normally on benign samples, while its predictions are misled whenever adversary-specified trigger patterns appear. Currently, clean-label backdoor attacks are usually regarded as the most stealthy methods in which adversaries can only poison samples from the target class without modifying their labels. However, these attacks can hardly succeed. In this paper, we reveal that the difficulty of clean-label attacks mainly lies in the antagonistic effects of 'robust features' related to the target class contained in poisoned samples. Specifically, robust features tend to be easily learned by victim models and thus undermine the learning of trigger patterns. Based on these understandings, we propose a simple yet effective plug-in method to enhance clean-label backdoor attacks by poisoning 'hard' instead of random samples. We adopt three classical difficulty metrics as examples to implement our method. We demonstrate that our method can consistently improve vanilla attacks, based on extensive experiments on benchmark datasets. [ABSTRACT FROM AUTHOR]

• We propose a robust training method by jointly minimizing standard risk and robust risk, which is naturally extended the semi-supervised mode. • By generalizing the definition of the perturbed neighborhood to cover different types of perturbations, our method achieves the joint robustness to different perturbations, such as the pixel-wise and spatial perturbation. • Experiments on benchmark datasets verify the superiority of the proposed SRT method to state-of-the-art adversarial training methods, as well as the robustness of SRT to pixel-wise and spatial perturbations simultaneously. Adversarial examples have been shown to be a severe threat to deep neural networks (DNNs). One of the most effective adversarial defense methods is adversarial training (AT) through minimizing the adversarial risk R a d v , which encourages both the benign example x and its adversarially perturbed neighborhoods within the ℓ p -ball to be predicted as the ground-truth label. In this paper, we propose a novel defense method, the robust training (RT), by jointly minimizing two separated risks (i. e. , R s t a n d and R r o b), which are with respect to the benign example and its neighborhoods, respectively. The motivation is to explicitly and jointly enhance the accuracy and the adversarial robustness. We prove that R a d v is upper-bounded by R s t a n d + R r o b , which implies that RT has similar effect as AT. Intuitively, minimizing the standard risk enforces the benign example to be correctly predicted, while the robust risk minimization encourages the predictions of the neighbor examples to be consistent with the prediction of the benign example. Besides, since R r o b is independent of the ground-truth label, RT is naturally extended to the semi-supervised mode (i. e. , SRT), to further enhance its effectiveness. Moreover, we extend the ℓ p -bounded neighborhood to a general case, which covers different types of perturbations, such as the pixel-wise (i. e. , x + δ) or the spatial perturbation (i. e. , A x + b). Extensive experiments on benchmark datasets not only verify the superiority of the proposed SRT to state-of-the-art methods for defending pixel-wise or spatial perturbations separately but also demonstrate its robustness to both perturbations simultaneously. Our work may shed the light on the understanding of universal model robustness and the potential of unlabeled samples. The code for reproducing main results is available at https://github.com/THUYimingLi/Semi-supervised_Robust_Training. [ABSTRACT FROM AUTHOR]

Decision making tasks carried out by the usage of deep neural networks are successfully taking over in many areas, including those that are security critical, such as healthcare, transportation, smart grids, where intentional and unintentional failures can be disastrous. Edge computing systems are becoming ubiquitous nowadays, often serving deep learning tasks that do not need to be sent over to servers. Therefore, there is a necessity to evaluate the potential attacks that can target deep learning in the edge. In this work, we present evaluation of deep neural networks (DNNs) reliability against fault injection attacks. We first experimentally evaluate DNNs implemented in an embedded device by using laser fault injection to get the insight on possible attack vectors. We show practical results on four activation functions, ReLu, softmax, sigmoid, and tanh. We then perform a deep study on DNNs based on derived fault models by using several different attack strategies based on random faults. We also investigate a powerful attacker who can find effective fault location based on genetic algorithm, to show the most efficient attacks in terms of misclassification success rates. Finally, we show how a state of the art countermeasure against model extraction attack can be bypassed with a fault attack. Our results can serve as a basis to outline the susceptibility of DNNs to physical attacks which can be considered a viable attack vector whenever a device is deployed in hostile environment. • Physical security of embedded implementations of deep neural networks is a valid threat. • Fault injection attack can change the outcome of activation functions in neurons. • With 10% of faulted neurons, the risk of misclassification is up to 70%. [ABSTRACT FROM AUTHOR]