Search

Advances in adversarial attack and defense technologies will enhance the reliability of deep learning (DL) systems spirally. Most existing adversarial attack methods make overly ideal assumptions, which creates the illusion that the DL system can be attacked simply and has restricted the further improvement on DL systems. To perform practical adversarial attacks, a detection tolerant black-box adversarial-attack (DTBA) method against DL-based automatic modulation classification (AMC) is presented in this article. In the DTBA method, the local DL model as a substitution of the remote target DL model is trained first. The training dataset is generated by an attacker, labeled by the target model, and augmented by Jacobian transformation. Then, the conventional gradient attack method is utilized to generate adversarial attack examples toward the local DL model. Moreover, before launching attack to the target model, the local model estimates the misclassification probability of the perturbed examples in advance and deletes those invalid adversarial examples. Compared with related attack methods of different criteria on public datasets, the DTBA method can reduce the attack cost while increasing the rate of successful attack. Adversarial attack transferability of the proposed method on the target model has increased by more than 20%. The DTBA method will be suitable for launching flexible and effective black-box adversarial attacks against DL-based AMC systems. [ABSTRACT FROM AUTHOR]

In recent years, artificial intelligence technology represented by deep learning has achieved remarkable results in image recognition, semantic analysis, natural language processing and other fields. In particular, deep neural networks have been widely used in different security-sensitive tasks. Fields, such as facial payment, smart medical and autonomous driving, which accelerate the construction of smart cities. Meanwhile, in order to fully unleash the potential of edge big data, there is an urgent need to push the AI frontier to the network edge. Edge AI, the combination of artificial intelligence and edge computing, supports the deployment of deep learning algorithms to edge devices that generate data, and has become a key driver of smart city development. However, the latest research shows that deep neural networks are vulnerable to attacks from adversarial example and output wrong results. This type of attack is called adversarial attack, which greatly limits the promotion of deep neural networks in tasks with extremely high security requirements. Due to the influence of adversarial attacks, researchers have also begun to pay attention to the research in the field of adversarial defense. In the game process of adversarial attacks and defense technologies, both attack and defense technologies have been developed rapidly. This article first introduces the principles and characteristics of adversarial attacks, and summarizes and analyzes the adversarial example generation methods in recent years. Then, it introduces the adversarial example defense technology in detail from the three directions of model, data, and additional network. Finally, combined with the current status of adversarial example generation and defense technology development, put forward challenges and prospects in this field. [ABSTRACT FROM AUTHOR]

Adversarial attacks on deep neural networks (DNNs) have been found for several years. However, the existing adversarial attacks have high success rates only when the information of the victim DNN is well-known or could be estimated by the structure similarity or massive queries. In this paper, we propose to Attack on Attention (AoA), a semantic property commonly shared by DNNs. AoA enjoys a significant increase in transferability when the traditional cross entropy loss is replaced with the attention loss. Since AoA alters the loss function only, it could be easily combined with other transferability-enhancement techniques and then achieve SOTA performance. We apply AoA to generate 50000 adversarial samples from ImageNet validation set to defeat many neural networks, and thus name the dataset as DAmageNet. 13 well-trained DNNs are tested on DAmageNet, and all of them have an error rate over 85 percent. Even with defenses or adversarial training, most models still maintain an error rate over 70 percent on DAmageNet. DAmageNet is the first universal adversarial dataset. It could be downloaded freely and serve as a benchmark for robustness testing and adversarial training. [ABSTRACT FROM AUTHOR]

Deep Learning is the most widely used tool in the contemporary field of computer vision. Its ability to accurately solve complex problems is employed in vision research to learn deep neural models for a variety of tasks, including security critical applications. However, it is now known that deep learning is vulnerable to adversarial attacks that can manipulate its predictions by introducing visually imperceptible perturbations in images and videos. Since the discovery of this phenomenon in 2013, it has attracted significant attention of researchers from multiple sub-fields of machine intelligence. In 2018, we published the first-ever review of the contributions made by the computer vision community in adversarial attacks on deep learning (and their defenses). Many of those contributions have inspired new directions in this area, which has matured significantly since witnessing the first generation methods. Hence, as a legacy sequel of our first literature survey, this review article focuses on the advances in this area since 2018. We thoroughly discuss the first generation attacks and comprehensively cover the modern attacks and their defenses appearing in the prestigious sources of computer vision and machine learning research. Besides offering the most comprehensive literature review of adversarial attacks and defenses to date, the article also provides concise definitions of technical terminologies for the non-experts. Finally, it discusses challenges and future outlook of this direction based on the literature since the advent of this research direction.

Convolutional neural networks (CNNs) have become one of the most important tools for image classification. However, many models are susceptible to adversarial attacks, and CNNs can perform misclassifications. In previous works, we successfully developed an EA-based black-box attack that creates adversarial images for the target scenario that fulfils two criteria. The CNN should classify the adversarial image in the target category with a confidence ≥ 0.95, and a human should not notice any difference between the adversarial and original images. Thanks to extensive experiments performed with the CNN ${\mathcal{C}}$ = VGG-16 trained on the CIFAR-10 dataset to classify images according to 10 categories, this paper, which substantially enhances most aspects of Chitic et al. (2021), addresses four issues. (1) From a pure EA point of view, we highlight the conceptual originality of our algorithm $\text {EA}_{d}^{\text {target}, {\mathcal{C}}}$ , versus the classical EA approach. The competitive advantage obtained was assessed experimentally during image classification. (2) We then measured the intrinsic performance of the EA-based attack for an extensive series of ancestor images. (3) We challenged the filter resistance of the adversarial images created by the EA for five well-known filters. (4) We proceed to the creation of natively filter-resistant adversarial images that can fool humans, CNNs, and CNNs composed with filters.

The vulnerability of deep neural network (DNN)-based systems makes them susceptible to adversarial perturbation and may cause classification task failure. In this work, we propose an adversarial attack model using the Artificial Bee Colony (ABC) algorithm to generate adversarial samples without the need for a further gradient evaluation and training of the substitute model, which can further improve the chance of task failure caused by adversarial perturbation. In untargeted attacks, the proposed method obtained 100%, 98.6%, and 90.00% success rates on the MNIST, CIFAR-10 and ImageNet datasets, respectively. The experimental results show that the proposed ABCAttack can not only obtain a high attack success rate with fewer queries in the black-box setting, but also break some existing defenses to a large extent, and is not limited by model structure or size, which provides further research directions for deep learning evasion attacks and defenses. [ABSTRACT FROM AUTHOR]

Neural networks have been widely applied but they are still vulnerable to adversarial examples. More and more defense models have been proposed and they can resist the attacks to the neural networks. In order to generate adversarial examples with good transferability, we propose the restricted region based iterative gradient method (RRI-GM) for non-targeted attack, which aims at generating adversarial examples to make black-box defense models output wrong decision. We first use object detection algorithm to restrict some key regions in the images, since we regard perturbation in the key region affects more than the whole image. To improve the efficiency of attacks, we use gradient-based attack methods and they show good performance. In addition, the process is iterated for multiple rounds to generate adversarial examples with good transferability. Furthermore, we conduct extensive experiments to validate the effectiveness of the proposed method, and the results show that our method can achieve good attack performance against black-box defense models.

SQL injection is a highly detrimental web attack technique that can result in significant data leakage and compromise system integrity. To counteract the harm caused by such attacks, researchers have devoted much attention to the examination of SQL injection detection techniques, which have progressed from traditional signature-based detection methods to machine- and deep-learning-based detection models. These detection techniques have demonstrated promising results on existing datasets; however, most studies have overlooked the impact of adversarial attacks, particularly black-box adversarial attacks, on detection methods. This study addressed the shortcomings of current SQL injection detection techniques and proposed a reinforcement-learning-based black-box adversarial attack method. The proposal included an innovative vector transformation approach for the original SQL injection payload, a comprehensive attack-rule matrix, and a reinforcement-learning-based method for the adaptive generation of adversarial examples. Our approach was evaluated on existing web application firewalls (WAF) and detection models based on machine- and deep-learning methods, and the generated adversarial examples successfully bypassed the detection method at a rate of up to 97.39%. Furthermore, there was a substantial decrease in the detection accuracy of the model after multiple attacks had been carried out on the detection model via the adversarial examples.

The research on image-classification-adversarial attacks is crucial in the realm of artificial intelligence (AI) security. Most of the image-classification-adversarial attack methods are for white-box settings, demanding target model gradients and network architectures, which is less practical when facing real-world cases. However, black-box adversarial attacks immune to the above limitations and reinforcement learning (RL) seem to be a feasible solution to explore an optimized evasion policy. Unfortunately, existing RL-based works perform worse than expected in the attack success rate. In light of these challenges, we propose an ensemble-learning-based adversarial attack (ELAA) targeting image-classification models which aggregate and optimize multiple reinforcement learning (RL) base learners, which further reveals the vulnerabilities of learning-based image-classification models. Experimental results show that the attack success rate for the ensemble model is about 35% higher than for a single model. The attack success rate of ELAA is 15% higher than those of the baseline methods.

Deep neural networks (DNNs) are proven vulnerable to attack against adversarial examples. Black-box transfer attacks pose a massive threat to AI applications without accessing target models. At present, the most effective black-box attack methods mainly adopt data enhancement methods, such as input transformation. Previous data enhancement frameworks only work on input transformations that satisfy accuracy or loss invariance. However, it does not work for other transformations that do not meet the above conditions, such as the transformation which will lose information. To solve this problem, we propose a new noise data enhancement framework (NDEF), which only transforms adversarial perturbation to avoid the above issues effectively. In addition, we introduce random erasing under this framework to prevent the over-fitting of adversarial examples. Experimental results show that the black-box attack success rate of our method Random Erasing Iterative Fast Gradient Sign Method (REI-FGSM) is 4.2% higher than DI-FGSM in six models on average and 6.6% higher than DI-FGSM in three defense models. REI-FGSM can combine with other methods to achieve excellent performance. The attack performance of SI-FGSM can be improved by 22.9% on average when combined with REI-FGSM. Besides, our combined version with DI-TI-MI-FGSM, i.e., DI-TI-MI-REI-FGSM can achieve an average attack success rate of 97.0% against three ensemble adversarial training models, which is greater than the current gradient iterative attack method. We also introduce Gaussian blur to prove the compatibility of our framework. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Researches have shown that Deep neural networks (DNNs) are vulnerable to adversarial examples. In this paper, we propose a generative model to explore how to produce adversarial examples that can deceive multiple deep learning models simultaneously. Unlike most of popular adversarial attack algorithms, the one proposed in this paper is based on the Generative Adversarial Networks (GAN). It can quickly produce adversarial examples and perform black-box attacks on multi-model. To enhance the transferability of the samples generated by our approach, we use multiple neural networks in the training process. Experimental results on MNIST showed that our method can efficiently generate adversarial examples. Moreover, it can successfully attack various classes of deep neural networks at the same time, such as fully connected neural networks (FCNN), convolutional neural networks (CNN) and recurrent neural networks (RNN). We performed a black-box attack on VGG16 and the experimental results showed that when the test data classes are ten (0-9), the attack success rate is 97.68%, and when the test data classes are seven (0-6), the attack success rate is up to 98.25%. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNNs) are proven vulnerable to attack against adversarial examples. Black-box transfer attacks pose a massive threat to AI applications without accessing target models. At present, the most effective black-box attack methods mainly adopt data enhancement methods, such as input transformation. Previous data enhancement frameworks only work on input transformations that satisfy accuracy or loss invariance. However, it does not work for other transformations that do not meet the above conditions, such as the transformation which will lose information. To solve this problem, we propose a new noise data enhancement framework (NDEF), which only transforms adversarial perturbation to avoid the above issues effectively. In addition, we introduce random erasing under this framework to prevent the over-fitting of adversarial examples. Experimental results show that the black-box attack success rate of our method Random Erasing Iterative Fast Gradient Sign Method (REI-FGSM) is 4.2% higher than DI-FGSM in six models on average and 6.6% higher than DI-FGSM in three defense models. REI-FGSM can combine with other methods to achieve excellent performance. The attack performance of SI-FGSM can be improved by 22.9% on average when combined with REI-FGSM. Besides, our combined version with DI-TI-MI-FGSM, i.e., DI-TI-MI-REI-FGSM can achieve an average attack success rate of 97.0% against three ensemble adversarial training models, which is greater than the current gradient iterative attack method. We also introduce Gaussian blur to prove the compatibility of our framework.

Deep neural networks (DNNs) have been applied to various industries. In particular, DNNs on embedded devices have attracted considerable interest because they allow real-time and distributed processing on site. However, adversarial examples (AEs), which add small perturbations to the input data of DNNs to cause misclassification, are serious threats to DNNs. In this paper, a novel black-box attack is proposed to craft AEs based only on processing time, i.e., the side-channel leaks from DNNs on embedded devices. Unlike several existing black-box attacks that utilize output probability, the proposed attack exploits the relationship between the number of activated nodes and processing time without using training data, model architecture, parameters, substitute models, or output probability. The perturbations for AEs are determined by the differential processing time based on the input data of the DNNs in the proposed attack. The experimental results show that the AEs of the proposed attack effectively cause an increase in the number of activated nodes and the misclassification of one of the incorrect labels against the DNNs on a microcontroller unit. Moreover, these results indicate that the attack can evade gradient-masking and confidence reduction countermeasures, which conceal the output probability, to prevent the crafting of AEs against several black-box attacks. Finally, the countermeasures against the attack are implemented and evaluated to clarify that the implementation of an activation function with data-dependent timing leaks is the cause of the proposed attack.

Deep neural networks are susceptible to tiny crafted adversarial perturbations which are always added to all the pixels of the image to craft an adversarial example. Most of the existing adversarial attacks can reduce the L2 distance between the adversarial image and the source image to a minimum but ignore the L0 distance which is still huge. To address this issue, we introduce a new black-box adversarial attack based on evolutionary method and bisection method, which can greatly reduce the L0 distance while limiting the L2 distance. By flipping pixels of the target image, an adversarial example is generated, in which a small number of pixels come from the target image and the rest pixels are from the source image. Experiments show that our attack method is able to generate high quality adversarial examples steadily. Especially for generating adversarial examples for large scale images, our method performs better. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Constructing adversarial examples in a black-box threat model injures the original images by introducing visual distortion. In this paper, we propose a novel black-box attack approach that can directly minimize the induced distortion by learning the noise distribution of the adversarial example, assuming only loss-oracle access to the black-box network. To quantify visual distortion, the perceptual distance between the adversarial example and the original image, is introduced in our loss. We first approximate the gradient of the corresponding non-differentiable loss function by sampling noise from the learned noise distribution. Then the distribution is updated using the estimated gradient to reduce visual distortion. The learning continues until an adversarial example is found. We validate the effectiveness of our attack on ImageNet. Our attack results in much lower distortion when compared to the state-of-the-art black-box attacks and achieves 100% success rate on InceptionV3, ResNet50 and VGG16bn. Furthermore, we theoretically prove the convergence of our model. The code is publicly available at https://github.com/Alina-1997/visual-distortion-in-attack. [ABSTRACT FROM AUTHOR]

• We propose a novel adversarial attack on face images captured in different spectra. • The attacks are transposed across different DNN architectures and different spectra. • The attacks are composed by a FGSM attack step and a noise transfer step. • The attacks are performed on two different dataset: VIS-NIR and ViS-TH. • The results show that the DNN architectures have different responses to the attack. The ability of Deep Neural Networks (DNNs) to make fast predictions with high accuracy made them very popular in real-time applications. DNNs are nowadays in use for secure access to services or mobile devices. However, as DNNs use increased, at the same time attack techniques are born to "break" them. This paper presents a particular way to fool DNNs by moving from one spectrum to another one. The application field we explore is face recognition. The attack is first built on a trained Face DNN on Visible, Near Infrared or Thermal images, then transposed to another spectrum to fool another DNN. The attacks performed are based on the Fast Gradient Sign Method with the aim to misclassify the subject knowing the DNN to attack (White-Box Attack) but without knowing the DNN on which the attack will be transposed (Black-Box Attack). Results show that this cross-spectral attack is able to fool the most popular DNN architectures. In worst cases the DNN becomes useless to perform face recognition after the attack. [ABSTRACT FROM AUTHOR]

Face recognition has achieved great success in the last five years due to the development of deep learning methods. However, deep convolutional neural networks (DCNNs) have been found to be vulnerable to adversarial examples. In particular, the existence of transferable adversarial examples can severely hinder the robustness of DCNNs since this type of attacks can be applied in a fully black-box manner without queries on the target system. In this work, we first investigate the characteristics of transferable adversarial attacks in face recognition by showing the superiority of feature-level methods over label-level methods. Then, to further improve transferability of feature-level adversarial examples, we propose DFANet, a dropout-based method used in convolutional layers, which can increase the diversity of surrogate models and obtain ensemble-like effects. Extensive experiments on state-of-the-art face models with various training databases, loss functions and network architectures show that the proposed method can significantly enhance the transferability of existing attack methods. Finally, by applying DFANet to the LFW database, we generate a new set of adversarial face pairs that can successfully attack four commercial APIs without any queries. This TALFW database is available to facilitate research on the robustness and defense of deep face recognition. [ABSTRACT FROM AUTHOR]

The main challenge in deceiving face recognition (FR) models lies in the target model under the black-box setting. Existing works seek to generate adversarial examples to improve the adversarial transferability for black-box attacks. However, the attack performance and quality of the crafted image still have room for improvement. In this work, we propose a novel method called TransMix to improve the transferability of adversarial face examples based on data augmentation. Our approach leverages the mixture of the original image with a mixed sample image that is randomly mixed using images from different identities or the same identities, incorporating information from diverse categories. Then, we perform random transformations N times to create diverse input patterns, exploiting the gradient from various images and other identities in the same iteration. Extensive experiments conducted on the CelebA dataset demonstrate that TransMix achieves a significantly higher attack success rate against different FR models and Vision Transformers (ViTs), outperforming the best competitor by a large margin of 5.6% and 8.8% when attacking the ViTs using adversarial images generated on the ArcFace model. Our results also confirm that adversarial examples crafted by TransMix exhibit good adversarial transferability against defense models, achieving an attack success rate of 52.3% on the Bit-Red model. • This paper presents TransMix as a solution to enhance adversarial transferability. • TransMix leverages image augmentation techniques for adversarial transferability. • TransMix generates mixed images and integrates them into the original images. • We calculate the gradient of both the original and mixed image. • TransMix exhibits superior cross-model transferability in face recognition models. [ABSTRACT FROM AUTHOR]

Recent works have shown that adversarial examples (AEs) can attack and successfully transfer across various neural networks, highlighting the potential danger they pose. However, current approaches that focus on task-specific loss functions may not be as effective across different tasks. Additionally, the use of cloud APIs in practice, which often involve combining multiple tasks, also weakens the effectiveness of existing attacks. To address these issues, we propose a method called Attentional Feature Erase, which is a task-agnostic attack with improved cross-task transferability and effectiveness on computer vision-based cloud APIs. We view the transferability of AEs as a latent contribution for each layer of deep neural networks. By focusing on the intermediate layers of model backbones and reducing high-value features in each intermediate feature map, we are able to maximize the attack performance. Additionally, to better aggregate the gradients and generate adversarial perturbations during backward propagation, Transferability Regularizer is proposed to calculate the attention heatmap for each intermediate feature map and systematically combine the gradients. Comprehensive set of experiments on the Google Cloud Vision APIs and public available datasets (i.e. ImageNet, PASCAL VOC and MS COCO) show that the proposed AFE attack is more effective and has better transferability compared to the state-of-the-art baselines. • Proposed adversarial examples effectively transfer across vision tasks and networks. • Reducing dispersion in intermediate features lessens robustness. • Gradient aggregation highlights key regions in the network. [ABSTRACT FROM AUTHOR]

In order to better solve the shortcomings of Deep Neural Networks (DNNs) susceptible to adversarial examples, evaluating existing neural network classification performance and increasing training sets to improve the robustness of classification models require more effective methods of the adversarial examples generation. Under the black-box condition, less information about parameters of the classification model, limited query times, and less feedback information available, it is difficult to generate adversarial examples against the black-box model. In order to further improve the efficiency of the adversarial images generation, we propose two different variants of Partial Swarm Optimization algorithm (vPSO) base on the traditional Partial Swarm Optimization for the targeted and non-targeted attack under conditions of the completely black-box. Compared with the existing of the state-of-the-art generation algorithm, the vPSO effectively reduce the number of queries to the black-box classifier and the dependence on the feedback information. The success rate of the targeted attack is up to 96.0% and the average number of queries for the black-box model is greatly reduced. Furthermore, we propose an efficient target image screening method in targeted attacks, as well as the concept of easy-to-attack and hard-to-attack images in non-targeted attacks, and give corresponding distinctions.

Deep neural networks(DNNs) are widely used in AI-controlled Cyber-Physical Systems (CPS) to controll cars, robotics, water treatment plants and railways. However, DNNs have vulnerabilities to well-designed input samples that are called adversarial examples. Adversary attack is one of the important techniques for detecting and improving the security of neural networks. Existing attacks, including state-of-the-art black-box attack have a lower success rate and make invalid queries that are not beneficial to obtain the direction of generating adversarial examples. For these reasons, this paper proposed a CMA-ES-based adversarial attack on black-box DNNs. Firstly, an efficient method to reduce the number of invalid queries is introduced. Secondly, a black-box attack of generating adversarial examples to fit a high-dimensional independent Gaussian distribution of the local solution space is proposed. Finally, a new CMA-based perturbation compression method is applied to make the process of reducing perturbation smoother. Experimental results on ImageNet classifiers show that the proposed attack has a higher success-rate than the state-of-the-art black-box attack but reduce the number of queries by 30% equally.

Much research on adversarial attacks has proved that deep neural networks have certain security vulnerabilities. Among potential attacks, black-box adversarial attacks are considered the most realistic based on the the natural hidden nature of deep neural networks. Such attacks have become a critical academic emphasis in the current security field. However, current black-box attack methods still have shortcomings, resulting in incomplete utilization of query information. Our research, based on the newly proposed Simulator Attack, proves the correctness and usability of feature layer information in a simulator model obtained by meta-learning for the first time. Then, we propose an optimized Simulator Attack+ based on this discovery. Our optimization methods used in Simulator Attack+ include: (1) a feature attentional boosting module that uses the feature layer information of the simulator to enhance the attack and accelerate the generation of adversarial examples; (2) a linear self-adaptive simulator-predict interval mechanism that allows the simulator model to be fully fine-tuned in the early stage of the attack and dynamically adjusts the interval for querying the black-box model; and (3) an unsupervised clustering module to provide a warm-start for targeted attacks. Results from experiments on the CIFAR-10 and CIFAR-100 datasets clearly show that Simulator Attack+ can further reduce the number of consuming queries to improve query efficiency while maintaining the attack.

The transferability of adversarial examples allows the attacker to fool deep neural networks (DNNs) without knowing any information about the target models. The current input transformation-based method generates adversarial examples by transforming the image in the input space, which implicitly integrates a set of models by concatenating image transformation into the trained model. However, the input transformation-based methods ignore the manifold embedding and hardly extract intrinsic information from high-dimensional data. To this end, we propose a novel feature transformation-based method (FTM), which conducts feature transformation in the feature space. FTM can improve the robustness of adversarial example by transforming the features of data. Combining with FTM, the intrinsic features of adversarial examples are extracted to generate transferable adversarial examples. The experimental results on two benchmark datasets show that FTM could effectively improve the attack success rate (ASR) of the state-of-the-art (SOTA) methods. FTM improves the attack success rate of the Scale-Invariant Method on Inception_v3 from 62.6% to 75.1% on ImageNet, which is a large margin of 12.5%.

Copyright of Journal of Data Acquisition & Processing / Shu Ju Cai Ji Yu Chu Li is the property of Editorial Department of Journal of Nanjing University of Aeronautics & Astronautics and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The vulnerability of deep neural network (DNN)-based systems makes them susceptible to adversarial perturbation and may cause classification task failure. In this work, we propose an adversarial attack model using the Artificial Bee Colony (ABC) algorithm to generate adversarial samples without the need for a further gradient evaluation and training of the substitute model, which can further improve the chance of task failure caused by adversarial perturbation. In untargeted attacks, the proposed method obtained 100%, 98.6%, and 90.00% success rates on the MNIST, CIFAR-10 and ImageNet datasets, respectively. The experimental results show that the proposed ABCAttack can not only obtain a high attack success rate with fewer queries in the black-box setting, but also break some existing defenses to a large extent, and is not limited by model structure or size, which provides further research directions for deep learning evasion attacks and defenses.

Deep learning is at the heart of the current rise of artificial intelligence. In the field of computer vision, it has become the workhorse for applications ranging from self-driving cars to surveillance and security. Whereas, deep neural networks have demonstrated phenomenal success (often beyond human capabilities) in solving complex problems, recent studies show that they are vulnerable to adversarial attacks in the form of subtle perturbations to inputs that lead a model to predict incorrect outputs. For images, such perturbations are often too small to be perceptible, yet they completely fool the deep learning models. Adversarial attacks pose a serious threat to the success of deep learning in practice. This fact has recently led to a large influx of contributions in this direction. This paper presents the first comprehensive survey on adversarial attacks on deep learning in computer vision. We review the works that design adversarial attacks, analyze the existence of such attacks and propose defenses against them. To emphasize that adversarial attacks are possible in practical conditions, we separately review the contributions that evaluate adversarial attacks in the real-world scenarios. Finally, drawing on the reviewed literature, we provide a broader outlook of this research direction.

Black-box attacks against deep neural network (DNN) classifiers are receiving increasing attention because they represent a more practical approach in the real world than white box attacks. In black-box environments, adversaries have limited knowledge regarding the target model. This makes it difficult to estimate gradients for crafting adversarial examples, such that powerful white-box algorithms cannot be directly applied to black-box attacks. Therefore, a well-known black-box attack strategy creates local DNNs, called substitute models, to emulate the target model. The adversaries then craft adversarial examples using the substitute models instead of the unknown target model. The substitute models repeat the query process and are trained by observing labels from the target model's responses to queries. However, emulating a target model usually requires numerous queries because new DNNs are trained from the beginning. In this study, we propose a new training method for substitute models to minimize the number of queries. We consider the number of queries as an important factor for practical black-box attacks because real-world systems often restrict queries for security and financial purposes. To decrease the number of queries, the proposed method does not emulate the entire target model and only adjusts the partial classification boundary based on a current attack. Furthermore, it does not use queries in the pre-training phase and creates queries only in the retraining phase. The experimental results indicate that the proposed method is effective in terms of the number of queries and attack success ratio against MNIST, VGGFace2, and ImageNet classifiers in query-limited black-box environments. Further, we demonstrate a black-box attack against a commercial classifier, Google AutoML Vision. [ABSTRACT FROM AUTHOR]

Adversarial attacks have stimulated research interests in the field of deep learning security. However, most of existing adversarial attack methods are developed on classification. In this paper, we use Projected Gradient Descent (PGD), the strongest first-order attack method on classification, to produce adversarial examples on the total loss of Faster R-CNN object detector. Compared with the state-of-the-art Dense Adversary Generation (DAG) method, our attack is more efficient and more powerful in both white-box and black-box attack settings, and is applicable in a variety of neural network architectures. On Pascal VOC2007, under white-box attack, DAG has 5.92% mAP on Faster R-CNN with VGG16 backbone using 41.42 iterations on average, while our method achieves 0.90% using only 4 iterations. We also analyze the difference of attacks between classification and detection, and find that in addition to misclassification, adversarial examples on detection also lead to mis-localization. Besides, we validate the adversarial effectiveness of both Region Proposal Network (RPN) and Fast R-CNN loss, the components of the total loss. Our research will provide inspiration for further efforts in adversarial attacks on other vision tasks. [ABSTRACT FROM AUTHOR]

Adversarial attack is aimed to add small perturbations to the model that are imperceptible to humans, resulting in incorrect outputs with high confidence. Currently, adversarial attack mainly focuses on image classification and object detection tasks, but are insufficient in visual tracking. Nevertheless, existing attack methods for object tracking are limited to Siamese networks, with other types of trackers being infrequently targeted. In order to expand the usage of adversarial attacks in object tracking, we propose a model-free black-box framework for learning to generate universal and sparse adversarial examples (USAE) for tracking task. To this end, we first randomly add a noisy patch to any interference image, and then apply standard projected gradient descent to optimize the generation process of adversarial examples which is subjected to a similarity constraint with original images, making its embedding feature closer to the patched interference image in l 2 -norm. Consequently, there is no significant difference between adversarial images and original images for human vision, but leading to tracking failure. Furthermore, our method just attacks 50 original images with adversarial images in each sequence, rather than an entire dataset. Numerous experiments on VOT2018, OTB2013, OTB2015, and GOT-10k datasets verify the effectiveness of USAE attack. Specifically, the number of lost reaches 1180 on VOT2018, the precision of OTB2015 decreased by 42.1%, and the success rate of GOT-10k is reduced to 1.8%, which shows the attack effect is remarkable. Moreover, USAE has a good transferability among various trackers like SiamRPN++, ATOM, DiMP, KYS, and ToMP. Notice that the proposed method is black-box and applicable to most realistic scenarios. • We propose a universal and black-box adversarial example generation framework for video object tracking task. • Our USAE only requires changing a small part of the entire datasets, which is sparse and effective to attack the trackers. • We expand the USAE to Siamese trackers and beyond various types of trackers, demonstrating its generality and transferability. [ABSTRACT FROM AUTHOR]

Deep learning （DL） models have been widely applied to security-sensitivity tasks， such as auto-driving，etc. Attacks and defenses concerned with the DL have gradually become hot spots in the field of machine learning. The black box attack，as a typical attack type and the most common attack method in the real context，can still perform effective attacks without knowing the specific structure of the model and parameters. Therefore，a reasonable analysis of the vulnerability of the DL model and design of a more robust model against black-box attacks has become an emergent topic. Traditional single-strength and multi-strength adversarial training methods based on single-model are infeasible to resist black-box attacks. Ensemble adversarial training based on multi-model still fails to resist attack samples that are high-intensity and diversify. In order to solve this problem，the mixed adversarial training defense strategy based on greedy search strength is proposed. Experimental results show that the proposed defensive strategy has robustness faced with the diversified black box attacks， and superior performance compared to conventional adversarial training methods. [ABSTRACT FROM AUTHOR]

These years, Deep Neural Networks (DNNs) have shown unprecedented performance in many areas. However, some recent studies revealed their vulnerability to small perturbations added on source inputs. Furthermore, we call the ways to generate these perturbations' adversarial attacks, which contain two types, black-box and white-box attacks, according to the adversaries' access to target models. In order to overcome the problem of black-box attackers' unreachabilities to the internals of target DNN, many researchers put forward a series of strategies. Previous works include a method of training a local substitute model for the target black-box model via Jacobian-based augmentation and then use the substitute model to craft adversarial examples using white-box methods. In this work, we improve the dataset augmentation to make the substitute models better fit the decision boundary of the target model. Unlike the previous work that just performed the non-targeted attack, we make it first to generate targeted adversarial examples via training substitute models. Moreover, to boost the targeted attacks, we apply the idea of ensemble attacks to the substitute training. Experiments on MNIST and GTSRB, two common datasets for image classification, demonstrate our effectiveness and efficiency of boosting a targeted black-box attack, and we finally attack the MNIST and GTSRB classifiers with the success rates of 97.7% and 92.8%. [ABSTRACT FROM AUTHOR]

Black-box attacks against deep neural network (DNN) classifiers are receiving increasing attention because they represent a more practical approach in the real world than white box attacks. In black-box environments, adversaries have limited knowledge regarding the target model. This makes it difficult to estimate gradients for crafting adversarial examples, such that powerful white-box algorithms cannot be directly applied to black-box attacks. Therefore, a well-known black-box attack strategy creates local DNNs, called substitute models, to emulate the target model. The adversaries then craft adversarial examples using the substitute models instead of the unknown target model. The substitute models repeat the query process and are trained by observing labels from the target model’s responses to queries. However, emulating a target model usually requires numerous queries because new DNNs are trained from the beginning. In this study, we propose a new training method for substitute models to minimize the number of queries. We consider the number of queries as an important factor for practical black-box attacks because real-world systems often restrict queries for security and financial purposes. To decrease the number of queries, the proposed method does not emulate the entire target model and only adjusts the partial classification boundary based on a current attack. Furthermore, it does not use queries in the pre-training phase and creates queries only in the retraining phase. The experimental results indicate that the proposed method is effective in terms of the number of queries and attack success ratio against MNIST, VGGFace2, and ImageNet classifiers in query-limited black-box environments. Further, we demonstrate a black-box attack against a commercial classifier, Google AutoML Vision.

The emergence of visually realistic GAN-generated facial images has raised concerns regarding potential misuse. In response, effective forensic algorithms have been developed to detect such synthetic images in recent years. However, the vulnerability of such forensic detectors to adversarial attacks remains an important issue that requires further investigation. In this paper, we propose a new black-box attack method against GAN-generated image detectors. It involves contrastive learning strategy to train an encoder–decoder anti-forensic network with a contrastive loss function. GAN-generated and corresponding simulated real images are constructed as positive and negative samples, respectively. By leveraging the trained attack model, we can apply imperceptible perturbation to input synthetic images for removing GAN fingerprint to some extent. GAN-generated image detectors may be deceived consequently. Extensive experimental results demonstrate that the proposed attack effectively reduces the accuracy of three state-of-the-art detectors on six popular GANs, while also achieving high visual quality of the attacked images. The source code will be available at https://github.com/ZXMMD/BAttGAND. [ABSTRACT FROM AUTHOR]

These years, Deep Neural Networks (DNNs) have shown unprecedented performance in many areas. However, some recent studies revealed their vulnerability to small perturbations added on source inputs. Furthermore, we call the ways to generate these perturbations’ adversarial attacks, which contain two types, black-box and white-box attacks, according to the adversaries’ access to target models. In order to overcome the problem of black-box attackers’ unreachabilities to the internals of target DNN, many researchers put forward a series of strategies. Previous works include a method of training a local substitute model for the target black-box model via Jacobian-based augmentation and then use the substitute model to craft adversarial examples using white-box methods. In this work, we improve the dataset augmentation to make the substitute models better fit the decision boundary of the target model. Unlike the previous work that just performed the non-targeted attack, we make it first to generate targeted adversarial examples via training substitute models. Moreover, to boost the targeted attacks, we apply the idea of ensemble attacks to the substitute training. Experiments on MNIST and GTSRB, two common datasets for image classification, demonstrate our effectiveness and efficiency of boosting a targeted black-box attack, and we finally attack the MNIST and GTSRB classifiers with the success rates of 97.7% and 92.8%.

Deep neural networks that play a pivotal role in fields such as images, text, and audio are vulnerable to adversarial attacks. In current textual adversarial attacks, the vast majority are configured with a black-box soft-label which is achieved by the gradient information or confidence of the model. Therefore, it becomes challenging and realistic to implement adversarial attacks using only the predicted top labels of the hard-label model. Existing methods to implement hard-label adversarial attacks use population-based genetic optimization algorithms. However, this approach requires significant query consumption, which is a considerable shortcoming. To solve this problem, we propose a new textual black-box hard-label adversarial attack algorithm based on the idea of differential evolution of populations, called the text-based differential evolution (TDE) algorithm. First, the method will judge the importance of the words of the initial rough adversarial examples, according to which only the keywords in the text sentence will be operated, and the rest of the words will be gradually replaced with the original words so as to reduce the words in the sentence in which the replacement occurs. Our method judges the quality of semantic similarity of the adversarial examples in the replacement process and deposits high-quality adversarial example individuals into the population. Secondly, the optimization process of adversarial examples is combined and optimized according to the word importance. Compared with existing methods based on genetic algorithm guidance, our method avoids a large number of meaningless repetitive queries and significantly improves the overall attack efficiency of the algorithm and the semantic quality of the generated adversarial examples. We experimented with multiple datasets on three text tasks of sentiment classification, natural language inference, and toxic comment, and also perform experimental comparisons on models and APIs in realistic scenarios. For example, in the Google Cloud commercial API adversarial attack experiment, compared to the existing hard-label method, our method reduces the average number of queries required for the attack from 6986 to 176, and increases semantic similarity from 0.844 to 0.876. It is shown through extensive experimental data that our approach not only significantly reduces the number of queries, but also significantly outperforms existing methods in terms of the quality of adversarial examples. [ABSTRACT FROM AUTHOR]