Search

Adversarial attacks, in particular data poisoning, can affect the behavior of machine learning models by inserting deliberately designed data into the training set. This study proposes an approach for identifying data poisoning attacks on machine learning models, the Weighted Average Analysis (VWA) algorithm. This algorithm evaluates the weighted averages of the input features to detect any irregularities that could be signs of poisoning efforts. The method finds deviations that can indicate manipulation by adding all the weighted averages and comparing them with the predicted value. Furthermore, it differentiates between binary and multiclass classification instances, accordingly modifying its analysis. The experimental results showed that the VWA algorithm can accurately detect and mitigate data poisoning attacks and improve the robustness and security of machine learning systems against adversarial threats. [ABSTRACT FROM AUTHOR]

Since social media such as Facebook and X (formerly known as Twitter) have permeated various aspects of daily life, people have strong incentives to influence information dissemination on these platforms and differentiate their content from the fierce competition. Existing dissemination strategies typically employ marketing techniques, such as seeking publicity through renowned actors or targeted advertising placements. Despite their various forms, most simply spread information to strengthen user impressions without conducting formal analyses of specific influence enhancement. And coupled with high costs, most fall short of expectations. To this end, we ingeniously formulate the task of social media dissemination as poisoning attacks, which influence specified content's dissemination among target users by intervening in some users' social media behaviors (including retweeting, following, and profile modifying). Correspondingly, we propose a novel poisoning attack, Influence-based Social Media Attack (ISMA) to generate discrete poisoning behaviors, which is difficult to achieve with existing attacks. In ISMA, we first contribute an efficient influence evaluator to quantify the spread influence of poisoning behaviors. Based on the estimated influence, we then present an imperceptible hierarchical selector and a profile modification method ProMix to select influential behaviors to poison. Notably, our attack is driven by custom attack objectives, which allows one to flexibly design different optimization goals to change the information flow, which could solve the blindness of existing influence maximization methods. Besides, behaviors such as retweeting are gentle and simple to implement. These properties make our attack more cost-effective and practical. Extensive experiments on two large-scale real-world datasets demonstrate the superiority of our method as it significantly outperforms baselines, and additionally, the proposed evaluator's analysis of user influence provides new insights for influence maximization on social media. [ABSTRACT FROM AUTHOR]

Federated learning (FL) is a novel framework that distributes the model training to the participant devices to realize privacy-preserving machine learning. To achieve this, clients upload the parameters of the local model to the central server for aggregation rather than the raw data. Despite the potential of FL, one of the significant challenges in FL applications is the communication constraints caused by the transmission of the high-dimensional parameter. To overcome this, federated distillation (FD) has been widely studied to address the significant communication overhead through transmitting the low-dimensional logits, which is used to assist the training of the local model rather than transmitting the model parameters. However, the traditional FD framework applies the centralized architecture, which is vulnerable to single-point-of-failure. Moreover, the emerging poisoning attacks also significantly impact the security of FD. Specifically, attackers can easily launch poisoning attack by uploading crafted logits, leading to inaccurate global logits aggregation and hazard the accuracy of local models. To address these issues, we propose a federated distillation framework based on blockchain, named BlockFD, by exploiting two mechanisms in blockchain architecture to realize decentralized and security FD. First, we propose a novel multi-dimension consensus algorithm (BlockFD-PoM) that leverages multiple attributions to perform consensus process, solving the existing computation-intensive and unfair problems of traditional consensus algorithms, such as the PoW and the PoS. Second, we introduce an aggregation-based validation algorithm (BFV) such that the legitimacy of local logits can be verified to guarantee the security of FD aggregation. Extensive evaluation results show that the proposed BlockFD framework can effectively and fairly realize decentralized federated distillation. Besides that, the proposed BFV algorithm can efficiently prevent federated distillation from poisoning attacks while maintaining the loss within 2.77%. [ABSTRACT FROM AUTHOR]

Federated learning (FL) is widely used in edge‐cloud collaborative training due to its distributed architecture and privacy‐preserving properties without sharing local data. FLTrust, the most state‐of‐the‐art FL defense method, is a federated learning defense system with trust guidance. However, we found that FLTrust is not very robust. Therefore, in the edge collaboration scenario, we mainly study the poisoning attack on the FLTrust defense system. Due to the aggregation rule, FLTrust, with trust guidance, the model updates of participants with a significant deviation from the root gradient direction will be eliminated, which makes the poisoning effect on the global model not obvious. To solve this problem, under the premise of not being deleted by the FLTrust aggregation rules, we construct malicious model updates that deviate from the trust gradient to the greatest extent to achieve model poisoning attacks. First, we utilize the rotation of high‐dimensional vectors around axes to construct malicious vectors with fixed orientations. Second, the malicious vector is constructed by the gradient inversion method to achieve an efficient and fast attack. Finally, a method of optimizing random noise is used to construct a malicious vector with a fixed direction. Experimental results show that our attack method reduces the model accuracy by 20%, severely undermining the usability of the model. Attacks are also successful hundreds of times faster than the FLTrust adaptive attack method. [ABSTRACT FROM AUTHOR]

In this study, we introduce a novel collaborative federated learning (FL) framework, aiming at enhancing robustness in distributed learning environments, particularly pertinent to IoT and industrial automation scenarios. At the core of our contribution is the development of an innovative grouping algorithm for edge clients. This algorithm employs a distinctive ID distribution function, enabling efficient and secure grouping of both normal and potentially malicious clients. Our proposed grouping scheme accurately determines the numerical difference between normal and malicious groups under various network scenarios. Our method addresses the challenge of model poisoning attacks, ensuring the accuracy of outcomes in a collaborative federated learning framework. Our numerical experiments demonstrate that our grouping scheme effectively limits the number of malicious groups. Additionally, our collaborative FL framework has shown resilience against various levels of poisoning attack abilities and maintained high prediction accuracy across a range of scenarios, showcasing its robustness against poisoning attacks. [ABSTRACT FROM AUTHOR]

The rapid expansion and pervasive reach of the internet in recent years have raised concerns about evolving and adaptable online threats, particularly with the extensive integration of Machine Learning (ML) systems into our daily routines. These systems are increasingly becoming targets of malicious attacks that seek to distort their functionality through the concept of poisoning. Such attacks aim to warp the intended operations of these services, deviating them from their true purpose. Poisoning renders systems susceptible to unauthorized access, enabling illicit users to masquerade as legitimate ones, compromising the integrity of smart technology-based systems like Network Intrusion Detection Systems (NIDSs). Therefore, it is necessary to continue working on studying the resilience of deep learning network systems while there are poisoning attacks, specifically interfering with the integrity of data conveyed over networks. This paper explores the resilience of deep learning (DL)—based NIDSs against untethered white-box attacks. More specifically, it introduces a designed poisoning attack technique geared especially for deep learning by adding various amounts of altered instances into training datasets at diverse rates and then investigating the attack's influence on model performance. We observe that increasing injection rates (from 1% to 50%) and random amplified distribution have slightly affected the overall performance of the system, which is represented by accuracy (0.93) at the end of the experiments. However, the rest of the results related to the other measures, such as PPV (0.082), FPR (0.29), and MSE (0.67), indicate that the data manipulation poisoning attacks impact the deep learning model. These findings shed light on the vulnerability of DL-based NIDS under poisoning attacks, emphasizing the significance of securing such systems against these sophisticated threats, for which defense techniques should be considered. Our analysis, supported by experimental results, shows that the generated poisoned data have significantly impacted the model performance and are hard to be detected. [ABSTRACT FROM AUTHOR]

Adversarial attacks, in particular data poisoning, can affect the behavior of machine learning models by inserting deliberately designed data into the training set. This study proposes an approach for identifying data poisoning attacks on machine learning models, the Weighted Average Analysis (VWA) algorithm. This algorithm evaluates the weighted averages of the input features to detect any irregularities that could be signs of poisoning efforts. The method finds deviations that can indicate manipulation by adding all the weighted averages and comparing them with the predicted value. Furthermore, it differentiates between binary and multiclass classification instances, accordingly modifying its analysis. The experimental results showed that the VWA algorithm can accurately detect and mitigate data poisoning attacks and improve the robustness and security of machine learning systems against adversarial threats.

Abstract Vertex classification using graph convolutional networks is susceptible to targeted poisoning attacks, in which both graph structure and node attributes can be changed in an attempt to misclassify a target node. This vulnerability decreases users' confidence in the learning method and can prevent adoption in high-stakes contexts. Defenses have been proposed, focused on filtering edges before creating the model or aggregating information from neighbors more robustly. This paper considers an alternative: we investigate the ability to exploit network phenomena in the training data selection process to improve classifier robustness. We propose two alternative methods of selecting training data: (1) to select the highest-degree nodes and (2) to select nodes with many connections to the test data. In four real datasets, we show that changing the training set often results in far more perturbations required for a successful attack on the graph structure; often a factor of 2 over the random training baseline. We also run a simulation study in which we demonstrate conditions under which the proposed methods outperform random selection, finding that they improve performance most when homophily is higher, clustering coefficient is higher, node degrees are more homogeneous, and attributes are less informative. In addition, we show that the methods are effective when applied to adaptive attacks, alleviating concerns about generalizability.

Federated learning (FL) is considered a de facto standard for privacy preservation in AI environments because it does not require data to be aggregated in some central place to train an AI model. Preserving data on the client side and sharing only the model’s parameters with a central server preserves privacy while training an AI model of higher generalizability. Unfortunately, sharing the model’s parameters with the server can create privacy leaks, and therefore, FL is unable to meet privacy requirements in many situations. Furthermore, FL is prone to other technical issues, such as data poisoning, model poisoning, fairness, client dropout, and convergence issues, to name just a few. In this work, we provide a multifaceted survey on FL, including its fundamentals, paradigm shifts, technical issues, recent developments, and future prospects. First, we discuss the fundamental concepts of FL (workflow, categorization, the differences between centralized learning and FL, and applications of FL in diverse fields), and we then discuss the paradigm shifts brought on by FL from a broader perspective (e.g., data use, AI model development, resource sharing, etc.). Later, we pinpoint ten practical issues currently hindering the viability of the FL landscape, and we discuss developments made under each issue by summarizing state-of-the-art (SOTA) literature. We highlight FL partnerships with two or more technologies that either improve practical aspects/issues in FL or extend its adoption to new areas/domains. We pinpoint various trade-offs that exist in an FL ecosystem, and the corresponding SOTA developments to mitigate them. We also discuss the latest studies that have been proposed to make FL trustworthy and beneficial for the community. Lastly, we suggest valuable research directions towards enhancing technical efficacy by guiding researchers to less explored topics in FL.

Federated learning (FL) provides autonomy and privacy by design to participating peers, who cooperatively build a machine learning (ML) model while keeping their private data in their devices. However, that same autonomy opens the door for malicious peers to poison the model by conducting either untargeted or targeted poisoning attacks. The label-flipping (LF) attack is a targeted poisoning attack where the attackers poison their training data by flipping the labels of some examples from one class (i.e., the source class) to another (i.e., the target class). Unfortunately, this attack is easy to perform and hard to detect, and it negatively impacts the performance of the global model. Existing defenses against LF are limited by assumptions on the distribution of the peers' data and/or do not perform well with high-dimensional models. In this paper, we deeply investigate the LF attack behavior. We find that the contradicting objectives of attackers and honest peers on the source class examples are reflected on the parameter gradients corresponding to the neurons of the source and target classes in the output layer. This makes those gradients good discriminative features for the attack detection. Accordingly, we propose LFighter, a novel defense against the LF attack that first dynamically extracts those gradients from the peers' local updates and then clusters the extracted gradients, analyzes the resulting clusters, and filters out potential bad updates before model aggregation. Extensive empirical analysis on three data sets shows the effectiveness of the proposed defense regardless of the data distribution or model dimensionality. Also, LFighter outperforms several state-of-the-art defenses by offering lower test error, higher overall accuracy, higher source class accuracy, lower attack success rate, and higher stability of the source class accuracy. Our code and data are available for reproducibility purposes at https://github.com/NajeebJebreel/LFighter. [ABSTRACT FROM AUTHOR]

Poisoning attack in deep learning (DL) refers to a type of adversarial attack that injects maliciously manipulated data samples into a training dataset for the purpose of forcing a DL model trained based on the poisoned training dataset to misclassify inputs and thus significantly degrading its performance and reliability. Meanwhile, a traditional defense approach against poisoning attacks tries to detect poisoned data samples from the training dataset and then remove them. However, since new sophisticated attacks avoiding existing detection methods continue to emerge, a detection method alone cannot effectively counter poisoning attacks. For this reason, in this paper, we propose a novel dilution-based defense method that mitigates the effect of poisoned data by adding clean data to the training dataset. According to our experiments, our dilution-based defense technique can significantly decrease the success rate of poisoning attacks and improve classification accuracy by effectively reducing the contamination ratio of the manipulated data. Especially, our proposed method outperformed an existing defense method (Cutmix data augmentation) by 20.9%p at most in terms of classification accuracy. [ABSTRACT FROM AUTHOR]

As computer networks become increasingly important in various domains, the need for secure and reliable networks becomes more pressing, particularly in the context of blockchain-enabled supply chain networks. One way to ensure network security is by using intrusion detection systems (IDSs), which are specialised devices that detect anomalies and attacks in the network. However, these systems are vulnerable to data poisoning attacks, such as label and distance-based flipping, which can undermine their effectiveness within blockchain-enabled supply chain networks. In this research paper, we investigate the effect of these attacks on a network intrusion detection system using several machine learning models, including logistic regression, random forest, SVC, and XGB Classifier, and evaluate each model via their F1 Score, confusion matrix, and accuracy. We run each model three times: once without any attack, once with random label flipping with a randomness of 20%, and once with distance-based label flipping attacks with a distance threshold of 0.5. Additionally, this research tests an eight-layer neural network using accuracy metrics and a classification report library. The primary goal of this research is to provide insights into the effect of data poisoning attacks on machine learning models within the context of blockchain-enabled supply chain networks. By doing so, we aim to contribute to developing more robust intrusion detection systems tailored to the specific challenges of securing blockchain-based supply chain networks. [ABSTRACT FROM AUTHOR]

Federated learning (FL) can push the limitation of "Data Island" while protecting data privacy has been a broad concern. However, the centralised FL is vulnerable to a single-point failure. While decentralised and tamper-proof blockchains can cope with the above issues, it is difficult to find a benign benchmark gradient and eliminate the poisoning attack in the later stage of global model aggregation. To address the above problems, we present a global to local based privacy-preserving federated consensus scheme against poisoning attacks (FedG2L). This scheme can effectively reduce the influence of poisoning attacks on model accuracy. In the global aggregation stage, a gradient-similarity-based secure consensus algorithm (SecPBFT) is designed to eliminate malicious gradients. During this procedure, the gradient of the data owner will not be leaked. Then, we propose an improved ACGAN algorithm to generate local data to further update the model without poisoning attacks. Finally, we theoretically prove the security and correctness of our scheme. Experimental results demonstrated that the model accuracy is improved by at least 55% than no defense scheme, and the attack success rate is reduced by more than 60%. [ABSTRACT FROM AUTHOR]

Recommender systems are often susceptible to well-crafted fake profiles, leading to biased recommendations. Among existing defense methods, data-processing-based methods inevitably exclude normal samples, while model-based methods struggle to enjoy both generalization and robustness. To this end, we suggest integrating data processing and the robust model to propose a general framework, Triple Cooperative Defense (TCD), which employs three cooperative models that mutually enhance data and thereby improve recommendation robustness. Furthermore, Considering that existing attacks struggle to balance bi-level optimization and efficiency, we revisit poisoning attacks in recommender systems and introduce an efficient attack strategy, Co-training Attack (Co-Attack), which cooperatively optimizes the attack optimization and model training, considering the bi-level setting while maintaining attack efficiency. Moreover, we reveal a potential reason for the insufficient threat of existing attacks is their default assumption of optimizing attacks in undefended scenarios. This overly optimistic setting limits the potential of attacks. Consequently, we put forth a Game-based Co-training Attack (GCoAttack), which frames the proposed CoAttack and TCD as a game-theoretic process, thoroughly exploring CoAttack's attack potential in the cooperative training of attack and defense. Extensive experiments on three real datasets demonstrate TCD's superiority in enhancing model robustness. Additionally, we verify that the two proposed attack strategies significantly outperform existing attacks, with game-based GCoAttack posing a greater poisoning threat than CoAttack. [ABSTRACT FROM AUTHOR]

Machine learning (ML) models are vulnerable to adversarial machine learning (AML) attacks. Unlike other contexts, the fraud detection domain is characterized by inherent challenges that make conventional approaches hardly applicable. In this article, we extend the application of AML techniques to the fraud detection task by studying poisoning attacks and their possible countermeasures. First, we present a novel approach for performing poisoning attacks that overcomes the fraud detection domain-specific constraints. It generates fraudulent candidate transactions and tests them against a machine learning-based Oracle, which simulates the target fraud detection system aiming at evading it. Misclassified fraudulent candidate transactions are then integrated into the target detection system's training set, poisoning its model and shifting its decision boundary. Second, we propose a novel approach that extends the adversarial training technique to mitigate AML attacks: During the training phase of the detection system, we generate artificial frauds by modifying random original legitimate transactions; then, we include them in the training set with the correct label. By doing so, we instruct our model to recognize evasive transactions before an attack occurs. Using two real bank datasets, we evaluate the security of several state-of-the-art fraud detection systems by deploying our poisoning attack with different degrees of attacker's knowledge and attacking strategies. The experimental results show that our attack works even when the attacker has minimal knowledge of the target system. Then, we demonstrate that the proposed countermeasure can mitigate adversarial attacks by reducing the stolen amount of money up to 100%. [ABSTRACT FROM AUTHOR]

Continuous learning from streaming data is among the most challenging topics in the contemporary machine learning. In this domain, learning algorithms must not only be able to handle massive volume of rapidly arriving data, but also adapt themselves to potential emerging changes. The phenomenon of evolving nature of data streams is known as concept drift. While there is a plethora of methods designed for detecting its occurrence, all of them assume that the drift is connected with underlying changes in the source of data. However, one must consider the possibility of a malicious injection of false data that simulates a concept drift. This adversarial setting assumes a poisoning attack that may be conducted in order to damage the underlying classification system by forcing an adaptation to false data. Existing drift detectors are not capable of differentiating between real and adversarial concept drift. In this paper, we propose a framework for robust concept drift detection in the presence of adversarial and poisoning attacks. We introduce the taxonomy for two types of adversarial concept drifts, as well as a robust trainable drift detector. It is based on the augmented restricted Boltzmann machine with improved gradient computation and energy function. We also introduce Relative Loss of Robustness—a novel measure for evaluating the performance of concept drift detectors under poisoning attacks. Extensive computational experiments, conducted on both fully and sparsely labeled data streams, prove the high robustness and efficacy of the proposed drift detection framework in adversarial scenarios. [ABSTRACT FROM AUTHOR]

The rapid expansion and pervasive reach of the internet in recent years have raised concerns about evolving and adaptable online threats, particularly with the extensive integration of Machine Learning (ML) systems into our daily routines. These systems are increasingly becoming targets of malicious attacks that seek to distort their functionality through the concept of poisoning. Such attacks aim to warp the intended operations of these services, deviating them from their true purpose. Poisoning renders systems susceptible to unauthorized access, enabling illicit users to masquerade as legitimate ones, compromising the integrity of smart technology-based systems like Network Intrusion Detection Systems (NIDSs). Therefore, it is necessary to continue working on studying the resilience of deep learning network systems while there are poisoning attacks, specifically interfering with the integrity of data conveyed over networks. This paper explores the resilience of deep learning (DL)—based NIDSs against untethered white-box attacks. More specifically, it introduces a designed poisoning attack technique geared especially for deep learning by adding various amounts of altered instances into training datasets at diverse rates and then investigating the attack’s influence on model performance. We observe that increasing injection rates (from 1% to 50%) and random amplified distribution have slightly affected the overall performance of the system, which is represented by accuracy (0.93) at the end of the experiments. However, the rest of the results related to the other measures, such as PPV (0.082), FPR (0.29), and MSE (0.67), indicate that the data manipulation poisoning attacks impact the deep learning model. These findings shed light on the vulnerability of DL-based NIDS under poisoning attacks, emphasizing the significance of securing such systems against these sophisticated threats, for which defense techniques should be considered. Our analysis, supported by experimental results, shows that the generated poisoned data have significantly impacted the model performance and are hard to be detected.

In this study, we introduce a novel collaborative federated learning (FL) framework, aiming at enhancing robustness in distributed learning environments, particularly pertinent to IoT and industrial automation scenarios. At the core of our contribution is the development of an innovative grouping algorithm for edge clients. This algorithm employs a distinctive ID distribution function, enabling efficient and secure grouping of both normal and potentially malicious clients. Our proposed grouping scheme accurately determines the numerical difference between normal and malicious groups under various network scenarios. Our method addresses the challenge of model poisoning attacks, ensuring the accuracy of outcomes in a collaborative federated learning framework. Our numerical experiments demonstrate that our grouping scheme effectively limits the number of malicious groups. Additionally, our collaborative FL framework has shown resilience against various levels of poisoning attack abilities and maintained high prediction accuracy across a range of scenarios, showcasing its robustness against poisoning attacks.

Smart farming, as a branch of the Internet of Things (IoT), combines the recognition of agricultural economic competencies and the progress of data and information collected from connected devices with statistical analysis to characterize the essentials of the assimilated information, allowing farmers to make intelligent conclusions that will maximize the harvest benefit. However, the integration of advanced technologies requires the adoption of high-tech security approaches. In this paper, we present a framework that promises to enhance the security and privacy of smart farms by leveraging the decentralized nature of blockchain technology. The framework stores and manages data acquired from IoT devices installed in smart farms using a distributed ledger architecture, which provides secure and tamper-proof data storage and ensures the integrity and validity of the data. The study uses the AWS cloud, ESP32, the smart farm security monitoring framework, and the Ethereum Rinkeby smart contract mechanism, which enables the automated execution of pre-defined rules and regulations. As a result of a proof-of-concept implementation, the system can detect and respond to security threats in real time, and the results illustrate its usefulness in improving the security of smart farms. The number of accepted blockchain transactions on smart farming requests fell from 189,000 to 109,450 after carrying out the first three tests while the next three testing phases showed a rise in the number of blockchain transactions accepted on smart farming requests from 176,000 to 290,786. We further observed that the lesser the time taken to induce the device alarm, the higher the number of blockchain transactions accepted on smart farming requests, which demonstrates the efficacy of blockchain-based poisoning attack mitigation in smart farming. [ABSTRACT FROM AUTHOR]

Machine learning (ML) has been universally adopted for automated decisions in a variety of fields, including recognition and classification applications, recommendation systems, natural language processing, and so on. However, in light of high expenses on training data and computing resources, recent years have witnessed a rapid increase in outsourced ML training, either partially or completely, which provides vulnerabilities for adversaries to exploit. A prime threat in training phase is called poisoning attack, where adversaries strive to subvert the behavior of machine learning systems by poisoning training data or other means of interference. Although a growing number of relevant studies have been proposed, the research among poisoning attack is still overly scattered, with each paper focusing on a particular task in a specific domain. In this survey, we summarize and categorize existing attack methods and corresponding defenses, as well as demonstrate compelling application scenarios, thus providing a unified framework to analyze poisoning attacks. Besides, we also discuss the main limitations of current works, along with the corresponding future directions to facilitate further researches. Our ultimate motivation is to provide a comprehensive and self-contained survey of this growing field of research and lay the foundation for a more standardized approach to reproducible studies. [ABSTRACT FROM AUTHOR]

Local differential privacy (LDP) is a promising technique to realize privacy-preserving data aggregation without a trusted aggregator. Normally, an LDP protocol requires each user to locally perturb his raw data and submit the perturbed data to the aggregator. Consequently, LDP is vulnerable to output poisoning attacks. Malicious users can skip the perturbation and submit carefully crafted data to the aggregator, altering the data aggregation results. Existing verifiable LDP protocols, which can verify the perturbation process and prevent output poisoning attacks, usually incur significant computation and communication costs, due to the use of zero-knowledge proofs. In this paper, we analyze the attacks on two classic LDP protocols for frequency estimation, namely GRR and OUE, and propose two verifiable LDP protocols. The proposed protocols are based on an interactive framework, where the user and the aggregator complete the perturbation together. By providing some additional information, which reveals nothing about the raw data but helps the verification, the user can convince the aggregator that he is incapable of launching an output poisoning attack. Simulation results demonstrate that the proposed protocols have good defensive performance and outperform existing approaches in terms of efficiency. [ABSTRACT FROM AUTHOR]

Metaverse is envisioned to be the next-generation human-centric Internet which can offer an immersive experience for users with a broad application in healthcare, education, entertainment, and industries. These applications require the analysis of massive data that contains private and sensitive information. A potential solution to preserving privacy is deploying distributed learning frameworks, including federated learning (FL) and split learning (SL), due to their ability to address privacy leakage and analyze personalised data without sharing raw data. However, it is known that FL and SL are still susceptible to adversarial poisoning attacks. In this paper, we analyse such critical issues for the privacy-preserving mechanism in Metaverse services. We develop a novel poisoning attack based on Bayesian optimisation to emulate the adversarial behaviour against FL (BO-FLPA) and SL (BO-SLPA) which is important for the development of effective defense algorithms in the future. Specifically, we develop a layer optimisation method using the intuition of black-box optimisation with assuming that there is a function between the prediction’s uncertainty and layer optimisation parameters. The result of this optimisation provides the optimal weight parameters for the hidden layer, such as the first or the second layer for FL, and the first layer for SL. Numerical results demonstrate that in both FL and SL, the poisoned hidden layers have the ability to increase the susceptibility of the model to adversarial attacks in terms of prediction with low confidence or having a larger deviation of the probability density function of the predictions.

Federated learning faces many security and privacy issues. Among them, poisoning attacks can significantly impact global models, and malicious attackers can prevent global models from converging or even manipulating the prediction results of global models. Defending against poisoning attacks is a very urgent and challenging task. However, the systematic reviews of poisoning attacks and their corresponding defense strategies from a privacy-preserving perspective still need more effort. This survey provides an in-depth and up-to-date overview of poisoning attacks and corresponding defense strategies in federated learning. We first classify the poisoning attacks according to their methods and targets. Next, we analyze the differences and connections between the various categories of poisoning attacks. In addition, we classify the defense strategies against poisoning attacks in federated learning into three categories and analyze their advantages and disadvantages. Finally, we discuss the privacy protection problem in poisoning attacks and their countermeasure and propose potential research directions from the perspective of attack and defense, respectively.

Deep Learning (DL) has been increasingly deployed in various real-world applications due to its unprecedented performance and automated capability of learning hidden representations. While DL can achieve high task performance, the training process of a DL model is both time- and resource-consuming. Therefore, current supply chains of the DL models assume the customers obtain pre-trained Deep Neural Networks (DNNs) from the third-party providers that have sufficient computing power. In the centralized setting, the model designer trains the DL model using the local dataset. However, the collected training data may contain erroneous or poisoned data points. The model designer might craft malicious training samples and inject a backdoor in the DL model distributed to the users. As a result, the user’s model will malfunction. In the federated learning setting, the cloud server aggregates local models trained on individual local datasets and updates the global model. In this scenario, the local client could poison the local training set and/or arbitrarily manipulate the local update. If the cloud server incorporates the malicious local gradients in model aggregation, the resulting global model will have degraded performance or backdoor behaviors. In this article, we present a comprehensive overview of contemporary data poisoning and model poisoning attacks against DL models in both centralized and federated learning scenarios. In addition, we review existing detection and defense techniques against various poisoning attacks. [ABSTRACT FROM AUTHOR]

Federated Learning (FL) is an emerging distributed framework that enables clients to conduct distributed learning and globally share models without requiring data to leave the local. In the FL process, participants are required to contribute data resources and computing resources for model training. However, the traditional FL lacks security guarantees and is vulnerable to attacks and damages by malicious adversaries. In addition, the existing incentive methods lack fairness to participants. Therefore, accurately identifying and preventing malicious nodes from doing evil, while effectively selecting and incentivizing participants, plays a vital role in improving the security and performance of FL. In this paper, we propose a Robust Federated Learning Based on Malicious Detection and Incentives (MDIFL). Specifically, MDIFL first uses a gradient similarity to calculate reputation, thereby maintaining the reputation of participants and identifying malicious opponents, and then designs an effective incentive mechanism based on contract theory to achieve collaborative fairness. Extensive experimental results demonstrate that the proposed MDIFL can not only preferentially select and effectively motivate high-quality participants, but also correctly identify malicious adversaries, achieve fairness, and improve model performance. [ABSTRACT FROM AUTHOR]

Deep learning pervades heavy data-driven disciplines in research and development. The Internet of Things and sensor systems, which enable smart environments and services, are settings where deep learning can provide invaluable utility. However, the data in these systems are very often directly or indirectly related to people, which raises privacy concerns. Federated learning (FL) mitigates some of these concerns and empowers deep learning in sensor-driven environments by enabling multiple entities to collaboratively train a machine learning model without sharing their data. Nevertheless, a number of works in the literature propose attacks that can manipulate the model and disclose information about the training data in FL. As a result, there has been a growing belief that FL is highly vulnerable to severe attacks. Although these attacks do indeed highlight security and privacy risks in FL, some of them may not be as effective in production deployment because they are feasible only given special—sometimes impractical—assumptions. In this paper, we investigate this issue by conducting a quantitative analysis of the attacks against FL and their evaluation settings in 48 papers. This analysis is the first of its kind to reveal several research gaps with regard to the types and architectures of target models. Additionally, the quantitative analysis allows us to highlight unrealistic assumptions in some attacks related to the hyper-parameters of the model and data distribution. Furthermore, we identify fallacies in the evaluation of attacks which raise questions about the generalizability of the conclusions. As a remedy, we propose a set of recommendations to promote adequate evaluations. [ABSTRACT FROM AUTHOR]

Malicious software (malware) is a major cyber threat that has to be tackled with Machine Learning (ML) techniques because millions of new malware examples are injected into cyberspace on a daily basis. However, ML is vulnerable to attacks known as adversarial examples. In this article, we survey and systematize the field of Adversarial Malware Detection (AMD) through the lens of a unified conceptual framework of assumptions, attacks, defenses, and security properties. This not only leads us to map attacks and defenses to partial order structures, but also allows us to clearly describe the attack-defense arms race in the AMD context. We draw a number of insights, including: knowing the defender’s feature set is critical to the success of transfer attacks; the effectiveness of practical evasion attacks largely depends on the attacker’s freedom in conducting manipulations in the problem space; knowing the attacker’s manipulation set is critical to the defender’s success; and the effectiveness of adversarial training depends on the defender’s capability in identifying the most powerful attack. We also discuss a number of future research directions. [ABSTRACT FROM AUTHOR]

As computer networks become increasingly important in various domains, the need for secure and reliable networks becomes more pressing, particularly in the context of blockchain-enabled supply chain networks. One way to ensure network security is by using intrusion detection systems (IDSs), which are specialised devices that detect anomalies and attacks in the network. However, these systems are vulnerable to data poisoning attacks, such as label and distance-based flipping, which can undermine their effectiveness within blockchain-enabled supply chain networks. In this research paper, we investigate the effect of these attacks on a network intrusion detection system using several machine learning models, including logistic regression, random forest, SVC, and XGB Classifier, and evaluate each model via their F1 Score, confusion matrix, and accuracy. We run each model three times: once without any attack, once with random label flipping with a randomness of 20%, and once with distance-based label flipping attacks with a distance threshold of 0.5. Additionally, this research tests an eight-layer neural network using accuracy metrics and a classification report library. The primary goal of this research is to provide insights into the effect of data poisoning attacks on machine learning models within the context of blockchain-enabled supply chain networks. By doing so, we aim to contribute to developing more robust intrusion detection systems tailored to the specific challenges of securing blockchain-based supply chain networks.