Search

Model extraction aims to create a functionally similar copy from a machine learning as a service (MLaaS) API with minimal overhead, typically for illicit profit or as a precursor to further attacks, posing a significant threat to the MLaaS ecosystem. However, recent studies have shown that model extraction is highly inefficient, particularly when the target task distribution is unavailable. In such cases, even substantially increasing the attack budget fails to produce a sufficiently similar replica, reducing the adversary's motivation to pursue extraction attacks. In this paper, we revisit the elementary design choices throughout the extraction lifecycle. We propose an embarrassingly simple yet dramatically effective algorithm, Efficient and Effective Model Extraction (E3), focusing on both query preparation and training routine. E3 achieves superior generalization compared to state-of-the-art methods while minimizing computational costs. For instance, with only 0.005 times the query budget and less than 0.2 times the runtime, E3 outperforms classical generative model based data-free model extraction by an absolute accuracy improvement of over 50% on CIFAR-10. Our findings underscore the persistent threat posed by model extraction and suggest that it could serve as a valuable benchmarking algorithm for future security evaluations.

$\textit{R}$Mn$_6$Sn$_6$ ($R=$~Y,~Gd$-$Lu) kagome metals are promising materials hosting flat electronic bands and Dirac points that interact with magnetism. The coupling between the two magnetic $R$ and Mn sublattices can drive complex magnetic states with potential consequences for spin and charge transport and other topological properties. Here, we use a detailed magnetic Hamiltonian to calculate and predict the magnetic phase diagrams for $\textit{R}$Mn$_6$Sn$_6$ kagome metals within the mean-field approximation. These calculations reveal a variety of collinear, non-collinear, and non-coplanar phases that arise from competition between various interlayer magnetic exchange interactions and magnetic anisotropies of the $R$ and Mn ions. We enumerate these phases and their magnetic space groups for future analysis of their impact on topological and trivial bands near the Fermi surface., Comment: 18 pages, 10 figures

With the rise of Machine Learning as a Service (MLaaS) platforms,safeguarding the intellectual property of deep learning models is becoming paramount. Among various protective measures, trigger set watermarking has emerged as a flexible and effective strategy for preventing unauthorized model distribution. However, this paper identifies an inherent flaw in the current paradigm of trigger set watermarking: evasion adversaries can readily exploit the shortcuts created by models memorizing watermark samples that deviate from the main task distribution, significantly impairing their generalization in adversarial settings. To counteract this, we leverage diffusion models to synthesize unrestricted adversarial examples as trigger sets. By learning the model to accurately recognize them, unique watermark behaviors are promoted through knowledge injection rather than error memorization, thus avoiding exploitable shortcuts. Furthermore, we uncover that the resistance of current trigger set watermarking against removal attacks primarily relies on significantly damaging the decision boundaries during embedding, intertwining unremovability with adverse impacts. By optimizing the knowledge transfer properties of protected models, our approach conveys watermark behaviors to extraction surrogates without aggressively decision boundary perturbation. Experimental results on CIFAR-10/100 and Imagenette datasets demonstrate the effectiveness of our method, showing not only improved robustness against evasion adversaries but also superior resistance to watermark removal attacks compared to state-of-the-art solutions.

To trace the copyright of deep neural networks, an owner can embed its identity information into its model as a watermark. The capacity of the watermark quantify the maximal volume of information that can be verified from the watermarked model. Current studies on capacity focus on the ownership verification accuracy under ordinary removal attacks and fail to capture the relationship between robustness and fidelity. This paper studies the capacity of deep neural network watermarks from an information theoretical perspective. We propose a new definition of deep neural network watermark capacity analogous to channel capacity, analyze its properties, and design an algorithm that yields a tight estimation of its upper bound under adversarial overwriting. We also propose a universal non-invasive method to secure the transmission of the identity message beyond capacity by multiple rounds of ownership verification. Our observations provide evidence for neural network owners and defenders that are curious about the tradeoff between the integrity of their ownership and the performance degradation of their products., Comment: Accepted by AAAI 2024

Large language models (LLMs) have exhibited great potential in autonomously completing tasks across real-world applications. Despite this, these LLM agents introduce unexpected safety risks when operating in interactive environments. Instead of centering on LLM-generated content safety in most prior studies, this work addresses the imperative need for benchmarking the behavioral safety of LLM agents within diverse environments. We introduce R-Judge, a benchmark crafted to evaluate the proficiency of LLMs in judging and identifying safety risks given agent interaction records. R-Judge comprises 162 records of multi-turn agent interaction, encompassing 27 key risk scenarios among 7 application categories and 10 risk types. It incorporates human consensus on safety with annotated safety labels and high-quality risk descriptions. Evaluation of 9 LLMs on R-Judge shows considerable room for enhancing the risk awareness of LLMs: The best-performing model, GPT-4, achieves 72.52% in contrast to the human score of 89.07%, while all other models score less than the random. Moreover, further experiments demonstrate that leveraging risk descriptions as environment feedback achieves substantial performance gains. With case studies, we reveal that correlated to parameter amount, risk awareness in open agent scenarios is a multi-dimensional capability involving knowledge and reasoning, thus challenging for current LLMs. R-Judge is publicly available at https://github.com/Lordog/R-Judge.

Backdoor-based watermarking schemes were proposed to protect the intellectual property of artificial intelligence models, especially deep neural networks, under the black-box setting. Compared with ordinary backdoors, backdoor-based watermarks need to digitally incorporate the owner's identity, which fact adds extra requirements to the trigger generation and verification programs. Moreover, these concerns produce additional security risks after the watermarking scheme has been published for as a forensics tool or the owner's evidence has been eavesdropped on. This paper proposes the capsulation attack, an efficient method that can invalidate most established backdoor-based watermarking schemes without sacrificing the pirated model's functionality. By encapsulating the deep neural network with a rule-based or Bayes filter, an adversary can block ownership probing and reject the ownership verification. We propose a metric, CAScore, to measure a backdoor-based watermarking scheme's security against the capsulation attack. This paper also proposes a new backdoor-based deep neural network watermarking scheme that is secure against the capsulation attack by reversing the encoding process and randomizing the exposure of triggers.

Federated learning (FL) has enabled global model training on decentralized data in a privacy-preserving way by aggregating model updates. However, for many natural language processing (NLP) tasks that utilize pre-trained language models (PLMs) with large numbers of parameters, there are considerable communication costs associated with FL. Recently, prompt tuning, which tunes some soft prompts without modifying PLMs, has achieved excellent performance as a new learning paradigm. Therefore we want to combine the two methods and explore the effect of prompt tuning under FL. In this paper, we propose "FedPrompt" to study prompt tuning in a model split aggregation way using FL, and prove that split aggregation greatly reduces the communication cost, only 0.01% of the PLMs' parameters, with little decrease on accuracy both on IID and Non-IID data distribution. This improves the efficiency of FL method while also protecting the data privacy in prompt tuning. In addition, like PLMs, prompts are uploaded and downloaded between public platforms and personal users, so we try to figure out whether there is still a backdoor threat using only soft prompts in FL scenarios. We further conduct backdoor attacks by data poisoning on FedPrompt. Our experiments show that normal backdoor attack can not achieve a high attack success rate, proving the robustness of FedPrompt. We hope this work can promote the application of prompt in FL and raise the awareness of the possible security threats.

Watermarking has become a plausible candidate for ownership verification and intellectual property protection of deep neural networks. Regarding image classification neural networks, current watermarking schemes uniformly resort to backdoor triggers. However, injecting a backdoor into a neural network requires knowledge of the training dataset, which is usually unavailable in the real-world commercialization. Meanwhile, established watermarking schemes oversight the potential damage of exposed evidence during ownership verification and the watermarking algorithms themselves. Those concerns decline current watermarking schemes from industrial applications. To confront these challenges, we propose a knowledge-free black-box watermarking scheme for image classification neural networks. The image generator obtained from a data-free distillation process is leveraged to stabilize the network's performance during the backdoor injection. A delicate encoding and verification protocol is designed to ensure the scheme's security against knowledgable adversaries. We also give a pioneering analysis of the capacity of the watermarking scheme. Experiment results proved the functionality-preserving capability and security of the proposed watermarking scheme., Comment: 11 pages

Finding influential users in social networks is a fundamental problem with many possible useful applications. Viewing the social network as a graph, the influence of a set of users can be measured by the number of neighbors located within a given number of hops in the network, where each hop marks a step of influence diffusion. In this paper, we reduce the problem of IM to a budget-constrained d-hop dominating set problem (kdDSP). We propose a unified machine learning (ML) framework, FastCover, to solve kdDSP by learning an efficient greedy strategy in an unsupervised way. As one critical component of the framework, we devise a novel graph neural network (GNN) architecture, graph reversed attention network (GRAT), that captures the diffusion process among neighbors. Unlike most heuristic algorithms and concurrent ML frameworks for combinatorial optimization problems, FastCover determines the entire seed set from the nodes' scores computed with only one forward propagation of the GNN and has a time complexity quasi-linear in the graph size. Experiments on synthetic graphs and real-world social networks demonstrate that FastCover finds solutions with better or comparable quality rendered by the concurrent algorithms while achieving a speedup of over 1000x.

Deep neural networks are playing an important role in many real-life applications. After being trained with abundant data and computing resources, a deep neural network model providing service is endowed with economic value. An important prerequisite in commercializing and protecting deep neural networks is the reliable identification of their genuine author. To meet this goal, watermarking schemes that embed the author's identity information into the networks have been proposed. However, current schemes can hardly meet all the necessary requirements for securely proving the authorship and mostly focus on models for classification. To explicitly meet the formal definitions of the security requirements and increase the applicability of deep neural network watermarking schemes, we propose a new framework based on multi-task learning. By treating the watermark embedding as an extra task, most of the security requirements are explicitly formulated and met with well-designed regularizers, the rest is guaranteed by using components from cryptography. Moreover, a decentralized verification protocol is proposed to standardize the ownership verification. The experiment results show that the proposed scheme is flexible, secure, and robust, hence a promising candidate in deep learning model protection., Comment: The updated version of this manuscript would appear in the AAAI-2022 Workshop on Artificial Intelligence Safety (SafeAI)

Diffusion models simulate the propagation of influence in networks. The design and evaluation of diffusion models has been subjective and empirical. When being applied to a network represented by a graph, the diffusion model generates a sequence of edges on which the influence flows, such sequence forms a temporal network. In most scenarios, the statistical properties or the characteristics of a network are inferred by analyzing the temporal networks generated by diffusion models. To analyze real temporal networks, the motif has been proposed as a reliable feature. However, it is unclear how the network topology and the diffusion model affect the motif feature of a generated temporal network. In this paper, we adopt the motif feature to evaluate the temporal graph generated by a diffusion model, thence the diffusion model itself. Two benchmarks for quantitively evaluating diffusion models with motif, stability and separability, are proposed and measured on numerous diffusion models. One motif-based metric is proposed to measure the similarity between diffusion models. The experiments suggest that the motif of a generated temporal network is dominated by the diffusion model, while the network topology is almost ignored. This result indicates that more practical and reliable diffusion models have to be designed with delicacy in order to capture the propagation patterns of real temporal networks., Comment: 7 pages, 3 figures

Selecting the optimal subset from all vertices as seeds to maximize the influence in a social network has been a task of interest. Various methods have been proposed to select the optimal vertices in a static network, however, they are challenged by the dynamics, i.e. the time-dependent variation of the social network structure. Such dynamics hinder the paradigm for static networks and leaves a seemingly unbridgeable gap between algorithms of influence maximization on static networks and those on dynamic ones. In this paper, we extend our previous work and demonstrate that conjugate learning automata (an elementary variant of reinforcement learning) that have been successfully applied to maximize influence on static networks can be applied to dynamic networks as well. The network dynamics is measured by the variation of the influence range and absorbed into the learning procedure. Our proposal delicately formulates the effect of network dynamics: the more the influence range varies, the more likely the seeds are to be learned from scratch. Under this assumption, the continuity of the network variation is fully taken advantage of. Experimental results on both synthetic and real-world networks verify the privileges of our proposal against alternative methods., Comment: 6 pages

In this paper we solve the three-player-game question. A three-player-game consists of a series of rounds. There are altogether three players. Two players participate in each round, at the end of the round the loser quits and the third player enters the ring and another round starts. The game terminates if all six win-lose relationships appear. During each round, two players win with equal probability. One is asked to calculate the expectation of the number of rounds. It turns out to be an exemplary question that involves probabiltiy theory and dynamic programming. It can serve as an instance or exercise in the chapter of conditional expectation of any elementary or advanced textbook on probability., Comment: 7 pages, 2 figures

Finding a counterfeit coin with the different weight from a set of visually identical coin using a balance, usually a two-armed balance, known as the balance question, is an intersting and inspiring question. Its variants involve diversified toolkits including information theory, coding theory, optimization, probabilistic theory, combinatorics and a lot of quick wits. In this paper some variants of the balance game are dicussed, especially from a probabilistic perspective. Unlike the gravity field setting, we adopt an electromagnetic field, where tighter bounds for some variants of the balance game can be found. We focus on the predetermined setting, where the player has to arrange the strategy without observing the outcome of the balancing. The sufficient condition for the balance to win is obtained by adopting a coding scheme. Apart from designing a delicate encoding framework, we also propose and analyze the performance of a completely randomized strategy. The optimal behavior of a randomized player is derived. Then we rise the dishonest balance game, in which the balance can adversely cheat the player. We present some elementary results on the analysis of dishonest balance game using probabilistic method at length. Its relationship with Shannon' s coding theorem in a noisy channel is also revealed., Comment: 13 pages, 5 figures

The diffusion model has been a crucial component in studies about social networks. Many studies, especially these about influence maximization concern the proof of the submodularity of particular diffusion models. Such proofs have been model-dependent and are somewhat ad hoc. In this paper, we prove a theorem that provides a necessary and sufficient condition for a diffusion model to be submodular. This theorem can be used to justify the submodularity of an arbitrary diffusion model. We also apply this theorem to build a projection operator that maps an arbitrary diffusion model into a submodular one. Moreover, we use the established theorem to propose a diffusion model of multiple heterogeneous pieces of information that partially features submodularity., Comment: Conference pre-print