Search

Pre-trained language models (PLMs) have revolutionized both the natural language processing research and applications. However, stereotypical biases (e.g., gender and racial discrimination) encoded in PLMs have raised negative ethical implications for PLMs, which critically limits their broader applications. To address the aforementioned unfairness issues, we present fairBERTs, a general framework for learning fair fine-tuned BERT series models by erasing the protected sensitive information via semantic and fairness-aware perturbations generated by a generative adversarial network. Through extensive qualitative and quantitative experiments on two real-world tasks, we demonstrate the great superiority of fairBERTs in mitigating unfairness while maintaining the model utility. We also verify the feasibility of transferring adversarial components in fairBERTs to other conventionally trained BERT-like models for yielding fairness improvements. Our findings may shed light on further research on building fairer fine-tuned PLMs.

Large Language Models have gained considerable attention for their revolutionary capabilities. However, there is also growing concern on their safety implications, making a comprehensive safety evaluation for LLMs urgently needed before model deployment. In this work, we propose S-Eval, a new comprehensive, multi-dimensional and open-ended safety evaluation benchmark. At the core of S-Eval is a novel LLM-based automatic test prompt generation and selection framework, which trains an expert testing LLM Mt combined with a range of test selection strategies to automatically construct a high-quality test suite for the safety evaluation. The key to the automation of this process is a novel expert safety-critique LLM Mc able to quantify the riskiness score of an LLM's response, and additionally produce risk tags and explanations. Besides, the generation process is also guided by a carefully designed risk taxonomy with four different levels, covering comprehensive and multi-dimensional safety risks of concern. Based on these, we systematically construct a new and large-scale safety evaluation benchmark for LLMs consisting of 220,000 evaluation prompts, including 20,000 base risk prompts (10,000 in Chinese and 10,000 in English) and 200,000 corresponding attack prompts derived from 10 popular adversarial instruction attacks against LLMs. Moreover, considering the rapid evolution of LLMs and accompanied safety threats, S-Eval can be flexibly configured and adapted to include new risks, attacks and models. S-Eval is extensively evaluated on 20 popular and representative LLMs. The results confirm that S-Eval can better reflect and inform the safety risks of LLMs compared to existing benchmarks. We also explore the impacts of parameter scales, language environments, and decoding parameters on the evaluation, providing a systematic methodology for evaluating the safety of LLMs., Comment: 18 pages, 11 figures

Spinodal decomposition in multicomponent alloys has attracted increasing attention due to its beneficial effect on their mechanical and functional properties and potential applications. Both based on the Cahn-Hillard equation, the reference element method (REM) and the projection matrix method (PMM) are the two main methods to predict the occurrence of spinodal decomposition in multicomponent alloys. In this work, it is mathematically proven that the two methods are equivalent, and therefore the advanced results based on one method can be applied to the other. Based on these methods, the $Fe{}_{15}$Co${}_{15}$Ni${}_{35}$Cu${}_{35}$ multicomponent alloy is designed as a case study. Experimental results confirm the spinodal decomposition in the heat-treated alloy, and its strength and ductility are simultaneously enhanced. This work can be the pavement for further theoretical and experimental studies on the spinodal decomposition in multicomponent alloys., Comment: 27 pages, 3 figures, 1 supplementary file

This article foucuses on $(P_3\cup P_2,K_4)$-free graph. In this paper, we prove that if G is $(P_3\cup P_2,K_4)$-free, then $\chi(G)\le 7$. We then use our result to obtain the upper bound of order and chromatic number of $(4K_1,\overline{P_3\cup P_2},K_{\omega})$-free graph ., Comment: 12 pages,8 figures

A hereditary class $\cal G$ of graphs is {\em $\chi$-bounded} if there is a {\em $\chi$-binding function}, say $f$, such that $\chi(G)\le f(\omega(G))$ for every $G\in\cal G$, where $\chi(G)(\omega(G))$ denotes the chromatic (clique) number of $G$. It is known that for every $(P_3\cup P_2)$-free graph $G$, $\chi(G)\le \frac{1}{6}\omega(G)(\omega(G)+1)(\omega(G)+2)$ \cite{BA18}, and the class of $(2K_2, 3K_1)$-free graphs does not admit a linear $\chi$-binding function\cite{BBS19}. In this paper, we prove that (\romannumeral 1) $\chi(G)\le2\omega(G)$ if $G$ is ($P_3\cup P_2$, kite)-free, (\romannumeral 2) $\chi(G)\le\omega^2(G)$ if $G$ is ($P_3\cup P_2$, hammer)-free, (\romannumeral 3) $\chi(G)\le\frac{3\omega^2(G)+\omega(G)}{2}$ if $G$ is ($P_3\cup P_2, C_5$)-free. Furthermore, we also discuss $\chi$-binding functions for $(P_3\cup P_2, K_4)$-free graphs., Comment: arXiv admin note: substantial text overlap with arXiv:2308.05442, arXiv:2308.08768

For two vertex disjoint graphs $H$ and $F$, we use $H\cup F$ to denote the graph with vertex set $V(H)\cup V(F)$ and edge set $E(H)\cup E(F)$, and use $H+F$ to denote the graph with vertex set $V(H)\cup V(F)$ and edge set $E(H)\cup E(F)\cup\{xy\;|\; x\in V(H), y\in V(F)$$\}$. A $W_4$ is the graph $K_1+C_4$. In this paper, we prove that $\chi(G)\le 2\omega(G)$ if $G$ is a ($P_3\cup P_2, W_4$)-free graph. This bound is tight when $\omega =2$ and $3$, and improves the main result of Wang and Zhang. Also, this bound partially generalizes some results of Prashant {\em et al.}., Comment: arXiv admin note: text overlap with arXiv:2308.05442, arXiv:2307.11946

Let $G$ and $H$ be two vertex disjoint graphs. The {\em union} $G\cup H$ is the graph with $V(G\cup H)=V(G)\cup V(H)$ and $E(G\cup H)=E(G)\cup E(H)$. We use $P_k$ to denote a {\em path} on $k$ vertices, use {\em house} to denote the complement of $P_5$. In this paper, we show that $\chi(G)\le2\omega(G)$ if $G$ is ($P_3\cup P_2$, house)-free. Moreover, this bound is optimal when $\omega(G)\ge2$., Comment: arXiv admin note: text overlap with arXiv:2307.11946

Topologically protected spin textures, such as skyrmions1,2 and vortices3,4, are robust against perturbations, serving as the building blocks for a range of topological devices5-9. In order to implement these topological devices, it is necessary to find ultra-small topological spin textures at room temperature, because small size implies the higher topological charge density, stronger signal of topological transport10,11 and the higher memory density or integration for topological quantum devices5-9. However, finding ultra-small topological spin textures at high temperatures is still a great challenge up to now. Here we find ultra-small topological spin textures in Fe78Si9B13 amorphous alloy. We measured a large topological Hall effect (THE) up to above room temperature, indicating the existence of highly densed and ultra-small topological spin textures in the samples. Further measurements by small-angle neutron scattering (SANS) reveal that the average size of ultra-small magnetic texture is around 1.3nm. Our Monte Carlo simulations show that such ultra-small spin texture is topologically equivalent to skyrmions, which originate from competing frustration and Dzyaloshinskii-Moriya interaction12,13 coming from amorphous structure14-17. Taking a single topological spin texture as one bit and ignoring the distance between them, we evaluated the ideal memory density of Fe78Si9B13, which reaches up to 4.44*104 gigabits (43.4 TB) per in2 and is 2 times of the value of GdRu2Si218 at 5K. More important, such high memory density can be obtained at above room temperature, which is 4 orders of magnitude larger than the value of other materials at the same temperature. These findings provide a unique candidate for magnetic memory devices with ultra-high density., Comment: 26 pages, 4 figures

Deep learning-based recommender systems (DRSs) are increasingly and widely deployed in the industry, which brings significant convenience to people's daily life in different ways. However, recommender systems are also shown to suffer from multiple issues,e.g., the echo chamber and the Matthew effect, of which the notation of "fairness" plays a core role.While many fairness notations and corresponding fairness testing approaches have been developed for traditional deep classification models, they are essentially hardly applicable to DRSs. One major difficulty is that there still lacks a systematic understanding and mapping between the existing fairness notations and the diverse testing requirements for deep recommender systems, not to mention further testing or debugging activities. To address the gap, we propose FairRec, a unified framework that supports fairness testing of DRSs from multiple customized perspectives, e.g., model utility, item diversity, item popularity, etc. We also propose a novel, efficient search-based testing approach to tackle the new challenge, i.e., double-ended discrete particle swarm optimization (DPSO) algorithm, to effectively search for hidden fairness issues in the form of certain disadvantaged groups from a vast number of candidate groups. Given the testing report, by adopting a simple re-ranking mitigation strategy on these identified disadvantaged groups, we show that the fairness of DRSs can be significantly improved. We conducted extensive experiments on multiple industry-level DRSs adopted by leading companies. The results confirm that FairRec is effective and efficient in identifying the deeply hidden fairness issues, e.g., achieving 95% testing accuracy with half to 1/8 time.

Despite of the superb performance on a wide range of tasks, pre-trained language models (e.g., BERT) have been proved vulnerable to adversarial texts. In this paper, we present RoChBERT, a framework to build more Robust BERT-based models by utilizing a more comprehensive adversarial graph to fuse Chinese phonetic and glyph features into pre-trained representations during fine-tuning. Inspired by curriculum learning, we further propose to augment the training dataset with adversarial texts in combination with intermediate samples. Extensive experiments demonstrate that RoChBERT outperforms previous methods in significant ways: (i) robust -- RoChBERT greatly improves the model robustness without sacrificing accuracy on benign texts. Specifically, the defense lowers the success rates of unlimited and limited attacks by 59.43% and 39.33% respectively, while remaining accuracy of 93.30%; (ii) flexible -- RoChBERT can easily extend to various language models to solve different downstream tasks with excellent performance; and (iii) efficient -- RoChBERT can be directly applied to the fine-tuning stage without pre-training language model from scratch, and the proposed data augmentation method is also low-cost., Comment: Accepted by Findings of EMNLP 2022

Detection of object anomalies is crucial in industrial processes, but unsupervised anomaly detection and localization is particularly important due to the difficulty of obtaining a large number of defective samples and the unpredictable types of anomalies in real life. Among the existing unsupervised anomaly detection and localization methods, the NF-based scheme has achieved better results. However, the two subnets (complex functions) $s_{i}(u_{i})$ and $t_{i}(u_{i})$ in NF are usually multilayer perceptrons, which need to squeeze the input visual features from 2D flattening to 1D, destroying the spatial location relationship in the feature map and losing the spatial structure information. In order to retain and effectively extract spatial structure information, we design in this study a complex function model with alternating CBAM embedded in a stacked $3\times3$ full convolution, which is able to retain and effectively extract spatial structure information in the normalized flow model. Extensive experimental results on the MVTec AD dataset show that CAINNFlow achieves advanced levels of accuracy and inference efficiency based on CNN and Transformer backbone networks as feature extractors, and CAINNFlow achieves a pixel-level AUC of $98.64\%$ for anomaly detection in MVTec AD.

This paper proves that, under a monotonicity condition, the invariant probability measure of a McKean--Vlasov process can be approximated by weighted empirical measures of some processes including itself. These processes are described by distribution dependent or empirical measure dependent stochastic differential equations constructed from the equation for the McKean--Vlasov process. Convergence of empirical measures is characterized by upper bound estimates for their Wasserstein distance to the invariant measure. The theoretical results are demonstrated via a mean-field Ornstein--Uhlenbeck process., Comment: 26 papes, 2 figures

Pre-trained general-purpose language models have been a dominating component in enabling real-world natural language processing (NLP) applications. However, a pre-trained model with backdoor can be a severe threat to the applications. Most existing backdoor attacks in NLP are conducted in the fine-tuning phase by introducing malicious triggers in the targeted class, thus relying greatly on the prior knowledge of the fine-tuning task. In this paper, we propose a new approach to map the inputs containing triggers directly to a predefined output representation of the pre-trained NLP models, e.g., a predefined output representation for the classification token in BERT, instead of a target label. It can thus introduce backdoor to a wide range of downstream tasks without any prior knowledge. Additionally, in light of the unique properties of triggers in NLP, we propose two new metrics to measure the performance of backdoor attacks in terms of both effectiveness and stealthiness. Our experiments with various types of triggers show that our method is widely applicable to different fine-tuning tasks (classification and named entity recognition) and to different models (such as BERT, XLNet, BART), which poses a severe threat. Furthermore, by collaborating with the popular online model repository Hugging Face, the threat brought by our method has been confirmed. Finally, we analyze the factors that may affect the attack performance and share insights on the causes of the success of our backdoor attack.