Search

Out-of-distribution (OOD) detection aims at identifying samples from unknown classes, playing a crucial role in trustworthy models against errors on unexpected inputs. Extensive research has been dedicated to exploring OOD detection in the vision modality. Vision-language models (VLMs) can leverage both textual and visual information for various multi-modal applications, whereas few OOD detection methods take into account information from the text modality. In this paper, we propose a novel post hoc OOD detection method, called NegLabel, which takes a vast number of negative labels from extensive corpus databases. We design a novel scheme for the OOD score collaborated with negative labels. Theoretical analysis helps to understand the mechanism of negative labels. Extensive experiments demonstrate that our method NegLabel achieves state-of-the-art performance on various OOD detection benchmarks and generalizes well on multiple VLM architectures. Furthermore, our method NegLabel exhibits remarkable robustness against diverse domain shifts. The codes are available at https://github.com/tmlr-group/NegLabel., Comment: ICLR 2024 Spotlight

Image interpolation based on diffusion models is promising in creating fresh and interesting images. Advanced interpolation methods mainly focus on spherical linear interpolation, where images are encoded into the noise space and then interpolated for denoising to images. However, existing methods face challenges in effectively interpolating natural images (not generated by diffusion models), thereby restricting their practical applicability. Our experimental investigations reveal that these challenges stem from the invalidity of the encoding noise, which may no longer obey the expected noise distribution, e.g., a normal distribution. To address these challenges, we propose a novel approach to correct noise for image interpolation, NoiseDiffusion. Specifically, NoiseDiffusion approaches the invalid noise to the expected distribution by introducing subtle Gaussian noise and introduces a constraint to suppress noise with extreme values. In this context, promoting noise validity contributes to mitigating image artifacts, but the constraint and introduced exogenous noise typically lead to a reduction in signal-to-noise ratio, i.e., loss of original image information. Hence, NoiseDiffusion performs interpolation within the noisy image space and injects raw images into these noisy counterparts to address the challenge of information loss. Consequently, NoiseDiffusion enables us to interpolate natural images without causing artifacts or information loss, thus achieving the best interpolation results., Comment: ICLR 2024

Post-hoc out-of-distribution (OOD) detection has garnered intensive attention in reliable machine learning. Many efforts have been dedicated to deriving score functions based on logits, distances, or rigorous data distribution assumptions to identify low-scoring OOD samples. Nevertheless, these estimate scores may fail to accurately reflect the true data density or impose impractical constraints. To provide a unified perspective on density-based score design, we propose a novel theoretical framework grounded in Bregman divergence, which extends distribution considerations to encompass an exponential family of distributions. Leveraging the conjugation constraint revealed in our theorem, we introduce a \textsc{ConjNorm} method, reframing density function design as a search for the optimal norm coefficient $p$ against the given dataset. In light of the computational challenges of normalization, we devise an unbiased and analytically tractable estimator of the partition function using the Monte Carlo-based importance sampling technique. Extensive experiments across OOD detection benchmarks empirically demonstrate that our proposed \textsc{ConjNorm} has established a new state-of-the-art in a variety of OOD detection setups, outperforming the current best method by up to 13.25$\%$ and 28.19$\%$ (FPR95) on CIFAR-100 and ImageNet-1K, respectively., Comment: ICLR24 poster

Using unlabeled data to regularize the machine learning models has demonstrated promise for improving safety and reliability in detecting out-of-distribution (OOD) data. Harnessing the power of unlabeled in-the-wild data is non-trivial due to the heterogeneity of both in-distribution (ID) and OOD data. This lack of a clean set of OOD samples poses significant challenges in learning an optimal OOD classifier. Currently, there is a lack of research on formally understanding how unlabeled data helps OOD detection. This paper bridges the gap by introducing a new learning framework SAL (Separate And Learn) that offers both strong theoretical guarantees and empirical effectiveness. The framework separates candidate outliers from the unlabeled data and then trains an OOD classifier using the candidate outliers and the labeled ID data. Theoretically, we provide rigorous error bounds from the lens of separability and learnability, formally justifying the two components in our algorithm. Our theory shows that SAL can separate the candidate outliers with small error rates, which leads to a generalization guarantee for the learned OOD classifier. Empirically, SAL achieves state-of-the-art performance on common benchmarks, reinforcing our theoretical insights. Code is publicly available at https://github.com/deeplearning-wisc/sal., Comment: ICLR 2024

Uncertain changes in data streams present challenges for machine learning models to dynamically adapt and uphold performance in real-time. Particularly, classification boundary change, also known as real concept drift, is the major cause of classification performance deterioration. However, accurately detecting real concept drift remains challenging because the theoretical foundations of existing drift detection methods - two-sample distribution tests and monitoring classification error rate, both suffer from inherent limitations such as the inability to distinguish virtual drift (changes not affecting the classification boundary, will introduce unnecessary model maintenance), limited statistical power, or high computational cost. Furthermore, no existing detection method can provide information on the trend of the drift, which could be invaluable for model maintenance. This work presents a novel real concept drift detection method based on Neighbor-Searching Discrepancy, a new statistic that measures the classification boundary difference between two samples. The proposed method is able to detect real concept drift with high accuracy while ignoring virtual drift. It can also indicate the direction of the classification boundary change by identifying the invasion or retreat of a certain class, which is also an indicator of separability change between classes. A comprehensive evaluation of 11 experiments is conducted, including empirical verification of the proposed theory using artificial datasets, and experimental comparisons with commonly used drift handling methods on real-world datasets. The results show that the proposed theory is robust against a range of distributions and dimensions, and the drift detection method outperforms state-of-the-art alternative methods.

Supervised learning aims to train a classifier under the assumption that training and test data are from the same distribution. To ease the above assumption, researchers have studied a more realistic setting: out-of-distribution (OOD) detection, where test data may come from classes that are unknown during training (i.e., OOD data). Due to the unavailability and diversity of OOD data, good generalization ability is crucial for effective OOD detection algorithms, and corresponding learning theory is still an open problem. To study the generalization of OOD detection, this paper investigates the probably approximately correct (PAC) learning theory of OOD detection that fits the commonly used evaluation metrics in the literature. First, we find a necessary condition for the learnability of OOD detection. Then, using this condition, we prove several impossibility theorems for the learnability of OOD detection under some scenarios. Although the impossibility theorems are frustrating, we find that some conditions of these impossibility theorems may not hold in some practical scenarios. Based on this observation, we next give several necessary and sufficient conditions to characterize the learnability of OOD detection in some practical scenarios. Lastly, we offer theoretical support for representative OOD detection works based on our OOD theory., Comment: Accepted by JMLR in 7th of April, 2024. This is a journal extension of the previous NeurIPS 2022 Outstanding Paper "Is Out-of-distribution Detection Learnable?" [arXiv:2210.14707]

Jiongdi Lu,1,2,* Zhi Zheng,3,* Yixuan Ding,1,2 Yuanxu Qu,1,2 Wentong Mei,1,2 Zhen Fang,1,2 Chang Qu,1,2,* Yulu Feng,4 Yulin Guo,1,2 Chongchong Gao,1,2 Feng Cao,1,2 Fei Li1,2 1Clinical Center of Acute Pancreatitis, Capital Medical University, Beijing, People’s Republic of China; 2Department of General Surgery, Xuanwu Hospital, Capital Medical University, Beijing, People’s Republic of China; 3Department of General Surgery, Beijing Friendship Hospital, Capital Medical University, Beijing, People’s Republic of China; 4Chui Yang Liu Hospital Affiliated Tsinghua University, Beijing, People’s Republic of China*These authors contributed equally to this workCorrespondence: Fei Li, Clinical Center of Acute Pancreatitis, Department of General Surgery, Department of Surgery, Xuanwu Hospital of Capital Medical University, No. 45, Changchun Street, Xicheng District, Beijing, 100053, People’s Republic of China, Tel +86-10-83198731, Fax +86-10-83198868, Email feili36@ccmu.edu.cn Feng Cao, Clinical Center of Acute Pancreatitis, Department of General Surgery, Xuanwu Hospital of Capital Medical University, No. 45, Changchun Street, Xicheng District, Beijing, 100053, People’s Republic of China, Email f.cao@xwhosp.orgObjective: To clarify the incidence of colonic complications in patients with NP and their impact on prognosis.Methods: The clinical data of NP patients admitted to the Department of General Surgery of Xuanwu Hospital, Capital Medical University from January 2014 to December 2020 were retrospectively analyzed. Patients were grouped according to the presence or absence of colonic complications, and the clinical prognosis of the two groups was analyzed after matching using a 1:1 propensity score, The primary study endpoint was patient mortality during hospitalization. Data are reported as median (range) or percentage of patients (%).Results: A total of 306 patients with NP were included in this study, and the incidence of colonic co

In this paper, we study the processes of $e^{+}e^{-}$ annihilation into two body final states, either two pseudoscalar mesons or one meson with a photon. The hadronic vacuum polarization form factors are calculated within the framework of resonance chiral theory in the energy region of $E \lesssim 2$ GeV, with final state interactions taken into account. A joint analysis on the processes of $e^{+}e^{-} \rightarrow \pi^{+}\pi^{-}$, $K^{+}K^{-}$, $K_{L}^{0}K_{S}^{0}$, $\pi^{0}\gamma$, and $\eta\gamma$ has been performed, and the latest experimental data are included. Based on the vacuum polarization form factors of these processes, we estimate their contributions to the lowest order of anomalous magnetic moment of the muon, $(g-2)_\mu$. Combined with other contributions from hadronic vacuum polarization and other interactions from the standard model, the discrepancy between theoretical prediction and experimental measurement is $\Delta a_{\mu}=(24.1\pm5.4)\times 10^{-10}$, i.e., 4.5$\sigma$., Comment: 45 pages, 6 figures, to be the same as the published version

The continental crust is strongly depleted in copper compared with its building blocks-primary arc magmas-and this depletion is intrinsically associated with continental crust formation. However, the process by which Cu removal occurs remains enigmatic. Here we show, using Cu isotopes, that subduction-zone processes and mantle melting produce limited fractionation of Cu isotopes in arc magmas, and, instead, the heterogeneous Cu isotopic compositions of lower crustal rocks, which negatively correlate with Cu contents, suggest segregation or accumulation of isotopically light sulfides during intracrustal differentiation of arc magmas. This is supported by the extremely light Cu isotopic compositions of lower crustal mafic cumulates and heavy Cu isotopic compositions of differentiated magmas in thick continental arcs. Intracrustal differentiation of mantle-derived magmas and subsequent foundering of sulfide-rich mafic cumulates preferentially removes isotopically light Cu, leaving a Cu-depleted and isotopically heavy continental crust.

Domain generalization (DG) aims to tackle the distribution shift between training domains and unknown target domains. Generating new domains is one of the most effective approaches, yet its performance gain depends on the distribution discrepancy between the generated and target domains. Distributionally robust optimization is promising to tackle distribution discrepancy by exploring domains in an uncertainty set. However, the uncertainty set may be overwhelmingly large, leading to low-confidence prediction in DG. It is because a large uncertainty set could introduce domains containing semantically different factors from training domains. To address this issue, we propose to perform a $\textbf{mo}$derately $\textbf{d}$istributional $\textbf{e}$xploration (MODE) for domain generalization. Specifically, MODE performs distribution exploration in an uncertainty $\textit{subset}$ that shares the same semantic factors with the training domains. We show that MODE can endow models with provable generalization performance on unknown target domains. The experimental results show that MODE achieves competitive performance compared to state-of-the-art baselines., Comment: Accepted by ICML 2023

Recent diffusion-based image editing approaches have exhibited impressive editing capabilities in images with simple compositions. However, localized editing in complex scenarios has not been well-studied in the literature, despite its growing real-world demands. Existing mask-based inpainting methods fall short of retaining the underlying structure within the edit region. Meanwhile, mask-free attention-based methods often exhibit editing leakage and misalignment in more complex compositions. In this work, we develop MAG-Edit, a training-free, inference-stage optimization method, which enables localized image editing in complex scenarios. In particular, MAG-Edit optimizes the noise latent feature in diffusion models by maximizing two mask-based cross-attention constraints of the edit token, which in turn gradually enhances the local alignment with the desired prompt. Extensive quantitative and qualitative experiments demonstrate the effectiveness of our method in achieving both text alignment and structure preservation for localized editing within complex scenarios., Comment: for project page, see https://mag-edit.github.io

Out-of-distribution (OOD) detection discerns OOD data where the predictor cannot make valid predictions as in-distribution (ID) data, thereby increasing the reliability of open-world classification. However, it is typically hard to collect real out-of-distribution (OOD) data for training a predictor capable of discerning ID and OOD patterns. This obstacle gives rise to data generation-based learning methods, synthesizing OOD data via data generators for predictor training without requiring any real OOD data. Related methods typically pre-train a generator on ID data and adopt various selection procedures to find those data likely to be the OOD cases. However, generated data may still coincide with ID semantics, i.e., mistaken OOD generation remains, confusing the predictor between ID and OOD data. To this end, we suggest that generated data (with mistaken OOD generation) can be used to devise an auxiliary OOD detection task to facilitate real OOD detection. Specifically, we can ensure that learning from such an auxiliary task is beneficial if the ID and the OOD parts have disjoint supports, with the help of a well-designed training procedure for the predictor. Accordingly, we propose a powerful data generation-based learning method named Auxiliary Task-based OOD Learning (ATOL) that can relieve the mistaken OOD generation. We conduct extensive experiments under various OOD detection setups, demonstrating the effectiveness of our method against its advanced counterparts., Comment: Accepted by NeurIPS 2023

Open-world classification systems should discern out-of-distribution (OOD) data whose labels deviate from those of in-distribution (ID) cases, motivating recent studies in OOD detection. Advanced works, despite their promising progress, may still fail in the open world, owing to the lack of knowledge about unseen OOD data in advance. Although one can access auxiliary OOD data (distinct from unseen ones) for model training, it remains to analyze how such auxiliary data will work in the open world. To this end, we delve into such a problem from a learning theory perspective, finding that the distribution discrepancy between the auxiliary and the unseen real OOD data is the key to affecting the open-world detection performance. Accordingly, we propose Distributional-Augmented OOD Learning (DAL), alleviating the OOD distribution discrepancy by crafting an OOD distribution set that contains all distributions in a Wasserstein ball centered on the auxiliary OOD distribution. We justify that the predictor trained over the worst OOD data in the ball can shrink the OOD distribution discrepancy, thus improving the open-world detection performance given only the auxiliary OOD data. We conduct extensive evaluations across representative OOD detection setups, demonstrating the superiority of our DAL over its advanced counterparts.

Continual Named Entity Recognition (CNER) is a burgeoning area, which involves updating an existing model by incorporating new entity types sequentially. Nevertheless, continual learning approaches are often severely afflicted by catastrophic forgetting. This issue is intensified in CNER due to the consolidation of old entity types from previous steps into the non-entity type at each step, leading to what is known as the semantic shift problem of the non-entity type. In this paper, we introduce a pooled feature distillation loss that skillfully navigates the trade-off between retaining knowledge of old entity types and acquiring new ones, thereby more effectively mitigating the problem of catastrophic forgetting. Additionally, we develop a confidence-based pseudo-labeling for the non-entity type, \emph{i.e.,} predicting entity types using the old model to handle the semantic shift of the non-entity type. Following the pseudo-labeling process, we suggest an adaptive re-weighting type-balanced learning strategy to handle the issue of biased type distribution. We carried out comprehensive experiments on ten CNER settings using three different datasets. The results illustrate that our method significantly outperforms prior state-of-the-art approaches, registering an average improvement of $6.3$\% and $8.0$\% in Micro and Macro F1 scores, respectively., Comment: Accepted by EMNLP2023 main conference as a long paper

Image-based Reinforcement Learning is a practical yet challenging task. A major hurdle lies in extracting control-centric representations while disregarding irrelevant information. While approaches that follow the bisimulation principle exhibit the potential in learning state representations to address this issue, they still grapple with the limited expressive capacity of latent dynamics and the inadaptability to sparse reward environments. To address these limitations, we introduce ReBis, which aims to capture control-centric information by integrating reward-free control information alongside reward-specific knowledge. ReBis utilizes a transformer architecture to implicitly model the dynamics and incorporates block-wise masking to eliminate spatiotemporal redundancy. Moreover, ReBis combines bisimulation-based loss with asymmetric reconstruction loss to prevent feature collapse in environments with sparse rewards. Empirical studies on two large benchmarks, including Atari games and DeepMind Control Suit, demonstrate that ReBis has superior performance compared to existing methods, proving its effectiveness.

Adapting models deployed to test distributions can mitigate the performance degradation caused by distribution shifts. However, privacy concerns may render model parameters inaccessible. One promising approach involves utilizing zeroth-order optimization (ZOO) to train a data adaptor to adapt the test data to fit the deployed models. Nevertheless, the data adaptor trained with ZOO typically brings restricted improvements due to the potential corruption of data features caused by the data adaptor. To address this issue, we revisit ZOO in the context of test-time data adaptation. We find that the issue directly stems from the unreliable estimation of the gradients used to optimize the data adaptor, which is inherently due to the unreliable nature of the pseudo-labels assigned to the test data. Based on this observation, we propose pseudo-label-robust data adaptation (SODA) to improve the performance of data adaptation. Specifically, SODA leverages high-confidence predicted labels as reliable labels to optimize the data adaptor with ZOO for label prediction. For data with low-confidence predictions, SODA encourages the adaptor to preserve data information to mitigate data corruption. Empirical results indicate that SODA can significantly enhance the performance of deployed models in the presence of distribution shifts without requiring access to model parameters.

Out-of-distribution (OOD) generalization is indispensable for learning models in the wild, where testing distribution typically unknown and different from the training. Recent methods derived from causality have shown great potential in achieving OOD generalization. However, existing methods mainly focus on the invariance property of causes, while largely overlooking the property of \textit{sufficiency} and \textit{necessity} conditions. Namely, a necessary but insufficient cause (feature) is invariant to distribution shift, yet it may not have required accuracy. By contrast, a sufficient yet unnecessary cause (feature) tends to fit specific data well but may have a risk of adapting to a new domain. To capture the information of sufficient and necessary causes, we employ a classical concept, the probability of sufficiency and necessary causes (PNS), which indicates the probability of whether one is the necessary and sufficient cause. To associate PNS with OOD generalization, we propose PNS risk and formulate an algorithm to learn representation with a high PNS value. We theoretically analyze and prove the generalizability of the PNS risk. Experiments on both synthetic and real-world benchmarks demonstrate the effectiveness of the proposed method. The details of the implementation can be found at the GitHub repository: https://github.com/ymy4323460/CaSN.

Out-of-distribution (OOD) detection is crucial to modern deep learning applications by identifying and alerting about the OOD samples that should not be tested or used for making predictions. Current OOD detection methods have made significant progress when in-distribution (ID) and OOD samples are drawn from static distributions. However, this can be unrealistic when applied to real-world systems which often undergo continuous variations and shifts in ID and OOD distributions over time. Therefore, for an effective application in real-world systems, the development of OOD detection methods that can adapt to these dynamic and evolving distributions is essential. In this paper, we propose a novel and more realistic setting called continuously adaptive out-of-distribution (CAOOD) detection which targets on developing an OOD detection model that enables dynamic and quick adaptation to a new arriving distribution, with insufficient ID samples during deployment time. To address CAOOD, we develop meta OOD learning (MOL) by designing a learning-to-adapt diagram such that a good initialized OOD detection model is learned during the training process. In the testing process, MOL ensures OOD detection performance over shifting distributions by quickly adapting to new distributions with a few adaptations. Extensive experiments on several OOD benchmarks endorse the effectiveness of our method in preserving both ID classification accuracy and OOD detection performance on continuously shifting distributions., Comment: Accepted by ICCV 2023

Multi-tenancy in public clouds may lead to co-location interference on shared resources, which possibly results in performance degradation of cloud applications. Cloud providers want to know when such events happen and how serious the degradation is, to perform interference-aware migrations and alleviate the problem. However, virtual machines (VM) in Infrastructure-as-a-Service public clouds are black-boxes to providers, where application-level performance information cannot be acquired. This makes performance monitoring intensely challenging as cloud providers can only rely on low-level metrics such as CPU usage and hardware counters. We propose a novel machine learning framework, Alioth, to monitor the performance degradation of cloud applications. To feed the data-hungry models, we first elaborate interference generators and conduct comprehensive co-location experiments on a testbed to build Alioth-dataset which reflects the complexity and dynamicity in real-world scenarios. Then we construct Alioth by (1) augmenting features via recovering low-level metrics under no interference using denoising auto-encoders, (2) devising a transfer learning model based on domain adaptation neural network to make models generalize on test cases unseen in offline training, and (3) developing a SHAP explainer to automate feature selection and enhance model interpretability. Experiments show that Alioth achieves an average mean absolute error of 5.29% offline and 10.8% when testing on applications unseen in the training stage, outperforming the baseline methods. Alioth is also robust in signaling quality-of-service violation under dynamicity. Finally, we demonstrate a possible application of Alioth's interpretability, providing insights to benefit the decision-making of cloud operators. The dataset and code of Alioth have been released on GitHub., Comment: Accepted by 2023 IEEE International Parallel & Distributed Processing Symposium (IPDPS)

Achieving a reliable LiDAR-based object detector in autonomous driving is paramount, but its success hinges on obtaining large amounts of precise 3D annotations. Active learning (AL) seeks to mitigate the annotation burden through algorithms that use fewer labels and can attain performance comparable to fully supervised learning. Although AL has shown promise, current approaches prioritize the selection of unlabeled point clouds with high uncertainty and/or diversity, leading to the selection of more instances for labeling and reduced computational efficiency. In this paper, we resort to a novel kernel coding rate maximization (KECOR) strategy which aims to identify the most informative point clouds to acquire labels through the lens of information theory. Greedy search is applied to seek desired point clouds that can maximize the minimal number of bits required to encode the latent features. To determine the uniqueness and informativeness of the selected samples from the model perspective, we construct a proxy network of the 3D detector head and compute the outer product of Jacobians from all proxy layers to form the empirical neural tangent kernel (NTK) matrix. To accommodate both one-stage (i.e., SECOND) and two-stage detectors (i.e., PVRCNN), we further incorporate the classification entropy maximization and well trade-off between detection performance and the total number of bounding boxes selected for annotation. Extensive experiments conducted on two 3D benchmarks and a 2D detection dataset evidence the superiority and versatility of the proposed approach. Our results show that approximately 44% box-level annotation costs and 26% computational time are reduced compared to the state-of-the-art AL method, without compromising detection performance., Comment: To appear in ICCV 2023

We consider an improved soft-wall AdS/QCD model coupled to an Einstein-dilaton system, which offers a way to study the deconfining and chiral transitions simultaneously. The correlation between these two transitions has been investigated in detail in the Einstein-dilaton-scalar system with the bulk scalar field representing the vacuum of matters in the flavor sector of the model. We find that the effects of the scaling dimension $\Delta$ of the dual operator of the dilaton manifest in chiral transitions, although the equation of state can all be matched with the two-flavor lattice results for $\Delta=2.5, 3, 3.5$ in the decoupling case of $\beta=0$. In the weak-coupling case with smaller $\beta$, both the equation of state and the chiral transition exhibit a crossover behavior and turn into first-order phase transitions with the increase of $\beta$.

Jiongdi Lu,1,2 Yixuan Ding,1,2 Yuanxu Qu,1,2 Wentong Mei,1,2 Yulin Guo,1,2 Zhen Fang,1,2 Chang Qu,1,2 Chongchong Gao,1,2 Feng Cao,1,2,* Fei Li,1,2,* Yulu Feng3 1Clinical Center of Acute Pancreatitis, Capital Medical University, Beijing, People’s Republic of China; 2Department of General Surgery, Xuanwu Hospital, Capital Medical University, Beijing, People’s Republic of China; 3Department of Pediatric, Chui Yang Liu Hospital Affiliated Tsinghua University, Beijing, People’s Republic of China*These authors contributed equally to this workCorrespondence: Fei Li; Feng Cao, Clinical Center of Acute Pancreatitis, Capital Medical University, Department of General Surgery, Department of Surgery, Xuanwu Hospital, Capital Medical University, No. 45, Changchun Street, Xicheng District, Beijing, 100053, People’s Republic of China, Tel +86-10-83198731, Fax +86-10-83198868, Email feili36@ccmu.edu.cn; f.cao@xwhosp.orgObjective: The incidence of acute pancreatitis (AP) is increasing. Twenty percent of AP patients with developing necrotizing pancreatitis (NP), while ~40– 70% of NP patients develop potentially fatal infectious complications. When patients are suspected or confirmed infected pancreatic necrosis (IPN), antibiotics should be administered timeously to control the infection, but long-term use of antibiotics can lead to multidrug-resistant bacteria (MDRB) infection and eventually to increased mortality. Our study aimed to determine the incidence of MDRB infection and evaluate the risk factors for MDRB infection in IPN patients.Methods: Clinical data of IPN patients admitted to the general surgery department of Xuanwu Hospital of Capital Medical University between January 1, 2014, and December 31, 2021, were retrospectively analyzed.Results: IPN patients (n = 267) were assigned to MDRB infection (n = 124) and non-MDRB infection (n = 143) groups. On admission, patients in the MDRB group had a higher modified computer tomography severity inde

Supervised learning aims to train a classifier under the assumption that training and test data are from the same distribution. To ease the above assumption, researchers have studied a more realistic setting: out-of-distribution (OOD) detection, where test data may come from classes that are unknown during training (i.e., OOD data). Due to the unavailability and diversity of OOD data, good generalization ability is crucial for effective OOD detection algorithms. To study the generalization of OOD detection, in this paper, we investigate the probably approximately correct (PAC) learning theory of OOD detection, which is proposed by researchers as an open problem. First, we find a necessary condition for the learnability of OOD detection. Then, using this condition, we prove several impossibility theorems for the learnability of OOD detection under some scenarios. Although the impossibility theorems are frustrating, we find that some conditions of these impossibility theorems may not hold in some practical scenarios. Based on this observation, we next give several necessary and sufficient conditions to characterize the learnability of OOD detection in some practical scenarios. Lastly, we also offer theoretical supports for several representative OOD detection works based on our OOD theory., Comment: NeurIPS 2022 Outstanding Paper

Federated learning (FL) has attracted growing attention via data-private collaborative training on decentralized clients. However, most existing methods unrealistically assume object classes of the overall framework are fixed over time. It makes the global model suffer from significant catastrophic forgetting on old classes in real-world scenarios, where local clients often collect new classes continuously and have very limited storage memory to store old classes. Moreover, new clients with unseen new classes may participate in the FL training, further aggravating the catastrophic forgetting of the global model. To address these challenges, we develop a novel Global-Local Forgetting Compensation (GLFC) model, to learn a global class incremental model for alleviating the catastrophic forgetting from both local and global perspectives. Specifically, to address local forgetting caused by class imbalance at the local clients, we design a class-aware gradient compensation loss and a class-semantic relation distillation loss to balance the forgetting of old classes and distill consistent inter-class relations across tasks. To tackle the global forgetting brought by the non-i.i.d class imbalance across clients, we propose a proxy server that selects the best old global model to assist the local relation distillation. Moreover, a prototype gradient-based communication mechanism is developed to protect privacy. Our model outperforms state-of-the-art methods by 4.4%-15.1% in terms of average accuracy on representative benchmark datasets., Comment: CVPR 2022, the first two authors contribute equally and they are ordered alphabetically

The theoretical analysis of multi-class classification has proved that the existing multi-class classification methods can train a classifier with high classification accuracy on the test set, when the instances are precise in the training and test sets with same distribution and enough instances can be collected in the training set. However, one limitation with multi-class classification has not been solved: how to improve the classification accuracy of multi-class classification problems when only imprecise observations are available. Hence, in this paper, we propose a novel framework to address a new realistic problem called multi-class classification with imprecise observations (MCIMO), where we need to train a classifier with fuzzy-feature observations. Firstly, we give the theoretical analysis of the MCIMO problem based on fuzzy Rademacher complexity. Then, two practical algorithms based on support vector machine and neural networks are constructed to solve the proposed new problem. Experiments on both synthetic and real-world datasets verify the rationality of our theoretical analysis and the efficacy of the proposed algorithms., Comment: This article has been accepted by IEEE Transactions on Cybernetics on June 4, 2022

The availability of massive labeled datasets results in the successes of supervised learning in many applications such as object detection, speech recognition and natural language processing. However, the curse of domain mismatch arises if the test samples (target samples) and training samples (source samples) are from different domains. To overcome the mismatch between domains, researchers have proposed transfer learning, which aims to leverage knowledge from source samples with abundant labels to help train a classifier for target samples with insufficient labels. Although many algorithms have been developed to solve transfer learning problem, most algorithms depend on additional assumptions to ensure their effectiveness. Many assumptions are difficult to be satisfied in real-world application. For example, the label sets of source and target samples are assumed to be same, however, in unsupervised setting, it is impossible to check this assumption. Those unrealistic assumptions limit the application of transfer learning. This thesis mainly studies two important problems faced by many existing transfer learning algorithms: 1) how to transfer knowledge in the unsupervised setting, when the target label set is larger than the source label set; and 2) how to transfer knowledge, when the source and target samples are from different feature spaces. Before solving problems 1) and 2), we first consider transfer learning which assumes that the feature spaces and label sets are both same. The setting is also called homogeneous domain adaptation. By studying the homogeneous domain adaptation, we aim to understand how the source knowledge can be used to help improve the target's performance under the simplest setting (i.e., homogeneous setting). To solve problem 1), this thesis considers a challenging task: unsupervised open-set domain adaptation (UOSDA). First, to understand why source samples can be used to help target samples achieve good performance in the open-set setti

The availability of massive labeled datasets results in the successes of supervised learning in many applications such as object detection, speech recognition and natural language processing. However, the curse of domain mismatch arises if the test samples (target samples) and training samples (source samples) are from different domains. To overcome the mismatch between domains, researchers have proposed transfer learning, which aims to leverage knowledge from source samples with abundant labels to help train a classifier for target samples with insufficient labels. Although many algorithms have been developed to solve transfer learning problem, most algorithms depend on additional assumptions to ensure their effectiveness. Many assumptions are difficult to be satisfied in real-world application. For example, the label sets of source and target samples are assumed to be same, however, in unsupervised setting, it is impossible to check this assumption. Those unrealistic assumptions limit the application of transfer learning. This thesis mainly studies two important problems faced by many existing transfer learning algorithms: 1) how to transfer knowledge in the unsupervised setting, when the target label set is larger than the source label set; and 2) how to transfer knowledge, when the source and target samples are from different feature spaces. Before solving problems 1) and 2), we first consider transfer learning which assumes that the feature spaces and label sets are both same. The setting is also called homogeneous domain adaptation. By studying the homogeneous domain adaptation, we aim to understand how the source knowledge can be used to help improve the target's performance under the simplest setting (i.e., homogeneous setting). To solve problem 1), this thesis considers a challenging task: unsupervised open-set domain adaptation (UOSDA). First, to understand why source samples can be used to help target samples achieve good performance in the open-set setti

Direct production of biodiesel by both esterification and transesterification of renewable oils with acid value (AV) of 9.6-28.9 mg KOH/g using calcined blast furnace dust is studied. Blast furnace dust calcined at 600-700 degrees C obviously promotes efficient biodiesel production because of the crystal form transformation from lead sulfate particles in dust into active lead oxide, lead sulfide, and lead particles. It is found for the first time that lead oxide, lead sulfide and lead particles can resist saponification from fatty acids to catalyze raw renewable oils to biodiesel with catalytic activity order as follows: lead oxide > lead > lead sulfide > lead sulfate. Biodiesel production process is optimized according to an orthogonal design with biodiesel yield of 92 wt% obtained at AV of 9.6 mg KOH/g (82 wt% after 5 cycles). Biodiesel yield of 84 wt% is achieved at AV of 28.9 mg KOH/g with acidity of 0.18 mmol/g. Calcined blast furnace dust presents potential applications in the production of biodiesel from renewable oils with high AVs., QC 20210720

Traditional supervised learning aims to train a classifier in the closed-set world, where training and test samples share the same label space. In this paper, we target a more challenging and realistic setting: open-set learning (OSL), where there exist test samples from the classes that are unseen during training. Although researchers have designed many methods from the algorithmic perspectives, there are few methods that provide generalization guarantees on their ability to achieve consistent performance on different training samples drawn from the same distribution. Motivated by the transfer learning and probably approximate correct (PAC) theory, we make a bold attempt to study OSL by proving its generalization error-given training samples with size n, the estimation error will get close to order O_p(1/\sqrt{n}). This is the first study to provide a generalization bound for OSL, which we do by theoretically investigating the risk of the target classifier on unknown classes. According to our theory, a novel algorithm, called auxiliary open-set risk (AOSR) is proposed to address the OSL problem. Experiments verify the efficacy of AOSR. The code is available at github.com/Anjin-Liu/Openset_Learning_AOSR., Comment: Open-set Learning, Open-set Recognition, Machine Learning Theory

In unsupervised domain adaptation (UDA), a classifier for the target domain is trained with massive true-label data from the source domain and unlabeled data from the target domain. However, collecting fully-true-label data in the source domain is high-cost and sometimes impossible. Compared to the true labels, a complementary label specifies a class that a pattern does not belong to, hence collecting complementary labels would be less laborious than collecting true labels. Thus, in this paper, we propose a novel setting that the source domain is composed of complementary-label data, and a theoretical bound for it is first proved. We consider two cases of this setting, one is that the source domain only contains complementary-label data (completely complementary unsupervised domain adaptation, CC-UDA), and the other is that the source domain has plenty of complementary-label data and a small amount of true-label data (partly complementary unsupervised domain adaptation, PC-UDA). To this end, a complementary label adversarial network} (CLARINET) is proposed to solve CC-UDA and PC-UDA problems. CLARINET maintains two deep networks simultaneously, where one focuses on classifying complementary-label source data and the other takes care of source-to-target distributional adaptation. Experiments show that CLARINET significantly outperforms a series of competent baselines on handwritten-digits-recognition and objects-recognition tasks., Comment: arXiv admin note: text overlap with arXiv:2007.14612

In unsupervised domain adaptation (UDA), classifiers for the target domain are trained with massive true-label data from the source domain and unlabeled data from the target domain. However, it may be difficult to collect fully-true-label data in a source domain given a limited budget. To mitigate this problem, we consider a novel problem setting where the classifier for the target domain has to be trained with complementary-label data from the source domain and unlabeled data from the target domain named budget-friendly UDA (BFUDA). The key benefit is that it is much less costly to collect complementary-label source data (required by BFUDA) than collecting the true-label source data (required by ordinary UDA). To this end, the complementary label adversarial network (CLARINET) is proposed to solve the BFUDA problem. CLARINET maintains two deep networks simultaneously, where one focuses on classifying complementary-label source data and the other takes care of the source-to-target distributional adaptation. Experiments show that CLARINET significantly outperforms a series of competent baselines., Comment: This paper has been accepted by IJCAI-PRICAI 2020. Yiyang Zhang, Feng Liu and Zhen Fang equally contribute to this paper

In the unsupervised open set domain adaptation (UOSDA), the target domain contains unknown classes that are not observed in the source domain. Researchers in this area aim to train a classifier to accurately: 1) recognize unknown target data (data with unknown classes) and, 2) classify other target data. To achieve this aim, a previous study has proven an upper bound of the target-domain risk, and the open set difference, as an important term in the upper bound, is used to measure the risk on unknown target data. By minimizing the upper bound, a shallow classifier can be trained to achieve the aim. However, if the classifier is very flexible (e.g., deep neural networks (DNNs)), the open set difference will converge to a negative value when minimizing the upper bound, which causes an issue where most target data are recognized as unknown data. To address this issue, we propose a new upper bound of target-domain risk for UOSDA, which includes four terms: source-domain risk, $\epsilon$-open set difference ($\Delta_\epsilon$), a distributional discrepancy between domains, and a constant. Compared to the open set difference, $\Delta_\epsilon$ is more robust against the issue when it is being minimized, and thus we are able to use very flexible classifiers (i.e., DNNs). Then, we propose a new principle-guided deep UOSDA method that trains DNNs via minimizing the new upper bound. Specifically, source-domain risk and $\Delta_\epsilon$ are minimized by gradient descent, and the distributional discrepancy is minimized via a novel open-set conditional adversarial training strategy. Finally, compared to existing shallow and deep UOSDA methods, our method shows the state-of-the-art performance on several benchmark datasets, including digit recognition (MNIST, SVHN, USPS), object recognition (Office-31, Office-Home), and face recognition (PIE).

Unsupervised domain adaptation (UDA) aims to train a target classifier with labeled samples from the source domain and unlabeled samples from the target domain. Classical UDA learning bounds show that target risk is upper bounded by three terms: source risk, distribution discrepancy, and combined risk. Based on the assumption that the combined risk is a small fixed value, methods based on this bound train a target classifier by only minimizing estimators of the source risk and the distribution discrepancy. However, the combined risk may increase when minimizing both estimators, which makes the target risk uncontrollable. Hence the target classifier cannot achieve ideal performance if we fail to control the combined risk. To control the combined risk, the key challenge takes root in the unavailability of the labeled samples in the target domain. To address this key challenge, we propose a method named E-MixNet. E-MixNet employs enhanced mixup, a generic vicinal distribution, on the labeled source samples and pseudo-labeled target samples to calculate a proxy of the combined risk. Experiments show that the proxy can effectively curb the increase of the combined risk when minimizing the source risk and distribution discrepancy. Furthermore, we show that if the proxy of the combined risk is added into loss functions of four representative UDA methods, their performance is also improved., Comment: 9 pages, 3 figures, Accepted by Association for the Advancement of Artificial Intelligence 2021 (AAAI 2021)

Nitrogen-containing compounds, especially primary amines, are vital building blocks in nature and industry. Herein, a protocol is developed that shows in situ formed N-formyl quasi-catalytic species afford highly selective synthesis of formamides or amines with controllable levels from a variety of aldehyde- and ketone-derived platform chemical substrates under solvent-free conditions. Up to 99% yields of mono-substituted formamides are obtained in 3 min. The C-N bond formation and N-formyl species are prevalent in the cascade reaction sequence. Kinetic and isotope labeling experiments explicitly demonstrate that the C-N bond is activated for subsequent hydrogenation, in which formic acid acts as acid catalyst, hydrogen donor and as N-formyl species source that stabilize amine intermediates elucidated with density functional theory. The protocol provides access to imides from aldehydes, ketones, carboxylic acids, and mixed-substrates, requires no special catalysts, solvents or techniques and provides new avenues for amination chemistry.

We give a further study on the improved soft-wall AdS/QCD model with two flavors. The chiral transition behaviors are studied in the case of finite chemical potential, with the chiral phase diagram obtained at zero quark mass. The thermal spectral functions of the vector and axial-vector mesons are calculated, and the in-medium melting properties of the mesons are investigated. We find that the chiral transition behaviors and the meson melting properties at finite chemical potential can be qualitatively described by the improved soft-wall AdS/QCD model, except in the region of large chemical potential. The reason for these inadequate descriptions may be that the background geometry adopted in the model is not a dynamical one which is able to produce the QCD equation of state. To give a quantitative description for these low-energy phenomenologies, we shall consider a more consistent AdS/QCD model which treats the background fields and the chiral fields on the same footing.

We construct an improved soft-wall AdS/QCD model with a cubic coupling term of the dilaton and the bulk scalar field. The background fields in this model are solved from the Einstein-dilaton system with a nontrivial dilaton potential, which has been shown to be able to reproduce the equation of state from lattice QCD with two flavors. The chiral transition behaviors are investigated in the improved soft-wall AdS/QCD model with the solved gravitational background, and the crossover transition can be realized. Our study provides a possibility to address the deconfining and chiral phase transitions simultaneously in the bottom-up holographic framework.

We give a further study on chiral phase diagram in the improved soft-wall AdS/QCD model with $2+1$ flavors. The equations of motion for the octet pseudoscalar, vector and axial-vector mesons are derived to compute the octet meson spectra and relevant decay constants, by which the model parameters are determined. The chemical potential effects on thermal transition of chiral condensate are investigated, which enables us to obtain the chiral phase diagram in the $\mu-T$ plane. We find that the critical end point linking the crossover transition with the first-order phase transition still exists and locates at $(\mu_B, T_c) \simeq (390 \MeV, 145 \MeV)$, which along with the crossover line are consistent with lattice result and experimental analysis from relativistic heavy ion collisions.

Nitrogen-containing compounds, especially primary amines, are vital building blocks in nature and industry. Herein, a protocol is developed that shows in situ formed N-formyl quasi-catalytic species afford highly selective synthesis of formamides or amines with controllable levels from a variety of aldehyde- and ketone-derived platform chemical substrates under solvent-free conditions. Up to 99% yields of mono-substituted formamides are obtained in 3 min. The C-N bond formation and N-formyl species are prevalent in the cascade reaction sequence. Kinetic and isotope labeling experiments explicitly demonstrate that the C-N bond is activated for subsequent hydrogenation, in which formic acid acts as acid catalyst, hydrogen donor and as N-formyl species source that stabilize amine intermediates elucidated with density functional theory. The protocol provides access to imides from aldehydes, ketones, carboxylic acids, and mixed-substrates, requires no special catalysts, solvents or techniques and provides new avenues for amination chemistry.

The aim of unsupervised domain adaptation is to leverage the knowledge in a labeled (source) domain to improve a model's learning performance with an unlabeled (target) domain -- the basic strategy being to mitigate the effects of discrepancies between the two distributions. Most existing algorithms can only handle unsupervised closed set domain adaptation (UCSDA), i.e., where the source and target domains are assumed to share the same label set. In this paper, we target a more challenging but realistic setting: unsupervised open set domain adaptation (UOSDA), where the target domain has unknown classes that are not found in the source domain. This is the first study to provide a learning bound for open set domain adaptation, which we do by theoretically investigating the risk of the target classifier on unknown classes. The proposed learning bound has a special term, namely open set difference, which reflects the risk of the target classifier on unknown classes. Further, we present a novel and theoretically guided unsupervised algorithm for open set domain adaptation, called distribution alignment with ppen difference (DAOD), which is based on regularizing this open set difference bound. The experiments on several benchmark datasets show the superior performance of the proposed UOSDA method compared with the state-of-the-art methods in the literature., Comment: This paper has been accepted by IEEE-TNNLS

We study the chemical potential effects on the chiral phase transition in a simply improved soft-wall AdS/QCD model, which can realize consistently the properties of linear confinement and spontaneous chiral symmetry breaking. The $\mu-T$ phase diagrams with both zero and physical quark masses have been obtained from this model. For the case of physical quark mass, the chiral transition has a crossover behavior at low chemical potential. With the increase of the chemical potential $\mu$, the critical temperature $T_c$ descends towards zero, and the crossover transition turns into a first-order one at some intermediate value of $\mu$, which implies that a critical end point naturally exists in this improved soft-wall model. All these features agree with our current perspective on the QCD phase diagram.

We study the chiral phase transition with 2 + 1 quark flavors in an improved soft-wall AdS/QCD model, which can produce the light meson spectrum and many other low-energy quantities consistent with experiments in the two-flavor case. The chiral transition behaviors of the quark condensates at different quark masses are analysed in detail, and the (m_{u,d}, m_s) phase diagram for the quark sector has been obtained from the improved soft-wall model. We find that the features of the calculated phase diagram are completely consistent with the standard scenario, which is supported by lattice simulations and theoretical arguments. The evidence of a tricritical point on the m_{u,d} = 0 boundary of the (m_{u,d}, m_s) phase diagram is first clearly presented in the bottom-up AdS/QCD.

We construct a series of new hyperbolic black hole solutions in Einstein-Scalar system and we apply holographic approach to investigate the spherical R\'{e}nyi entropy in various deformations of dual conformal field theories (CFTs). Especially, we introduce various powers of scalars in the scalar potentials for massive and massless scalar. These scalar potentials correspond to deformation of dual CFTs. Then we solve asymptotically hyperbolic AdS black hole solutions numerically. We map the instabilities of these black hole solutions to phase transitions of field theory in terms of CHM mapping between hyperbolic hairy AdS black hole and spherical R\'{e}nyi entropy in dual field theories. Based on these solutions, we study the temperature dependent condensation of dual operator of massive and massless scalar respectively. These condensations show that there might exist phase transitions in dual deformed CFTs. We also compare free energy between asymptotically hyperbolic AdS black hole solutions and hyperbolic AdS Schwarz (AdS-SW) black hole to test phase transitions. In order to confirm the existence of phase transitions, we turn on linear in-homogenous perturbation to test stability of these hyperbolic hairy AdS black holes. In this paper, we show how potential parameters affect the stability of hyperbolic black holes in several specific examples. For general values of potential parameters, it needs further studies to see how the transition happens. Finally, we comment on these instabilities associated with spherical R\'{e}nyi entropy in dual deformed CFTs., Comment: 40 Pages, 18 figures. Title changed, References updated, Structure reorganized, Typos and Mistakes corrected. Accepted by NPB

We construct a series of new hyperbolic black hole solutions in Einstein-Scalar system and we apply holographic approach to investigate the spherical R\'{e}nyi entropy in various deformations of dual conformal field theories (CFTs). Especially, we introduce various powers of scalars in the scalar potentials for massive and massless scalar. These scalar potentials correspond to deformation of dual CFTs. Then we solve asymptotically hyperbolic AdS black hole solutions numerically. We map the instabilities of these black hole solutions to phase transitions of field theory in terms of CHM mapping between hyperbolic hairy AdS black hole and spherical R\'{e}nyi entropy in dual field theories. Based on these solutions, we study the temperature dependent condensation of dual operator of massive and massless scalar respectively. These condensations show that there might exist phase transitions in dual deformed CFTs. We also compare free energy between asymptotically hyperbolic AdS black hole solutions and hyperbolic AdS Schwarz (AdS-SW) black hole to test phase transitions. In order to confirm the existence of phase transitions, we turn on linear in-homogenous perturbation to test stability of these hyperbolic hairy AdS black holes. In this paper, we show how potential parameters affect the stability of hyperbolic black holes in several specific examples. For general values of potential parameters, it needs further studies to see how the transition happens. Finally, we comment on these instabilities associated with spherical R\'{e}nyi entropy in dual deformed CFTs., Comment: 40 Pages, 18 figures. Title changed, References updated, Structure reorganized, Typos and Mistakes corrected. Accepted by NPB