Search

Out-of-distribution (OOD) detection discerns OOD data where the predictor cannot make valid predictions as in-distribution (ID) data, thereby increasing the reliability of open-world classification. However, it is typically hard to collect real out-of-distribution (OOD) data for training a predictor capable of discerning ID and OOD patterns. This obstacle gives rise to data generation-based learning methods, synthesizing OOD data via data generators for predictor training without requiring any real OOD data. Related methods typically pre-train a generator on ID data and adopt various selection procedures to find those data likely to be the OOD cases. However, generated data may still coincide with ID semantics, i.e., mistaken OOD generation remains, confusing the predictor between ID and OOD data. To this end, we suggest that generated data (with mistaken OOD generation) can be used to devise an auxiliary OOD detection task to facilitate real OOD detection. Specifically, we can ensure that learning from such an auxiliary task is beneficial if the ID and the OOD parts have disjoint supports, with the help of a well-designed training procedure for the predictor. Accordingly, we propose a powerful data generation-based learning method named Auxiliary Task-based OOD Learning (ATOL) that can relieve the mistaken OOD generation. We conduct extensive experiments under various OOD detection setups, demonstrating the effectiveness of our method against its advanced counterparts. The code is publicly available at: https://github.com/tmlr-group/ATOL.

Adversarial detection aims to determine whether a given sample is an adversarial one based on the discrepancy between natural and adversarial distributions. Unfortunately, estimating or comparing two data distributions is extremely difficult, especially in high-dimension spaces. Recently, the gradient of log probability density (a.k.a., score) w.r.t. the sample is used as an alternative statistic to compute. However, we find that the score is sensitive in identifying adversarial samples due to insufficient information with one sample only. In this paper, we propose a new statistic called expected perturbation score (EPS), which is essentially the expected score of a sample after various perturbations. Specifically, to obtain adequate information regarding one sample, we perturb it by adding various noises to capture its multi-view observations. We theoretically prove that EPS is a proper statistic to compute the discrepancy between two samples under mild conditions. In practice, we can use a pre-trained diffusion model to estimate EPS for each sample. Last, we propose an EPS-based adversarial detection (EPS-AD) method, in which we develop EPS-based maximum mean discrepancy (MMD) as a metric to measure the discrepancy between the test sample and natural samples. We also prove that the EPS-based MMD between natural and adversarial samples is larger than that among natural samples. Extensive experiments show the superior adversarial detection performance of our EPS-AD.

Addressing global biodiversity loss requires an expanded focus on multiple dimensions of biodiversity. While most studies have focused on the consequences of plant interspecific diversity, our mechanistic understanding of how genetic diversity within plant species affects plant productivity remains limited. Here, we use a tree species × genetic diversity experiment to disentangle the effects of species diversity and genetic diversity, and how they are related to tree functional diversity and trophic feedbacks. We found that tree species diversity increased tree productivity via increased tree functional diversity, reduced soil fungal diversity and marginally reduced herbivory. The effects of tree genetic diversity on productivity via functional diversity and soil fungal diversity were negative in monocultures but positive in the mixture of the four tree species tested. Given the complexity of interactions between species and genetic diversity, tree functional diversity and trophic feedbacks on productivity, we suggest that both tree species and genetic diversity should be considered in afforestation.   &nbsp

Addressing global biodiversity loss requires an expanded focus on multiple dimensions of biodiversity. While most studies have focused on the consequences of plant interspecific diversity, our mechanistic understanding of how genetic diversity within plant species affects plant productivity remains limited. Here, we use a tree species × genetic diversity experiment to disentangle the effects of species diversity and genetic diversity, and how they are related to tree functional diversity and trophic feedbacks. We found that tree species diversity increased tree productivity via increased tree functional diversity, reduced soil fungal diversity and marginally reduced herbivory. The effects of tree genetic diversity on productivity via functional diversity and soil fungal diversity were negative in monocultures but positive in the mixture of the four tree species tested. Given the complexity of interactions between species and genetic diversity, tree functional diversity and trophic feedbacks on productivity, we suggest that both tree species and genetic diversity should be considered in afforestation.   &nbsp

Background: Despite immunotherapy advancements for patients with advanced or metastatic non-small-cell lung cancer (NSCLC), pivotal first-line trials were limited to patients with an Eastern Cooperative Oncology Group performance status (ECOG PS) 0–1 and a median age of 65 years or younger. We aimed to compare the efficacy and safety of first-line atezolizumab monotherapy with single-agent chemotherapy in patients ineligible for platinum-based chemotherapy. Methods: This trial was a phase 3, open-label, randomised controlled study conducted at 91 sites in 23 countries across Asia, Europe, North America, and South America. Eligible patients had stage IIIB or IV NSCLC in whom platinum-doublet chemotherapy was deemed unsuitable by the investigator due to an ECOG PS 2 or 3, or alternatively, being 70 years or older with an ECOG PS 0–1 with substantial comorbidities or contraindications for platinum-doublet chemotherapy. Patients were randomised 2:1 by permuted-block randomisation (block size of six) to receive 1200 mg of atezolizumab given intravenously every 3 weeks or single-agent chemotherapy (vinorelbine [oral or intravenous] or gemcitabine [intravenous]; dosing per local label) at 3-weekly or 4-weekly cycles. The primary endpoint was overall survival assessed in the intention-to-treat population. Safety analyses were conducted in the safety-evaluable population, which included all randomised patients who received any amount of atezolizumab or chemotherapy. This trial is registered with ClinicalTrials.gov, NCT03191786. Findings: Between Sept 11, 2017, and Sept 23, 2019, 453 patients were enrolled and randomised to receive atezolizumab (n=302) or chemotherapy (n=151). Atezolizumab improved overall survival compared with chemotherapy (median overall survival 10·3 months [95% CI 9·4–11·9] vs 9·2 months [5·9–11·2]; stratified hazard ratio 0·78 [0·63–0·97], p=0·028), with a 2-year survival rate of 24% (95% CI 19·3–29·4) with atezolizumab compared with 12% (6·7–18·0) with

Symbiotic cnidarians such as corals and anemones form highly productive and biodiverse coral reef ecosystems in nutrient-poor ocean environments, a phenomenon known as Darwin's paradox. Resolving this paradox requires elucidating the molecular bases of efficient nutrient distribution and recycling in the cnidarian-dinoflagellate symbiosis. Using the sea anemone Aiptasia, we show that during symbiosis, the increased availability of glucose and the presence of the algae jointly induce the coordinated up-regulation and relocalization of glucose and ammonium transporters. These molecular responses are critical to support symbiont functioning and organism-wide nitrogen assimilation through glutamine synthetase/glutamate synthase-mediated amino acid biosynthesis. Our results reveal crucial aspects of the molecular mechanisms underlying nitrogen conservation and recycling in these organisms that allow them to thrive in the nitrogen-poor ocean environments.

Accurate delineation of plant cell organelles from electron microscope images is essential for understanding subcellular behaviour and function. Here we develop a deep-learning pipeline, called the organelle segmentation network (OrgSegNet), for pixel-wise segmentation to identify chloroplasts, mitochondria, nuclei and vacuoles. OrgSegNet was evaluated on a large manually annotated dataset collected from 19 plant species and achieved state-of-the-art segmentation performance. We defined three digital traits (shape complexity, electron density and cross-sectional area) to track the quantitative features of individual organelles in 2D images and released an open-source web tool called Plantorganelle Hunter for quantitatively profiling subcellular morphology. In addition, the automatic segmentation method was successfully applied to a serial-sectioning scanning microscope technique to create a 3D cell model that offers unique views of the morphology and distribution of these organelles. The functionalities of Plantorganelle Hunter can be easily operated, which will increase efficiency and productivity for the plant science community, and enhance understanding of subcellular biology.

Supervised learning aims to train a classifier under the assumption that training and test data are from the same distribution. To ease the above assumption, researchers have studied a more realistic setting: out-of-distribution (OOD) detection, where test data may come from classes that are unknown during training (i.e., OOD data). Due to the unavailability and diversity of OOD data, good generalization ability is crucial for effective OOD detection algorithms. To study the generalization of OOD detection, in this paper, we investigate the probably approximately correct (PAC) learning theory of OOD detection, which is proposed by researchers as an open problem. First, we find a necessary condition for the learnability of OOD detection. Then, using this condition, we prove several impossibility theorems for the learnability of OOD detection under some scenarios. Although the impossibility theorems are frustrating, we find that some conditions of these impossibility theorems may not hold in some practical scenarios. Based on this observation, we next give several necessary and sufficient conditions to characterize the learnability of OOD detection in some practical scenarios. Lastly, we also offer theoretical supports for several representative OOD detection works based on our OOD theory.

Generating unlabeled data has been recently shown to help address the few-shot hypothesis adaptation (FHA) problem, where we aim to train a classifier for the target domain with a few labeled target-domain data and a well-trained source-domain classifier (i.e., a source hypothesis), for the additional information of the highly-compatible unlabeled data. However, the generated data of the existing methods are extremely similar or even the same. The strong dependency among the generated data will lead the learning to fail. In this paper, we propose a diversity-enhancing generative network (DEG-Net) for the FHA problem, which can generate diverse unlabeled data with the help of a kernel independence measure: the Hilbert-Schmidt independence criterion (HSIC). Specifically, DEG-Net will generate data via minimizing the HSIC value (i.e., maximizing the independence) among the semantic features of the generated data. By DEG-Net, the generated unlabeled data are more diverse and more effective for addressing the FHA problem. Experimental results show that the DEG-Net outperforms existing FHA baselines and further verifies that generating diverse data plays a vital role in addressing the FHA problem.

Domain generalization (DG) aims to tackle the distribution shift between training domains and unknown target domains. Generating new domains is one of the most effective approaches, yet its performance gain depends on the distribution discrepancy between the generated and target domains. Distributionally robust optimization is promising to tackle distribution discrepancy by exploring domains in an uncertainty set. However, the uncertainty set may be overwhelmingly large, leading to low-confidence prediction in DG. It is because a large uncertainty set could introduce domains containing semantically different factors from training domains. To address this issue, we propose to perform a moderately distributional exploration (MODE) for domain generalization. Specifically, MODE performs distribution exploration in an uncertainty subset that shares the same semantic factors with the training domains. We show that MODE can endow models with provable generalization performance on unknown target domains. The experimental results show that MODE achieves competitive performance compared to state-of-the-art baselines.

In label-noise learning, estimating the transition matrix is a hot topic as the matrix plays an important role in building statistically consistent classifiers. Traditionally, the transition from clean labels to noisy labels (i.e., clean-label transition matrix (CLTM)) has been widely exploited on class-dependent label-noise (wherein all samples in a clean class share the same label transition matrix). However, the CLTM cannot handle the more common instance-dependent label-noise well (wherein the clean-to-noisy label transition matrix needs to be estimated at the instance level by considering the input quality). Motivated by the fact that classifiers mostly output Bayes optimal labels for prediction, in this paper, we study to directly model the transition from Bayes optimal labels to noisy labels (i.e., Bayes-Label Transition Matrix (BLTM)) and learn a classifier to predict Bayes optimal labels. Note that given only noisy data, it is ill-posed to estimate either the CLTM or the BLTM. But favorably, Bayes optimal labels have no uncertainty compared with the clean labels, i.e., the class posteriors of Bayes optimal labels are one-hot vectors while those of clean labels are not. This enables two advantages to estimate the BLTM, i.e., (a) a set of examples with theoretically guaranteed Bayes optimal labels can be collected out of noisy data; (b) the feasible solution space is much smaller. By exploiting the advantages, this work proposes a parametrical model for estimating the instance-dependent label-noise transition matrix by employing a deep neural network, leading to better generalization and superior classification performance.

Open-world classification systems should discern out-of-distribution (OOD) data whose labels deviate from those of in-distribution (ID) cases, motivating recent studies in OOD detection. Advanced works, despite their promising progress, may still fail in the open world, owing to the lack of knowledge about unseen OOD data in advance. Although one can access auxiliary OOD data (distinct from unseen ones) for model training, it remains to analyze how such auxiliary data will work in the open world. To this end, we delve into such a problem from a learning theory perspective, finding that the distribution discrepancy between the auxiliary and the unseen real OOD data is the key to affecting the open-world detection performance. Accordingly, we propose Distributional-Augmented OOD Learning (DAL), alleviating the OOD distribution discrepancy by crafting an OOD distribution set that contains all distributions in a Wasserstein ball centered on the auxiliary OOD distribution. We justify that the predictor trained over the worst OOD data in the ball can shrink the OOD distribution discrepancy, thus improving the open-world detection performance given only the auxiliary OOD data. We conduct extensive evaluations across representative OOD detection setups, demonstrating the superiority of our DAL over its advanced counterparts. The code is publicly available at: https://github.com/tmlr-group/DAL.

Learning with noisy labels has become imperative in the Big Data era, which saves expensive human labors on accurate annotations. Previous noise-transition-based methods have achieved theoretically-grounded performance under the Class-Conditional Noise model (CCN). However, these approaches builds upon an ideal but impractical anchor set available to pre-estimate the noise transition. Even though subsequent works adapt the estimation as a neural layer, the ill-posed stochastic learning of its parameters in back-propagation easily falls into undesired local minimums. We solve this problem by introducing a Latent Class-Conditional Noise model (LCCN) to parameterize the noise transition under a Bayesian framework. By projecting the noise transition into the Dirichlet space, the learning is constrained on a simplex characterized by the complete dataset, instead of some ad-hoc parametric space wrapped by the neural layer. We then deduce a dynamic label regression method for LCCN, whose Gibbs sampler allows us efficiently infer the latent true labels to train the classifier and to model the noise. Our approach safeguards the stable update of the noise transition, which avoids previous arbitrarily tuning from a mini-batch of samples. We further generalize LCCN to different counterparts compatible with open-set noisy labels, semi-supervised learning as well as cross-model training. A range of experiments demonstrate the advantages of LCCN and its variants over the current state-of-the-art methods. The code is available at here.