Search

Traditional knowledge distillation focuses on aligning the student's predicted probabilities with both ground-truth labels and the teacher's predicted probabilities. However, the transition to predicted probabilities from logits would obscure certain indispensable information. To address this issue, it is intuitive to additionally introduce a logit-level loss function as a supplement to the widely used probability-level loss function, for exploiting the latent information of logits. Unfortunately, we empirically find that the amalgamation of the newly introduced logit-level loss and the previous probability-level loss will lead to performance degeneration, even trailing behind the performance of employing either loss in isolation. We attribute this phenomenon to the collapse of the classification head, which is verified by our theoretical analysis based on the neural collapse theory. Specifically, the gradients of the two loss functions exhibit contradictions in the linear classifier yet display no such conflict within the backbone. Drawing from the theoretical analysis, we propose a novel method called dual-head knowledge distillation, which partitions the linear classifier into two classification heads responsible for different losses, thereby preserving the beneficial effects of both losses on the backbone while eliminating adverse influences on the classification head. Extensive experiments validate that our method can effectively exploit the information inside the logits and achieve superior performance against state-of-the-art counterparts., Comment: Preprint

We present 850 $\mu$m polarization observations of the IC 348 star-forming region in the Perseus molecular cloud as part of the B-fields In STar-forming Region Observation (BISTRO) survey. We study the magnetic properties of two cores (HH 211 MMS and IC 348 MMS) and a filamentary structure of IC 348. We find that the overall field tends to be more perpendicular than parallel to the filamentary structure of the region. The polarization fraction decreases with intensity, and we estimate the trend by power-law and the mean of the Rice distribution fittings. The power indices for the cores are much smaller than 1, indicative of possible grain growth to micron size in the cores. We also measure the magnetic field strengths of the two cores and the filamentary area separately by applying the Davis-Chandrasekhar-Fermi method and its alternative version for compressed medium. The estimated mass-to-flux ratios are 0.45-2.20 and 0.63-2.76 for HH 211 MMS and IC 348 MMS, respectively, while the ratios for the filament is 0.33-1.50. This result may suggest that the transition from subcritical to supercritical conditions occurs at the core scale ($\sim$ 0.05 pc) in the region. In addition, we study the energy balance of the cores and find that the relative strength of turbulence to the magnetic field tends to be stronger for IC 348 MMS than HH 211 MMS. The result could potentially explain the different configurations inside the two cores: a single protostellar system in HH 211 MMS and multiple protostars in IC 348 MMS., Comment: Accepted for publication in ApJ. 21 pages, 12 figures

Active learning (AL) has achieved great success by selecting the most valuable examples from unlabeled data. However, they usually deteriorate in real scenarios where open-set noise gets involved, which is studied as open-set annotation (OSA). In this paper, we owe the deterioration to the unreliable predictions arising from softmax-based translation invariance and propose a Dirichlet-based Coarse-to-Fine Example Selection (DCFS) strategy accordingly. Our method introduces simplex-based evidential deep learning (EDL) to break translation invariance and distinguish known and unknown classes by considering evidence-based data and distribution uncertainty simultaneously. Furthermore, hard known-class examples are identified by model discrepancy generated from two classifier heads, where we amplify and alleviate the model discrepancy respectively for unknown and known classes. Finally, we combine the discrepancy with uncertainties to form a two-stage strategy, selecting the most informative examples from known classes. Extensive experiments on various openness ratio datasets demonstrate that DCFS achieves state-of-art performance.

Large language models (LLMs) have shown great potential in code-related tasks, yet open-source models lag behind their closed-source counterparts. To bridge this performance gap, existing methods generate vast amounts of synthetic data for fine-tuning, leading to inefficiencies in training. Motivated by the need for more effective and efficient training, we propose the Code Adaptive Compute-efficient Tuning (CodeACT) framework. CodeACT introduces the Complexity and Diversity Aware Sampling (CDAS) method to select high-quality training data based on complexity and diversity, and the Dynamic Pack padding strategy to reduce computational resource usage by minimizing padding tokens during training. Experimental results demonstrate that CodeACT-DeepSeek-Coder-6.7B, fine-tuned on only 40% of the EVOL-Instruct data, achieves an 8.6% performance increase on HumanEval, reduces training time by 78%, and decreases peak GPU memory usage by 27%. These findings underscore CodeACT's ability to enhance the performance and efficiency of open-source models. By optimizing both the data selection and training processes, CodeACT offers a comprehensive approach to improving the capabilities of open-source LLMs while significantly reducing computational requirements, addressing the dual challenges of data quality and training efficiency, and paving the way for more resource-efficient and performant models.

Semi-supervised multi-label learning (SSMLL) is a powerful framework for leveraging unlabeled data to reduce the expensive cost of collecting precise multi-label annotations. Unlike semi-supervised learning, one cannot select the most probable label as the pseudo-label in SSMLL due to multiple semantics contained in an instance. To solve this problem, the mainstream method developed an effective thresholding strategy to generate accurate pseudo-labels. Unfortunately, the method neglected the quality of model predictions and its potential impact on pseudo-labeling performance. In this paper, we propose a dual-perspective method to generate high-quality pseudo-labels. To improve the quality of model predictions, we perform dual-decoupling to boost the learning of correlative and discriminative features, while refining the generation and utilization of pseudo-labels. To obtain proper class-wise thresholds, we propose the metric-adaptive thresholding strategy to estimate the thresholds, which maximize the pseudo-label performance for a given metric on labeled data. Experiments on multiple benchmark datasets show the proposed method can achieve the state-of-the-art performance and outperform the comparative methods with a significant margin.

Current knowledge distillation (KD) methods primarily focus on transferring various structured knowledge and designing corresponding optimization goals to encourage the student network to imitate the output of the teacher network. However, introducing too many additional optimization objectives may lead to unstable training, such as gradient conflicts. Moreover, these methods ignored the guidelines of relative learning difficulty between the teacher and student networks. Inspired by human cognitive science, in this paper, we redefine knowledge from a new perspective -- the student and teacher networks' relative difficulty of samples, and propose a pixel-level KD paradigm for semantic segmentation named Relative Difficulty Distillation (RDD). We propose a two-stage RDD framework: Teacher-Full Evaluated RDD (TFE-RDD) and Teacher-Student Evaluated RDD (TSE-RDD). RDD allows the teacher network to provide effective guidance on learning focus without additional optimization goals, thus avoiding adjusting learning weights for multiple losses. Extensive experimental evaluations using a general distillation loss function on popular datasets such as Cityscapes, CamVid, Pascal VOC, and ADE20k demonstrate the effectiveness of RDD against state-of-the-art KD methods. Additionally, our research showcases that RDD can integrate with existing KD methods to improve their upper performance bound.

Active learning (AL) for multiple target models aims to reduce labeled data querying while effectively training multiple models concurrently. Existing AL algorithms often rely on iterative model training, which can be computationally expensive, particularly for deep models. In this paper, we propose a one-shot AL method to address this challenge, which performs all label queries without repeated model training. Specifically, we extract different representations of the same dataset using distinct network backbones, and actively learn the linear prediction layer on each representation via an $\ell_p$-regression formulation. The regression problems are solved approximately by sampling and reweighting the unlabeled instances based on their maximum Lewis weights across the representations. An upper bound on the number of samples needed is provided with a rigorous analysis for $p\in [1, +\infty)$. Experimental results on 11 benchmarks show that our one-shot approach achieves competitive performances with the state-of-the-art AL methods for multiple target models., Comment: The proof of Lemma 3.11 is fixed

Recent studies showed that the generalization of neural networks is correlated with the sharpness of the loss landscape, and flat minima suggests a better generalization ability than sharp minima. In this paper, we propose a novel method called \emph{optimum shifting}, which changes the parameters of a neural network from a sharp minimum to a flatter one while maintaining the same training loss value. Our method is based on the observation that when the input and output of a neural network are fixed, the matrix multiplications within the network can be treated as systems of under-determined linear equations, enabling adjustment of parameters in the solution space, which can be simply accomplished by solving a constrained optimization problem. Furthermore, we introduce a practical stochastic optimum shifting technique utilizing the Neural Collapse theory to reduce computational costs and provide more degrees of freedom for optimum shifting. Extensive experiments (including classification and detection) with various deep neural network architectures on benchmark datasets demonstrate the effectiveness of our method.

Molecular deuteration is commonly seen in starless cores and is expected to occur on a timescale comparable to that of the core contraction. Thus, the deuteration serves as a chemical clock, allowing us to investigate dynamical theories of core formation. We aim to provide a 3D cloud description for the starless core L 1498 located in the nearby low-mass star-forming region Taurus, and explore the possible core formation mechanism of L 1498. We carried out non-local thermal equilibrium radiative transfer with multi-transition observations of the high-density tracer N$_2$H$^+$ to derive the density and temperature profiles of the L 1498 core. Combining with the spectral observations of the deuterated species, ortho-H$_2$D$^+$, N$_2$D$^+$, and DCO$^+$, we derived the abundance profiles for observed species and performed chemical modeling of the deuteration profiles across L 1498 to constrain the contraction timescale. We present the first ortho-H$_2$D$^+$ (1$_{10}$-1$_{11}$) detection toward L 1498. We find a peak molecular hydrogen density of $1.6_{-0.3}^{+3.0}\times10^{5}$~cm$^{-3}$, a temperature of 7.5$_{-0.5}^{+0.7}$~K, and a N$_2$H$^+$ deuteration of 0.27$_{-0.15}^{+0.12}$ in the center. We derive a lower limit of the core age for L 1498 of 0.16~Ma which is compatible with the typical free-fall time, indicating that L 1498 likely formed rapidly., Comment: 21 pages, 12 figures, accepted for publication in A&A

Continual learning (CL) provides a framework for training models in ever-evolving environments. Although re-occurrence of previously seen objects or tasks is common in real-world problems, the concept of repetition in the data stream is not often considered in standard benchmarks for CL. Unlike with the rehearsal mechanism in buffer-based strategies, where sample repetition is controlled by the strategy, repetition in the data stream naturally stems from the environment. This report provides a summary of the CLVision challenge at CVPR 2023, which focused on the topic of repetition in class-incremental learning. The report initially outlines the challenge objective and then describes three solutions proposed by finalist teams that aim to effectively exploit the repetition in the stream to learn continually. The experimental results from the challenge highlight the effectiveness of ensemble-based solutions that employ multiple versions of similar modules, each trained on different but overlapping subsets of classes. This report underscores the transformative potential of taking a different perspective in CL by employing repetition in the data stream to foster innovative strategy design., Comment: Preprint; Challenge Report of the 4th Workshop on Continual Learning in Computer Vision at CVPR

Protostars are born in magnetized environments. As a consequence, the formation of protostellar disks can be suppressed by the magnetic field efficiently removing angular momentum of the infalling material. Non-ideal MHD effects are proposed to as one way to allow protostellar disks to form. Thus, it is important to understand their contributions in observations of protostellar systems. We derive an analytical equation to estimate the ambipolar diffusivity coefficient at the edge of the protostellar disk in the Class 0/I protostar, HOPS-370, for the first time, under the assumption that the disk radius is set by ambipolar diffusion. Using previous results of the protostellar mass, disk mass, disk radius, density and temperature profiles and magnetic field strength, we estimate the ambipolar diffusivity coefficient to be $1.7^{+1.5}_{-1.4}\times10^{19}\,\mathrm{cm^{2}\,s^{-1}}$. We quantify the contribution of ambipolar diffusion by estimating its dimensionless Els\"{a}sser number to be $\sim1.7^{+1.0}_{-1.0}$, indicating its dynamical importance in this region. We compare to chemical calculations of the ambipolar diffusivity coefficient using the Non-Ideal magnetohydrodynamics Coefficients and Ionisation Library (NICIL), which is consistent with our results. In addition, we compare our derived ambipolar diffusivity coefficient to the diffusivity coefficients for Ohmic dissipation and the Hall effect, and find ambipolar diffusion is dominant in our density regime. These results demonstrate a new methodology to understand non-ideal MHD effects in observations of protostellar disks. More detailed modeling of the magnetic field, envelope and microphysics, along with a larger sample of protostellar systems is needed to further understand the contributions of non-ideal MHD., Comment: 20 pages, 5 figures. Accepted for publication in ApJ

The key to multi-label image classification (MLC) is to improve model performance by leveraging label correlations. Unfortunately, it has been shown that overemphasizing co-occurrence relationships can cause the overfitting issue of the model, ultimately leading to performance degradation. In this paper, we provide a causal inference framework to show that the correlative features caused by the target object and its co-occurring objects can be regarded as a mediator, which has both positive and negative impacts on model predictions. On the positive side, the mediator enhances the recognition performance of the model by capturing co-occurrence relationships; on the negative side, it has the harmful causal effect that causes the model to make an incorrect prediction for the target object, even when only co-occurring objects are present in an image. To address this problem, we propose a counterfactual reasoning method to measure the total direct effect, achieved by enhancing the direct effect caused only by the target object. Due to the unknown location of the target object, we propose patching-based training and inference to accomplish this goal, which divides an image into multiple patches and identifies the pivot patch that contains the target object. Experimental results on multiple benchmark datasets with diverse configurations validate that the proposed method can achieve state-of-the-art performance.

Active learning (AL) in open set scenarios presents a novel challenge of identifying the most valuable examples in an unlabeled data pool that comprises data from both known and unknown classes. Traditional methods prioritize selecting informative examples with low confidence, with the risk of mistakenly selecting unknown-class examples with similarly low confidence. Recent methods favor the most probable known-class examples, with the risk of picking simple already mastered examples. In this paper, we attempt to query examples that are both likely from known classes and highly informative, and propose a Bidirectional Uncertainty-based Active Learning (BUAL) framework. Specifically, we achieve this by first pushing the unknown class examples toward regions with high-confidence predictions, i.e., the proposed Random Label Negative Learning method. Then, we propose a Bidirectional Uncertainty sampling strategy by jointly estimating uncertainty posed by both positive and negative learning to perform consistent and stable sampling. BUAL successfully extends existing uncertainty-based AL methods to complex open-set scenarios. Extensive experiments on multiple datasets with varying openness demonstrate that BUAL achieves state-of-the-art performance. The code is available at https://github.com/chenchenzong/BUAL., Comment: Accepted to ECCV 2024

The rise of large language models (LLMs) has revolutionized the way that we interact with artificial intelligence systems through natural language. However, LLMs often misinterpret user queries because of their uncertain intention, leading to less helpful responses. In natural human interactions, clarification is sought through targeted questioning to uncover obscure information. Thus, in this paper, we introduce LaMAI (Language Model with Active Inquiry), designed to endow LLMs with this same level of interactive engagement. LaMAI leverages active learning techniques to raise the most informative questions, fostering a dynamic bidirectional dialogue. This approach not only narrows the contextual gap but also refines the output of the LLMs, aligning it more closely with user expectations. Our empirical studies, across a variety of complex datasets where LLMs have limited conversational context, demonstrate the effectiveness of LaMAI. The method improves answer accuracy from 31.9% to 50.9%, outperforming other leading question-answering frameworks. Moreover, in scenarios involving human participants, LaMAI consistently generates responses that are superior or comparable to baseline methods in more than 82% of the cases. The applicability of LaMAI is further evidenced by its successful integration with various LLMs, highlighting its potential for the future of interactive language models.

We report 850 $\mu$m continuum polarization observations toward the filamentary high-mass star-forming region NGC 2264, taken as part of the B-fields In STar forming Regions Observations (BISTRO) large program on the James Clerk Maxwell Telescope (JCMT). These data reveal a well-structured non-uniform magnetic field in the NGC 2264C and 2264D regions with a prevailing orientation around 30 deg from north to east. Field strengths estimates and a virial analysis for the major clumps indicate that NGC 2264C is globally dominated by gravity while in 2264D magnetic, gravitational, and kinetic energies are roughly balanced. We present an analysis scheme that utilizes the locally resolved magnetic field structures, together with the locally measured gravitational vector field and the extracted filamentary network. From this, we infer statistical trends showing that this network consists of two main groups of filaments oriented approximately perpendicular to one another. Additionally, gravity shows one dominating converging direction that is roughly perpendicular to one of the filament orientations, which is suggestive of mass accretion along this direction. Beyond these statistical trends, we identify two types of filaments. The type-I filament is perpendicular to the magnetic field with local gravity transitioning from parallel to perpendicular to the magnetic field from the outside to the filament ridge. The type-II filament is parallel to the magnetic field and local gravity. We interpret these two types of filaments as originating from the competition between radial collapsing, driven by filament self-gravity, and the longitudinal collapsing, driven by the region's global gravity., Comment: Accepted for publication in the Astrophysical Journal. 43 pages, 32 figures, and 4 tables (including Appendix)

Learning with noisy labels can significantly hinder the generalization performance of deep neural networks (DNNs). Existing approaches address this issue through loss correction or example selection methods. However, these methods often rely on the model's predictions obtained from the softmax function, which can be over-confident and unreliable. In this study, we identify the translation invariance of the softmax function as the underlying cause of this problem and propose the \textit{Dirichlet-based Prediction Calibration} (DPC) method as a solution. Our method introduces a calibrated softmax function that breaks the translation invariance by incorporating a suitable constant in the exponent term, enabling more reliable model predictions. To ensure stable model training, we leverage a Dirichlet distribution to assign probabilities to predicted labels and introduce a novel evidence deep learning (EDL) loss. The proposed loss function encourages positive and sufficiently large logits for the given label, while penalizing negative and small logits for other labels, leading to more distinct logits and facilitating better example selection based on a large-margin criterion. Through extensive experiments on diverse benchmark datasets, we demonstrate that DPC achieves state-of-the-art performance. The code is available at https://github.com/chenchenzong/DPC.

We present JCMT POL-2 850 um dust polarization observations and Mimir H band stellar polarization observations toward the starless core L1512. We detect the highly-ordered core-scale magnetic field traced by the POL-2 data, of which the field orientation is consistent with the parsec-scale magnetic fields traced by Planck data, suggesting the large-scale fields thread from the low-density region to the dense core region in this cloud. The surrounding magnetic field traced by the Mimir data shows a wider variation in the field orientation, suggesting there could be a transition of magnetic field morphology at the envelope scale. L1512 was suggested to be presumably older than 1.4 Myr in a previous study via time-dependent chemical analysis, hinting that the magnetic field could be strong enough to slow the collapse of L1512. In this study, we use the Davis-Chandrasekhar-Fermi method to derive a plane-of-sky magnetic field strength ($B_{pos}$) of 18$\pm$7 uG and an observed mass-to-flux ratio ($\lambda_{obs}$) of 3.5$\pm$2.4, suggesting that L1512 is magnetically supercritical. However, the absence of significant infall motion and the presence of an oscillating envelope are inconsistent with the magnetically supercritical condition. Using a Virial analysis, we suggest the presence of a hitherto hidden line-of-sight magnetic field strength of ~27 uG with a mass-to-flux ratio ($\lambda_{tot}$) of ~1.6, in which case both magnetic and kinetic pressures are important in supporting the L1512 core. On the other hand, L1512 may have just reached supercriticality and will collapse at any time., Comment: 25 pages, 10 figures, accepted for publication in ApJ

The internal structure of the prestellar core G208.68-19.02-N2 (G208-N2) in the Orion Molecular Cloud 3 (OMC-3) region has been studied with the Atacama Large Millimeter/submillimeter Array (ALMA). The dust continuum emission revealed a filamentary structure with a length of $\sim$5000 au and an average H$_2$ volume density of $\sim$6 $\times$ 10$^7$ cm$^{-3}$. At the tip of this filamentary structure, there is a compact object, which we call a ``nucleus", with a radius of $\sim$150--200 au and a mass of $\sim$0.1 M$_{\odot}$. The nucleus has a central density of $\sim$2 $\times$ 10$^9$ cm$^{-3}$ with a radial density profile of $r^{-1.87{\pm}0.11}$. The density scaling of the nucleus is $\sim$3.7 times higher than that of the singular isothermal sphere. This as well as the very low virial parameter of 0.39 suggest that the gravity is dominant over the pressure everywhere in the nucleus. However, there is no sign of CO outflow localized to this nucleus. The filamentary structure is traced by the N$_2$D$^+$ 3--2 emission, but not by the C$^{18}$O 2--1 emission, implying the significant CO depletion due to high density and cold temperature. Toward the nucleus, the N$_2$D$^+$ also shows the signature of depletion. This could imply either the depletion of the parent molecule, N$_2$, or the presence of the embedded very-low luminosity central source that could sublimate the CO in the very small area. The nucleus in G208-N2 is considered to be a prestellar core on the verge of first hydrostatic core (FHSC) formation or a candidate for the FHSC., Comment: 27 pages, 16 figures

When taking images against strong light sources, the resulting images often contain heterogeneous flare artifacts. These artifacts can importantly affect image visual quality and downstream computer vision tasks. While collecting real data pairs of flare-corrupted/flare-free images for training flare removal models is challenging, current methods utilize the direct-add approach to synthesize data. However, these methods do not consider automatic exposure and tone mapping in image signal processing pipeline (ISP), leading to the limited generalization capability of deep models training using such data. Besides, existing methods struggle to handle multiple light sources due to the different sizes, shapes and illuminance of various light sources. In this paper, we propose a solution to improve the performance of lens flare removal by revisiting the ISP and remodeling the principle of automatic exposure in the synthesis pipeline and design a more reliable light sources recovery strategy. The new pipeline approaches realistic imaging by discriminating the local and global illumination through convex combination, avoiding global illumination shifting and local over-saturation. Our strategy for recovering multiple light sources convexly averages the input and output of the neural network based on illuminance levels, thereby avoiding the need for a hard threshold in identifying light sources. We also contribute a new flare removal testing dataset containing the flare-corrupted images captured by ten types of consumer electronics. The dataset facilitates the verification of the generalization capability of flare removal methods. Extensive experiments show that our solution can effectively improve the performance of lens flare removal and push the frontier toward more general situations., Comment: ICCV 2023

Existing knowledge distillation methods typically work by imparting the knowledge of output logits or intermediate feature maps from the teacher network to the student network, which is very successful in multi-class single-label learning. However, these methods can hardly be extended to the multi-label learning scenario, where each instance is associated with multiple semantic labels, because the prediction probabilities do not sum to one and feature maps of the whole example may ignore minor classes in such a scenario. In this paper, we propose a novel multi-label knowledge distillation method. On one hand, it exploits the informative semantic knowledge from the logits by dividing the multi-label learning problem into a set of binary classification problems; on the other hand, it enhances the distinctiveness of the learned feature representations by leveraging the structural information of label-wise embeddings. Experimental results on multiple benchmark datasets validate that the proposed method can avoid knowledge counteraction among labels, thus achieving superior performance against diverse comparing methods. Our code is available at: https://github.com/penghui-yang/L2D, Comment: Accepted by ICCV 2023. The first two authors contributed equally to this work

We present observations of polarized dust emission at 850 $\mu$m from the L43 molecular cloud which sits in the Ophiuchus cloud complex. The data were taken using SCUBA-2/POL-2 on the James Clerk Maxwell Telescope as a part of the BISTRO large program. L43 is a dense ($N_{\rm H_2}\sim 10^{22}$-10$^{23}$ cm$^{-2}$) complex molecular cloud with a submillimetre-bright starless core and two protostellar sources. There appears to be an evolutionary gradient along the isolated filament that L43 is embedded within, with the most evolved source closest to the Sco OB2 association. One of the protostars drives a CO outflow that has created a cavity to the southeast. We see a magnetic field that appears to be aligned with the cavity walls of the outflow, suggesting interaction with the outflow. We also find a magnetic field strength of up to $\sim$160$\pm$30 $\mu$G in the main starless core and up to $\sim$90$\pm$40 $\mu$G in the more diffuse, extended region. These field strengths give magnetically super- and sub-critical values respectively and both are found to be roughly trans-Alfv\'enic. We also present a new method of data reduction for these denser but fainter objects like starless cores., Comment: Accepted for publication in ApJ. 23 pages, 9 figures (7 main text, 2 appendix)

Learning from noisy data has attracted much attention, where most methods focus on closed-set label noise. However, a more common scenario in the real world is the presence of both open-set and closed-set noise. Existing methods typically identify and handle these two types of label noise separately by designing a specific strategy for each type. However, in many real-world scenarios, it would be challenging to identify open-set examples, especially when the dataset has been severely corrupted. Unlike the previous works, we explore how models behave when faced with open-set examples, and find that \emph{a part of open-set examples gradually get integrated into certain known classes}, which is beneficial for the separation among known classes. Motivated by the phenomenon, we propose a novel two-step contrastive learning method CECL (Class Expansion Contrastive Learning) which aims to deal with both types of label noise by exploiting the useful information of open-set examples. Specifically, we incorporate some open-set examples into closed-set classes to enhance performance while treating others as delimiters to improve representative ability. Extensive experiments on synthetic and real-world datasets with diverse label noise demonstrate the effectiveness of CECL.

Pseudo-labeling has emerged as a popular and effective approach for utilizing unlabeled data. However, in the context of semi-supervised multi-label learning (SSMLL), conventional pseudo-labeling methods encounter difficulties when dealing with instances associated with multiple labels and an unknown label count. These limitations often result in the introduction of false positive labels or the neglect of true positive ones. To overcome these challenges, this paper proposes a novel solution called Class-Aware Pseudo-Labeling (CAP) that performs pseudo-labeling in a class-aware manner. The proposed approach introduces a regularized learning framework incorporating class-aware thresholds, which effectively control the assignment of positive and negative pseudo-labels for each class. Notably, even with a small proportion of labeled examples, our observations demonstrate that the estimated class distribution serves as a reliable approximation. Motivated by this finding, we develop a class-distribution-aware thresholding strategy to ensure the alignment of pseudo-label distribution with the true distribution. The correctness of the estimated class distribution is theoretically verified, and a generalization error bound is provided for our proposed method. Extensive experiments on multiple benchmark datasets confirm the efficacy of CAP in addressing the challenges of SSMLL problems.

Evaluating the performance of low-light image enhancement (LLE) is highly subjective, thus making integrating human preferences into image enhancement a necessity. Existing methods fail to consider this and present a series of potentially valid heuristic criteria for training enhancement models. In this paper, we propose a new paradigm, i.e., aesthetics-guided low-light image enhancement (ALL-E), which introduces aesthetic preferences to LLE and motivates training in a reinforcement learning framework with an aesthetic reward. Each pixel, functioning as an agent, refines itself by recursive actions, i.e., its corresponding adjustment curve is estimated sequentially. Extensive experiments show that integrating aesthetic assessment improves both subjective experience and objective evaluation. Our results on various benchmarks demonstrate the superiority of ALL-E over state-of-the-art methods.

Physics-informed neural networks (PINNs) have effectively been demonstrated in solving forward and inverse differential equation problems, but they are still trapped in training failures when the target functions to be approximated exhibit high-frequency or multi-scale features. In this paper, we propose to employ implicit stochastic gradient descent (ISGD) method to train PINNs for improving the stability of training process. We heuristically analyze how ISGD overcome stiffness in the gradient flow dynamics of PINNs, especially for problems with multi-scale solutions. We theoretically prove that for two-layer fully connected neural networks with large hidden nodes, randomly initialized ISGD converges to a globally optimal solution for the quadratic loss function. Empirical results demonstrate that ISGD works well in practice and compares favorably to other gradient-based optimization methods such as SGD and Adam, while can also effectively address the numerical stiffness in training dynamics via gradient descent., Comment: 17 pages, published as a conference paper at AAAI23

We present BISTRO Survey 850 {\mu}m dust emission polarisation observations of the L1495A-B10 region of the Taurus molecular cloud, taken at the JCMT. We observe a roughly triangular network of dense filaments. We detect 9 of the dense starless cores embedded within these filaments in polarisation, finding that the plane-of-sky orientation of the core-scale magnetic field lies roughly perpendicular to the filaments in almost all cases. We also find that the large-scale magnetic field orientation measured by Planck is not correlated with any of the core or filament structures, except in the case of the lowest-density core. We propose a scenario for early prestellar evolution that is both an extension to, and consistent with, previous models, introducing an additional evolutionary transitional stage between field-dominated and matter-dominated evolution, observed here for the first time. In this scenario, the cloud collapses first to a sheet-like structure. Uniquely, we appear to be seeing this sheet almost face-on. The sheet fragments into filaments, which in turn form cores. However, the material must reach a certain critical density before the evolution changes from being field-dominated to being matter-dominated. We measure the sheet surface density and the magnetic field strength at that transition for the first time and show consistency with an analytical prediction that had previously gone untested for over 50 years (Mestel 1965)., Comment: 14 pages, 5 figures. ApJ accepted

Recent aerial object detection models rely on a large amount of labeled training data, which requires unaffordable manual labeling costs in large aerial scenes with dense objects. Active learning effectively reduces the data labeling cost by selectively querying the informative and representative unlabelled samples. However, existing active learning methods are mainly with class-balanced settings and image-based querying for generic object detection tasks, which are less applicable to aerial object detection scenarios due to the long-tailed class distribution and dense small objects in aerial scenes. In this paper, we propose a novel active learning method for cost-effective aerial object detection. Specifically, both object-level and image-level informativeness are considered in the object selection to refrain from redundant and myopic querying. Besides, an easy-to-use class-balancing criterion is incorporated to favor the minority objects to alleviate the long-tailed class distribution problem in model training. We further devise a training loss to mine the latent knowledge in the unlabeled image regions. Extensive experiments are conducted on the DOTA-v1.0 and DOTA-v2.0 benchmarks to validate the effectiveness of the proposed method. For the ReDet, KLD, and SASM detectors on the DOTA-v2.0 dataset, the results show that our proposed MUS-CDB method can save nearly 75\% of the labeling cost while achieving comparable performance to other active learning methods in terms of mAP.Code is publicly online (https://github.com/ZJW700/MUS-CDB)., Comment: 13 pages, 7 figures

We present 850 $\mu$m dust polarization observations of the massive DR21 filament from the B-fields In STar-forming Region Observations (BISTRO) survey, using the POL-2 polarimeter and the SCUBA-2 camera on the James Clerk Maxwell Telescope. We detect ordered magnetic fields perpendicular to the parsec-scale ridge of the DR21 main filament. In the sub-filaments, the magnetic fields are mainly parallel to the filamentary structures and smoothly connect to the magnetic fields of the main filament. We compare the POL-2 and Planck dust polarization observations to study the magnetic field structures of the DR21 filament on 0.1--10 pc scales. The magnetic fields revealed in the Planck data are well aligned with those of the POL-2 data, indicating a smooth variation of magnetic fields from large to small scales. The plane-of-sky magnetic field strengths derived from angular dispersion functions of dust polarization are 0.6--1.0 mG in the DR21 filament and $\sim$ 0.1 mG in the surrounding ambient gas. The mass-to-flux ratios are found to be magnetically supercritical in the filament and slightly subcritical to nearly critical in the ambient gas. The alignment between column density structures and magnetic fields changes from random alignment in the low-density ambient gas probed by Planck to mostly perpendicular in the high-density main filament probed by JCMT. The magnetic field structures of the DR21 filament are in agreement with MHD simulations of a strongly magnetized medium, suggesting that magnetic fields play an important role in shaping the DR21 main filament and sub-filaments., Comment: 26 pages, 13 figures, ApJ accepted