Search

Autoregressive models have demonstrated great performance in natural language processing (NLP) with impressive scalability, adaptability and generalizability. Inspired by their notable success in NLP field, autoregressive models have been intensively investigated recently for computer vision, which perform next-token predictions by representing visual data as visual tokens and enables autoregressive modelling for a wide range of vision tasks, ranging from visual generation and visual understanding to the very recent multimodal generation that unifies visual generation and understanding with a single autoregressive model. This paper provides a systematic review of vision autoregressive models, including the development of a taxonomy of existing methods and highlighting their major contributions, strengths, and limitations, covering various vision tasks such as image generation, video generation, image editing, motion generation, medical image analysis, 3D generation, robotic manipulation, unified multimodal generation, etc. Besides, we investigate and analyze the latest advancements in autoregressive models, including thorough benchmarking and discussion of existing methods across various evaluation datasets. Finally, we outline key challenges and promising directions for future research, offering a roadmap to guide further advancements in vision autoregressive models., Comment: This work will be integrated into another project

This work proposes and analyzes an efficient numerical method for solving the nonlinear Schr\"odinger equation with quasiperiodic potential, where the projection method is applied in space to account for the quasiperiodic structure and the Strang splitting method is used in time.While the transfer between spaces of low-dimensional quasiperiodic and high-dimensional periodic functions and its coupling with the nonlinearity of the operator splitting scheme make the analysis more challenging. Meanwhile, compared to conventional numerical analysis of periodic Schr\"odinger systems, many of the tools and theories are not applicable to the quasiperiodic case. We address these issues to prove the spectral accuracy in space and the second-order accuracy in time. Numerical experiments are performed to substantiate the theoretical findings.

Remote Sensing Image Captioning (RSIC) presents unique challenges and plays a critical role in applications. Traditional RSIC methods often struggle to produce rich and diverse descriptions. Recently, with advancements in VLMs, efforts have emerged to integrate these models into the remote sensing domain and to introduce descriptive datasets specifically designed to enhance VLM training. This paper proposes RS-MoE, a first Mixture of Expert based VLM specifically customized for remote sensing domain. Unlike traditional MoE models, the core of RS-MoE is the MoE Block, which incorporates a novel Instruction Router and multiple lightweight Large Language Models (LLMs) as expert models. The Instruction Router is designed to generate specific prompts tailored for each corresponding LLM, guiding them to focus on distinct aspects of the RSIC task. This design not only allows each expert LLM to concentrate on a specific subset of the task, thereby enhancing the specificity and accuracy of the generated captions, but also improves the scalability of the model by facilitating parallel processing of sub-tasks. Additionally, we present a two-stage training strategy for tuning our RS-MoE model to prevent performance degradation due to sparsity. We fine-tuned our model on the RSICap dataset using our proposed training strategy. Experimental results on the RSICap dataset, along with evaluations on other traditional datasets where no additional fine-tuning was applied, demonstrate that our model achieves state-of-the-art performance in generating precise and contextually relevant captions. Notably, our RS-MoE-1B variant achieves performance comparable to 13B VLMs, demonstrating the efficiency of our model design. Moreover, our model demonstrates promising generalization capabilities by consistently achieving state-of-the-art performance on the Remote Sensing Visual Question Answering (RSVQA) task.

Generalizing to out-of-distribution data while being aware of model fairness is a significant and challenging problem in meta-learning. The goal of this problem is to find a set of fairness-aware invariant parameters of classifier that is trained using data drawn from a family of related training domains with distribution shift on non-sensitive features as well as different levels of dependence between model predictions and sensitive features so that the classifier can achieve good generalization performance on unknown but distinct test domains. To tackle this challenge, existing state-of-the-art methods either address the domain generalization problem but completely ignore learning with fairness or solely specify shifted domains with various fairness levels. This paper introduces an approach to fairness-aware meta-learning that significantly enhances domain generalization capabilities. Our framework, Fairness-Enhanced Meta-Learning for Domain Generalization (FEED), disentangles latent data representations into content, style, and sensitive vectors. This disentanglement facilitates the robust generalization of machine learning models across diverse domains while adhering to fairness constraints. Unlike traditional methods that focus primarily on domain invariance or sensitivity to shifts, our model integrates a fairness-aware invariance criterion directly into the meta-learning process. This integration ensures that the learned parameters uphold fairness consistently, even when domain characteristics vary widely. We validate our approach through extensive experiments across multiple benchmarks, demonstrating not only superior performance in maintaining high accuracy and fairness but also significant improvements over existing state-of-the-art methods in domain generalization tasks., Comment: IEEE International Conference on Big Data 2024

Crystalline defects play a critical role in determining the properties of crystalline solids, underscoring the need for accurate computational methods to study them. Lattice deformation in dislocation simulations, which involves changes in atomic positions, can be described either microscopically by specific atomic configurations or macroscopically by continuum elasticity, each with inherent limitations. The complex amplitude expansion of the phase field crystal (APFC) model provides a mesoscopic approach that bridges these scales. In this paper, we introduce a Fourier pseudospectral method for efficiently solving the APFC model in the context of crystalline defect simulations. This study marks the first application of the Fourier pseudospectral method to the APFC model. The method fully exploits the system's periodicity and facilitates the implementation of periodic boundary conditions, thanks to its high accuracy and computational efficiency. Numerical experiments conducted on two-dimensional triangular lattices and three-dimensional body-centered cubic lattices for edge dislocation geometry optimization have produced strain field images that align well with by continuum elasticity predictions. The findings demonstrate the potential of the APFC model to accurately capture the complex strain fields associated with dislocations at the mesoscopic scales, a key step toward modeling more intricate crystalline defect structures and dynamics.

Recent studies on generalizable object detection have attracted increasing attention with additional weak supervision from large-scale datasets with image-level labels. However, weakly-supervised detection learning often suffers from image-to-box label mismatch, i.e., image-level labels do not convey precise object information. We design Language Hierarchical Self-training (LHST) that introduces language hierarchy into weakly-supervised detector training for learning more generalizable detectors. LHST expands the image-level labels with language hierarchy and enables co-regularization between the expanded labels and self-training. Specifically, the expanded labels regularize self-training by providing richer supervision and mitigating the image-to-box label mismatch, while self-training allows assessing and selecting the expanded labels according to the predicted reliability. In addition, we design language hierarchical prompt generation that introduces language hierarchy into prompt generation which helps bridge the vocabulary gaps between training and testing. Extensive experiments show that the proposed techniques achieve superior generalization performance consistently across 14 widely studied object detection datasets., Comment: NeurIPS 2024 Camera Ready

X-shaped liquid crystalline molecules (XLCMs) are obtained by tethering two flexible end A-blocks and two flexible side B-blocks to a rigid backbone (R). A rich array of ordered structures can be formed from XLCMs, driven by the competition between the interactions between the chemically distinct blocks and the molecular connectivity. Here, we report a theoretical study on the phase behaviour of XLCMs with symmetric and asymmetric side blocks by using the self-consistent field theory (SCFT). A large number of ordered structures, including stable smectic-A, triangle-square, pentagon and giant polygon, are obtained as solutions of the SCFT equations. Phase diagrams of XLCMs as a function of the total length and asymmetric ratio of the side chains are constructed. For XLCMs with symmetric side blocks, the theoretically predicted phase transition sequence is in good agreement with experiments. For XLCMs with a fixed total side chain length, transitions between layered structure to polygonal phases, as well as between different polygonal phases, could be induced by varying the asymmetry of the side chains. The free energy density, domain size, side-chain stretching , and molecular orientation are analyzed to elucidate mechanisms stabilizing the different ordered phases.

Quasicrystals are intriguing ordered structures characterized by the lack of translational symmetry and the existence of rotational symmetry. The tiling of different geometric units such as triangles and squares in two-dimensional space can result in a great variety of quasicrystals that could be realized by the self-assembly of liquid crystalline molecules. In this study, we introduce three self-similar dodecagonal tilings, including a novel Diamond-Square-Triangle pattern, composed of triangular and quadrangular tiles and examine their thermodynamic stability by using the self-consistent field theory applied to T-shaped liquid crystalline molecules. Specifically, we detail the inflation rules for the construction of these dodecagonal tilings and analyze their self-similarity, and show that these tilings can be viewed as projections of higher-dimensional periodic lattice points with projection windows. Using these dodecagonal tilings as initial configurations of the SCFT results in solutions corresponding to quasicrystals that could form from the T-shaped liquid crystalline molecules. The relative stability of these aperiodic phases is analyzed to obtain design rules that could stabilize quasicrystals. Meanwhile, we provide two criteria for distinguishing three dodecagonal quasicrystals and their approximants by analyzing their diffraction peaks. These findings shed new lighten on the discovery of new quasicrystals in soft materials.

The recent experiment shows the superconducting transition temperature in the Ruddlesden-Popper bilayer La$_3$Ni$_2$O$_{7}$ decreases monotonically with increasing pressure above 14 GPa. In order to unravel the underlying mechanism for this unusual dependence, we performed theoretical investigations by combining the density functional theory (DFT) and the unbiased functional renormalization group (FRG). Our DFT calculations show that the Fermi pockets are essentially unchanged with increasing pressure (above 14 GPa), but the bandwidth is enlarged, and particularly the interlayer hopping integral between the nickel $3d_{3z^2-r^2}$ orbitals is enhanced. From the DFT band structure, we construct the bilayer tight-binding model in terms of the nickel $3d_{3z^2-r^2}$ and $3d_{x^2-y^2}$ orbitals. On this basis, we investigate the superconductivity induced by correlation effects by FRG calculations. We find consistently $s_\pm$-wave pairing triggered by spin fluctuations, but the latter are weakened by pressure and lead to a decreasing transition temperature versus pressure, in qualitatively agreement with the experiment. We emphasize that the itinerancy of the $d$-orbitals is important and captured naturally in our FRG calculations, and we argue that the unusual pressure dependence would be unnatural, if not impossible, in the otherwise local-moment picture of the nickel $d$-orbitals. This sheds lights on the pertinent microscopic description of, and more importantly the mechanism of superconductivity in La$_3$Ni$_2$O$_{7}$., Comment: 8 pages, 4 figures

Interparticle interactions with multiple length scales play a pivotal role in the formation and stability of quasicrystals. Choosing a minimal set of length scales to stabilize a given quasicrystal is a challenging problem. To address this challenge, we propose an intelligent screening method (ISM) to design a Landau theory with a minimal number of length scales -- referred to as the minimal Landau theory -- that includes only the essential length scales necessary to stabilize quasicrystals. Based on a generalized multiple-length-scale Landau theory, ISM first evaluates various spectral configurations of candidate structures under a hard constraint. It then identifies the configuration with the lowest free energy. Using this optimal configuration, ISM calculates phase diagrams to explore the thermodynamic stability of desired quasicrystals. ISM can design a minimal Landau theory capable of stabilizing the desired quasicrystals by incrementally increasing the number of length scales. Our application of ISM has not only confirmed known behaviors in 10- and 12-fold quasicrystals but also led to a significant prediction that quasicrystals with 8-, 14-, 16-, and 18-fold symmetry could be stable within three-length-scale Landau models., Comment: 23 pages, 10 figures, 2 long tables

The phase behavior of symmetric diblock copolymers under three-dimensional (3D) soft confinement is investigated using the self-consistent field theory. The soft confinement is realized in binary blends composed AB diblock copolymers and C homopolymers, where the copolymers self-assemble to form a droplet embedded in the homopolymer matrix. The phase behavior of the confined block copolymers is regulated by the degree of confinement and the selectivity of the homopolymers, resulting in a rich variety of novel structures. When the C homopolymers are neutral to the A- and B-blocks, stacked lamellae (SL) are formed where the number of layers increases with the droplet volume, resulting in a morphological transition sequence from Janus particle to square SL. When the C homopolymers are strongly selective to the B-blocks, a series of non-lamellar morphologies, including onion-, hamburger-, cross-, ring-, and cookie-like structures, are observed. A detailed free energy analysis reveals a first-order reversible transformation between SL and onion-like (OL) structures when the selectivity of the homopolymers is changed. Our results provide a comprehensive understanding of how various factors, such as the copolymer concentration, homopolymer chain length, degree of confinement, homopolymer selectivity, affect the self-assembled structures of diblock copolymers under soft 3D confinement., Comment: 9pages,7figures

Recently, deep learning-based hybrid iterative methods (DL-HIM) have emerged as a promising approach for designing fast neural solvers to tackle large-scale sparse linear systems. DL-HIM combine the smoothing effect of simple iterative methods with the spectral bias of neural networks, which allows them to effectively eliminate both high-frequency and low-frequency error components. However, their efficiency may decrease if simple iterative methods can not provide effective smoothing, making it difficult for the neural network to learn mid-frequency and high-frequency components. This paper first establishes a convergence analysis framework for general DL-HIM from a spectral viewpoint, concluding that under reasonable assumptions, DL-HIM exhibit a convergence rate independent of grid size $h$ and physical parameters $\boldsymbol{\mu}$. To meet these assumptions, we design a neural network from an eigen perspective, focusing on learning the eigenvalues and eigenvectors corresponding to error components that simple iterative methods struggle to eliminate. Specifically, the eigenvalues are learned by a meta subnet, while the eigenvectors are approximated using Fourier modes with a transition matrix provided by another meta subnet. The resulting DL-HIM, termed the Fourier Neural Solver (FNS), is expected to achieve a $h, \boldsymbol{\mu}$-independent convergence rate with the guarantee of universal approximation and the discretization invariant property of the meta subnets. We verify the performance of FNS on four types of linear parametric PDEs.

In this work, we study the nucleation of quasicrystals from liquid or periodic crystals by developing an efficient order-order phase transition algorithm, namely the nullspace-preserving saddle search method. Specifically, we focus on nucleation and phase transitions of the decagonal quasicrystal (DQC) based on the Lifshitz-Petrich model. We present the nucleation path of DQC from the liquid and demonstrate one- and two-stage transition paths between DQC and periodic crystals. We provide a perspective of the group-subgroup phase transition and nucleation rates to understand the nucleation and phase transition mechanisms involving DQC. These results reveal the one-step and stepwise modes of symmetry breaking or recovery in the phase transition from DQC, where the stepwise modes are more probable.

The mental health assessment of middle school students has always been one of the focuses in the field of education. This paper introduces a new ensemble learning network based on BERT, employing the concept of enhancing model performance by integrating multiple classifiers. We trained a range of BERT-based learners, which combined using the majority voting method. We collect social network text data of middle school students through China's Weibo and apply the method to the task of classifying emotional tendencies in middle school students' social network texts. Experimental results suggest that the ensemble learning network has a better performance than the base model and the performance of the ensemble learning model, consisting of three single-layer BERT models, is barely the same as a three-layer BERT model but requires 11.58% more training time. Therefore, in terms of balancing prediction effect and efficiency, the deeper BERT network should be preferred for training. However, for interpretability, network ensembles can provide acceptable solutions.

Finding index-1 saddle points is crucial for understanding phase transitions. In this work, we propose a simple yet efficient approach, the spring pair method (SPM), to accurately locate saddle points. Without requiring Hessian information, SPM evolves a single pair of spring-coupled particles on the potential energy surface. By cleverly designing complementary drifting and climbing dynamics based on gradient decomposition, the spring pair converges onto the minimum energy path (MEP) and spontaneously aligns its orientation with the MEP tangent, providing a reliable ascent direction for efficient convergence to saddle points. SPM fundamentally differs from traditional surface walking methods, which rely on the eigenvectors of Hessian that may deviate from the MEP tangent, potentially leading to convergence failure or undesired saddle points. The efficiency of SPM for finding saddle points is verified by ample examples, including high-dimensional Lennard-Jones cluster rearrangement and the Landau energy functional involving quasicrystal phase transitions., Comment: 9 pages, 7 figures

In this work, we investigate [110] symmetric tilt FCC grain boundaries (GBs) by a recently developed approach for quasiperiodic interfaces using the Landau-Brazovskii model. On special tilt angles associated with quadratic algebraic numbers, quasiperiodic GBs exhibit generalized Fibonacci substitution rules. The transition mechanism from quasiperiodic GBs to periodic GBs is explored through sphere offsets and spectral coalescence. We also propose an accurate method to calculate the GB energy for arbitrary tilt angle and analyze the factors affecting the GB energy. The revealing GB energy change is continue along with tilt angle except periodic GBs.

The majority of existing hyperspectral anomaly detection (HAD) methods use the low-rank representation (LRR) model to separate the background and anomaly components, where the anomaly component is optimized by handcrafted sparse priors (e.g., $\ell_{2,1}$-norm). However, this may not be ideal since they overlook the spatial structure present in anomalies and make the detection result largely dependent on manually set sparsity. To tackle these problems, we redefine the optimization criterion for the anomaly component in the LRR model with a self-supervised network called self-supervised anomaly prior (SAP). This prior is obtained by the pretext task of self-supervised learning, which is customized to learn the characteristics of hyperspectral anomalies. Specifically, this pretext task is a classification task to distinguish the original hyperspectral image (HSI) and the pseudo-anomaly HSI, where the pseudo-anomaly is generated from the original HSI and designed as a prism with arbitrary polygon bases and arbitrary spectral bands. In addition, a dual-purified strategy is proposed to provide a more refined background representation with an enriched background dictionary, facilitating the separation of anomalies from complex backgrounds. Extensive experiments on various hyperspectral datasets demonstrate that the proposed SAP offers a more accurate and interpretable solution than other advanced HAD methods.

Self-consistent field theory (SCFT) is one of the most widely-used framework in studying the equilibrium phase behaviors of inhomogenous polymers. For liquid crystalline polymeric systems, the main numerical challenges of solving SCFT encompass efficiently solving plenty of six dimensional partial differential equations (PDEs), precisely determining the subtle energy difference among self-assembled structures, and developing effective iterative methods for nonlinear SCF iteration. To address these challenges, this work introduces a suite of high-order and efficient numerical methods tailored for SCFT of liquid-crystalline polymers. These methods include various advaced PDE solvers, an improved Anderson iteration algorithm to accelerate SCFT calculations, and an optimization technique of adjusting the computational domain during the SCF iterations. Extensive numerical tests demonstrate the efficiency of the proposed methods. Based on these algorithms, we further explore the self-assembly behavior of liquid crystalline polymers through simulations in four, five, and six dimensions, uncovering intricate three-dimensional spatial structures., Comment: 34 pages, 13figures

In this study, we address the challenge of solving elliptic equations with quasiperiodic coefficients. To achieve accurate and efficient computation, we introduce the projection method, which enables the embedding of quasiperiodic systems into higher-dimensional periodic systems. To enhance the computational efficiency, we propose a compressed storage strategy for the stiffness matrix by its multi-level block circulant structure, significantly reducing memory requirements. Furthermore, we design a diagonal preconditioner to efficiently solve the resulting high-dimensional linear system by reducing the condition number of the stiffness matrix. These techniques collectively contribute to the computational effectiveness of our proposed approach. Convergence analysis shows the spectral accuracy of the proposed method. We demonstrate the effectiveness and accuracy of our approach through a series of numerical examples. Moreover, we apply our method to achieve a highly accurate computation of the homogenized coefficients for a quasiperiodic multiscale elliptic equation.

In this paper, we propose a new algorithm, the irrational-window-filter projection method (IWFPM), for quasiperiodic systems with concentrated spectral point distribution. Based on the projection method (PM), IWFPM filters out dominant spectral points by defining an irrational window and uses a corresponding index-shift transform to make the FFT available. The error analysis on the function approximation level is also given. We apply IWFPM to 1D, 2D, and 3D quasiperiodic Schr\"odinger eigenproblems (QSEs) to demonstrate its accuracy and efficiency. IWFPM exhibits a significant computational advantage over PM for both extended and localized quantum states. More importantly, by using IWFPM, the existence of Anderson localization in 2D and 3D QSEs is numerically verified.

Solving high-wavenumber and heterogeneous Helmholtz equations presents a long-standing challenge in scientific computing. In this paper, we introduce a deep learning-enhanced multigrid solver to address this issue. By conducting error analysis on standard multigrid applied to a discrete Helmholtz equation, we devise a strategy to handle errors with different frequencies separately. For error components with frequencies distant from the wavenumber, we perform simple smoothing based on local operations at different levels to eliminate them. On the other hand, to address error components with frequencies near the wavenumber, we utilize another multigrid V-cycle to solve an advection-diffusion-reaction (ADR) equation at a coarse scale. The resulting solver, named Wave-ADR-NS, involves parameters learned through unsupervised training. Numerical results demonstrate that Wave-ADR-NS effectively resolves heterogeneous 2D Helmholtz equation with wavenumber up to 2000. Comparative experiments against classical multigrid preconditioners and existing deep learning-based multigrid preconditioners reveals the superior performance of Wave-ADR-NS.

The fairness-aware online learning framework has emerged as a potent tool within the context of continuous lifelong learning. In this scenario, the learner's objective is to progressively acquire new tasks as they arrive over time, while also guaranteeing statistical parity among various protected sub-populations, such as race and gender, when it comes to the newly introduced tasks. A significant limitation of current approaches lies in their heavy reliance on the i.i.d (independent and identically distributed) assumption concerning data, leading to a static regret analysis of the framework. Nevertheless, it's crucial to note that achieving low static regret does not necessarily translate to strong performance in dynamic environments characterized by tasks sampled from diverse distributions. In this paper, to tackle the fairness-aware online learning challenge in evolving settings, we introduce a unique regret measure, FairSAR, by incorporating long-term fairness constraints into a strongly adapted loss regret framework. Moreover, to determine an optimal model parameter at each time step, we introduce an innovative adaptive fairness-aware online meta-learning algorithm, referred to as FairSAOML. This algorithm possesses the ability to adjust to dynamic environments by effectively managing bias control and model accuracy. The problem is framed as a bi-level convex-concave optimization, considering both the model's primal and dual parameters, which pertain to its accuracy and fairness attributes, respectively. Theoretical analysis yields sub-linear upper bounds for both loss regret and the cumulative violation of fairness constraints. Our experimental evaluation on various real-world datasets in dynamic environments demonstrates that our proposed FairSAOML algorithm consistently outperforms alternative approaches rooted in the most advanced prior online learning methods., Comment: Accepted by TKDD, extended from KDD 2022. arXiv admin note: substantial text overlap with arXiv:2205.11264

We study four double-gyroid (DG) grain boundaries (GBs) with different orientations numerically using the Landau--Brazovskii free energy, including the (422) twin boundary studied recently, a network switching GB, and two tilt GBs. Topological variations and geometric deformations are investigated. It is found that deviations in strut lengths and dihedral angles from the bulk DG substantially exceed changes in strut angles and nodal coplanarity. We also examine the spectra along the contact plane of two grains and utilize them to evaluate the GB widths. Of the four GBs we study, the network switching GB changes to the least extent topologically and geometrically, meanwhile has the lowest energy and the smallest GB width.

The recent discovery of bulk superconductivity in trilayer nickelate La$_4$Ni$_3$O$_{10}$ with the critical temperature $T_c$ near $30$K under high pressure is attracting a new wave of research interest, after the breakthrough of bilayer La$_3$Ni$_2$O$_7$ with $T_c$ near $80$K. The similarities and differences of electronic structure and superconducting mechanism in these two systems are urgent theoretical issues. In this Letter, we study the electronic band structure and construct a minimal trilayer tight-binding model for the high-pressure phase of La$_4$Ni$_3$O$_{10}$ in terms of the nickel $3d_{x^2-y^2}$ and $3d_{3z^2-r^2}$ orbitals, and study the superconducting mechanism due to local Coulomb interactions by the unbiased functional renormalization group. We find antiferromagnetic correlations between the outer layers instead of neighboring ones, apart from the inplane correlations. The effective interaction induces Cooper pairing with the $s_\pm$-wave symmetry, which changes sign across the Fermi pockets. We find $T_c$ in La$_4$Ni$_3$O$_{10}$ is systematically lower than that in La$_3$Ni$_2$O$_7$, and electron doping can enhance $T_c$., Comment: 4 pages, 4 figures, 1 table

Large-vocabulary object detectors (LVDs) aim to detect objects of many categories, which learn super objectness features and can locate objects accurately while applied to various downstream data. However, LVDs often struggle in recognizing the located objects due to domain discrepancy in data distribution and object vocabulary. At the other end, recent vision-language foundation models such as CLIP demonstrate superior open-vocabulary recognition capability. This paper presents KGD, a Knowledge Graph Distillation technique that exploits the implicit knowledge graphs (KG) in CLIP for effectively adapting LVDs to various downstream domains. KGD consists of two consecutive stages: 1) KG extraction that employs CLIP to encode downstream domain data as nodes and their feature distances as edges, constructing KG that inherits the rich semantic relations in CLIP explicitly; and 2) KG encapsulation that transfers the extracted KG into LVDs to enable accurate cross-domain object classification. In addition, KGD can extract both visual and textual KG independently, providing complementary vision and language knowledge for object localization and object classification in detection tasks over various downstream domains. Experiments over multiple widely adopted detection benchmarks show that KGD outperforms the state-of-the-art consistently by large margins.

Camera-only Bird's Eye View (BEV) has demonstrated great potential in environment perception in a 3D space. However, most existing studies were conducted under a supervised setup which cannot scale well while handling various new data. Unsupervised domain adaptive BEV, which effective learning from various unlabelled target data, is far under-explored. In this work, we design DA-BEV, the first domain adaptive camera-only BEV framework that addresses domain adaptive BEV challenges by exploiting the complementary nature of image-view features and BEV features. DA-BEV introduces the idea of query into the domain adaptation framework to derive useful information from image-view and BEV features. It consists of two query-based designs, namely, query-based adversarial learning (QAL) and query-based self-training (QST), which exploits image-view features or BEV features to regularize the adaptation of the other. Extensive experiments show that DA-BEV achieves superior domain adaptive BEV perception performance consistently across multiple datasets and tasks such as 3D object detection and 3D scene segmentation.

Segment Anything Models (SAMs) like SEEM and SAM have demonstrated great potential in learning to segment anything. The core design of SAMs lies with Promptable Segmentation, which takes a handcrafted prompt as input and returns the expected segmentation mask. SAMs work with two types of prompts including spatial prompts (e.g., points) and semantic prompts (e.g., texts), which work together to prompt SAMs to segment anything on downstream datasets. Despite the important role of prompts, how to acquire suitable prompts for SAMs is largely under-explored. In this work, we examine the architecture of SAMs and identify two challenges for learning effective prompts for SAMs. To this end, we propose spatial-semantic prompt learning (SSPrompt) that learns effective semantic and spatial prompts for better SAMs. Specifically, SSPrompt introduces spatial prompt learning and semantic prompt learning, which optimize spatial prompts and semantic prompts directly over the embedding space and selectively leverage the knowledge encoded in pre-trained prompt encoders. Extensive experiments show that SSPrompt achieves superior image segmentation performance consistently across multiple widely adopted datasets.

Accurate cloud recognition and warning are crucial for various applications, including in-flight support, weather forecasting, and climate research. However, recent deep learning algorithms have predominantly focused on detecting cloud regions in satellite imagery, with insufficient attention to the specificity required for accurate cloud recognition. This limitation inspired us to develop the novel FY-4A-Himawari-8 (FYH) dataset, which includes nine distinct cloud categories and uses precise domain adaptation methods to align 70,419 image-label pairs in terms of projection, temporal resolution, and spatial resolution, thereby facilitating the training of supervised deep learning networks. Given the complexity and diversity of cloud formations, we have thoroughly analyzed the challenges inherent to cloud recognition tasks, examining the intricate characteristics and distribution of the data. To effectively address these challenges, we designed a Distribution-aware Interactive-Attention Network (DIAnet), which preserves pixel-level details through a high-resolution branch and a parallel multi-resolution cross-branch. We also integrated a distribution-aware loss (DAL) to mitigate the imbalance across cloud categories. An Interactive Attention Module (IAM) further enhances the robustness of feature extraction combined with spatial and channel information. Empirical evaluations on the FYH dataset demonstrate that our method outperforms other cloud recognition networks, achieving superior performance in terms of mean Intersection over Union (mIoU). The code for implementing DIAnet is available at https://github.com/icey-zhang/DIAnet.