Search

State space models (SSMs) with selection mechanisms and hardware-aware architectures, namely Mamba, have recently demonstrated significant promise in long-sequence modeling. Since the self-attention mechanism in transformers has quadratic complexity with image size and increasing computational demands, the researchers are now exploring how to adapt Mamba for computer vision tasks. This paper is the first comprehensive survey aiming to provide an in-depth analysis of Mamba models in the field of computer vision. It begins by exploring the foundational concepts contributing to Mamba's success, including the state space model framework, selection mechanisms, and hardware-aware design. Next, we review these vision mamba models by categorizing them into foundational ones and enhancing them with techniques such as convolution, recurrence, and attention to improve their sophistication. We further delve into the widespread applications of Mamba in vision tasks, which include their use as a backbone in various levels of vision processing. This encompasses general visual tasks, Medical visual tasks (e.g., 2D / 3D segmentation, classification, and image registration, etc.), and Remote Sensing visual tasks. We specially introduce general visual tasks from two levels: High/Mid-level vision (e.g., Object detection, Segmentation, Video classification, etc.) and Low-level vision (e.g., Image super-resolution, Image restoration, Visual generation, etc.). We hope this endeavor will spark additional interest within the community to address current challenges and further apply Mamba models in computer vision.

The escalating influx of data generated by networked edge devices, coupled with the growing awareness of data privacy, has promoted a transformative shift in computing paradigms from centralized data processing to privacy-preserved distributed data processing. Federated analytics (FA) is an emerging technique to support collaborative data analytics among diverse data owners without centralizing the raw data. Despite the wide applications of FA in industry and academia, a comprehensive examination of existing research efforts in FA has been notably absent. This survey aims to bridge this gap by first providing an overview of FA, elucidating key concepts, and discussing its relationship with similar concepts. We then conduct a thorough examination of FA, including its taxonomy, key challenges, and enabling techniques. Diverse FA applications, including statistical metrics, set computation, frequency-related applications, database query operations, model-based applications, FL-assisting FA tasks, and other wireless network applications are then carefully reviewed. We complete the survey with several open research issues and future directions. This survey intends to provide a holistic understanding of the emerging FA techniques and foster the continued evolution of privacy-preserving distributed data processing in the emerging networked society., Comment: This survey has been submitted to IEEE Communications Surveys & Tutorials

This paper revisits few-shot 3D point cloud semantic segmentation (FS-PCS), with a focus on two significant issues in the state-of-the-art: foreground leakage and sparse point distribution. The former arises from non-uniform point sampling, allowing models to distinguish the density disparities between foreground and background for easier segmentation. The latter results from sampling only 2,048 points, limiting semantic information and deviating from the real-world practice. To address these issues, we introduce a standardized FS-PCS setting, upon which a new benchmark is built. Moreover, we propose a novel FS-PCS model. While previous methods are based on feature optimization by mainly refining support features to enhance prototypes, our method is based on correlation optimization, referred to as Correlation Optimization Segmentation (COSeg). Specifically, we compute Class-specific Multi-prototypical Correlation (CMC) for each query point, representing its correlations to category prototypes. Then, we propose the Hyper Correlation Augmentation (HCA) module to enhance CMC. Furthermore, tackling the inherent property of few-shot training to incur base susceptibility for models, we propose to learn non-parametric prototypes for the base classes during training. The learned base prototypes are used to calibrate correlations for the background class through a Base Prototypes Calibration (BPC) module. Experiments on popular datasets demonstrate the superiority of COSeg over existing methods. The code is available at: https://github.com/ZhaochongAn/COSeg, Comment: Accepted to CVPR 2024

First-principles calculations have been used to investigate the electronic and topological properties of the two-dimensional C6-2x(BN)x biphenylene network, a graphene-like structure composed of not only hexagonal ring but also octagonal and square rings. Nontrivial topological properties have been found in two of them, with a stoichiometry of C4BN and C2(BN)2. The former C4BN is predicted to be a type-II Dirac semimetal with a superconducting critical temperature Tc=0.38K, which is similar to the pure carbon biphenylene network (C-BPN). The latter shows a novel isolated edge state exists between the conduction and valence bands. By regulation of strains and virtual-crystal approximation calculations, we found the annihilation of two pairs of Dirac points (DPs) in the non-high symmetric region (non-HSR) causes the two corresponding edge states stick together to generate this isolated edge state. In addition, we found that one pair of DPs arises from the shift of DPs in the C-BPN, while another new pair of DPs emerges around the Time Reversal Invariant Momenta (TRIM) point X due to the doping of boron and nitrogen. We constructed a tight-binding (TB) model to reveal the mechanism of forming the isolated edge state from the C-BPN to C2(BN)2. This study not only demonstrates the existence and mechanism of forming the isolated edge state in semimetals, but also provides an example in which the DPs can move away from the high-symmetry region., Comment: 32 pages, 10 figures, with supplementary materials

Robust metallic surface states (MSS) of topological insulator (TI) against imperfections and perturbations are important in broad applications such as chemical catalysis and quantum computing. Unfortunately, they are suffered from the narrow band gap that can be accessed. Searching for MSS with large bulk band gap beyond conventional TIs becomes a quest. In this work, inspired by the adiabatic connection principle in real space, we identify that all electrides, a new class of emerging materials, must host robust and universal MSS that resists any disturbances, in spite of the fact that some of them could be classified as trivial in standard topology theory. This counterintuitive property is traced to the specific charge localization-delocalization change intrinsic to electride when approaching the crystalline surface or interface, which is a kind of interstice-centered to atom-centered transition in the real-space topology of the charge density distribution, and is sharply different from the band inversion in the standard topology theory. The new mechanism circumvents the obstacle that limits the band gap of TI. Robust and universal MSS in an electride that conventionally-determined as trivial but with a colossal band gap beyond 6.13 eV are demonstrated. This gap size is about 6-fold larger than the highest record of known "wide-gap" TIs, thus opens up new avenues to universal MSS with gigantic bulk gap., Comment: 32 pages, 7 figures, with supporting information

The emerging Web 3.0 paradigm aims to decentralize existing web services, enabling desirable properties such as transparency, incentives, and privacy preservation. However, current Web 3.0 applications supported by blockchain infrastructure still cannot support complex data analytics tasks in a scalable and privacy-preserving way. This paper introduces the emerging federated analytics (FA) paradigm into the realm of Web 3.0 services, enabling data to stay local while still contributing to complex web analytics tasks in a privacy-preserving way. We propose FedWeb, a tailored FA design for important frequent pattern mining tasks in Web 3.0. FedWeb remarkably reduces the number of required participating data owners to support privacy-preserving Web 3.0 data analytics based on a novel distributed differential privacy technique. The correctness of mining results is guaranteed by a theoretically rigid candidate filtering scheme based on Hoeffding's inequality and Chebychev's inequality. Two response budget saving solutions are proposed to further reduce participating data owners. Experiments on three representative Web 3.0 scenarios show that FedWeb can improve data utility by ~25.3% and reduce the participating data owners by ~98.4%., Comment: Accepted by IEEE International Conference on Computer Communications (INFOCOM'24)

Embodied carbon is the total carbon released from the processes associated with a product from cradle to gate. In many industry sectors, embodied carbon dominates the overall carbon footprint. Embodied carbon accounting, i.e., to estimate the embodied carbon of a product, has become an important research topic. Existing studies derive the embodied carbon through life cycle analysis (LCA) reports. Current LCA reports only provide the carbon emission of a product class, e.g., 28nm CPU, yet a product instance can be manufactured from diverse regions and in diverse time periods, e.g., a winter period of Ireland (Intel). It is known that the carbon emission depends on the electricity generation process which has spatial and temporal dynamics. Therefore, the embodied carbon of a specific product instance can largely differ from its product class. In this paper, we present new spatial-temporal embodied carbon models for embodied carbon accounting. We observe significant differences between current embodied carbon models and our spatial-temporal embodied carbon models, e.g., for 7nm CPU the difference can be 13.69%.

Optimization via simulation has been well established to find optimal solutions and designs in complex systems. However, it still faces modeling and computational challenges when extended to the multi-stage setting. This survey reviews the models and methodologies of single-stage optimization via simulation and multi-stage stochastic programming. These are necessary theoretical components to push forward the development of sequential decision making via simulation. We identify the key challenge of sequential decision making via simulation as the appropriate modeling of the stage-wise value function, for which we survey the state-of-the-art meta-models and their potential solution algorithms.

Difference features obtained by comparing the images of two periods play an indispensable role in the change detection (CD) task. However, a pair of bi-temporal images can exhibit diverse changes, which may cause various difference features. Identifying changed pixels with differ difference features to be the same category is thus a challenge for CD. Most nowadays' methods acquire distinctive difference features in implicit ways like enhancing image representation or supervision information. Nevertheless, informative image features only guarantee object semantics are modeled and can not guarantee that changed pixels have similar semantics in the difference feature space and are distinct from those unchanged ones. In this work, the generalized representation of various changes is learned straightforwardly in the difference feature space, and a novel Changes-Aware Transformer (CAT) for refining difference features is proposed. This generalized representation can perceive which pixels are changed and which are unchanged and further guide the update of pixels' difference features. CAT effectively accomplishes this refinement process through the stacked cosine cross-attention layer and self-attention layer. After refinement, the changed pixels in the difference feature space are closer to each other, which facilitates change detection. In addition, CAT is compatible with various backbone networks and existing CD methods. Experiments on remote sensing CD data set and street scene CD data set show that our method achieves state-of-the-art performance and has excellent generalization.

This paper proposes decentralized stability conditions for multi-converter systems based on the combination of the small gain theorem and the small phase theorem. Instead of directly computing the closed-loop dynamics, e.g., eigenvalues of the state-space matrix, or using the generalized Nyquist stability criterion, the proposed stability conditions are more scalable and computationally lighter, which aim at evaluating the closed-loop system stability by comparing the individual converter dynamics with the network dynamics in a decentralized and open-loop manner. Moreover, our approach can handle heterogeneous converters' dynamics and is suitable to analyze large-scale multi-converter power systems that contain grid-following (GFL), grid-forming (GFM) converters, and synchronous generators. Compared with other decentralized stability conditions, e.g., passivity-based stability conditions, the proposed conditions are significantly less conservative and can be generally satisfied in practice across the whole frequency range.

Multifocus image fusion is an effective way to overcome the limitation of optical lenses. Many existing methods obtain fused results by generating decision maps. However, such methods often assume that the focused areas of the two source images are complementary, making it impossible to achieve simultaneous fusion of multiple images. Additionally, the existing methods ignore the impact of hard pixels on fusion performance, limiting the visual quality improvement of fusion image. To address these issues, a combining generation and recombination model, termed as GRFusion, is proposed. In GRFusion, focus property detection of each source image can be implemented independently, enabling simultaneous fusion of multiple source images and avoiding information loss caused by alternating fusion. This makes GRFusion free from the number of inputs. To distinguish the hard pixels from the source images, we achieve the determination of hard pixels by considering the inconsistency among the detection results of focus areas in source images. Furthermore, a multi-directional gradient embedding method for generating full focus images is proposed. Subsequently, a hard-pixel-guided recombination mechanism for constructing fused result is devised, effectively integrating the complementary advantages of feature reconstruction-based method and focused pixel recombination-based method. Extensive experimental results demonstrate the effectiveness and the superiority of the proposed method.The source code will be released on https://github.com/xxx/xxx.

With the emerging diffusion models, recently, text-to-video generation has aroused increasing attention. But an important bottleneck therein is that generative videos often tend to carry some flickers and artifacts. In this work, we propose a dual-stream diffusion net (DSDN) to improve the consistency of content variations in generating videos. In particular, the designed two diffusion streams, video content and motion branches, could not only run separately in their private spaces for producing personalized video variations as well as content, but also be well-aligned between the content and motion domains through leveraging our designed cross-transformer interaction module, which would benefit the smoothness of generated videos. Besides, we also introduce motion decomposer and combiner to faciliate the operation on video motion. Qualitative and quantitative experiments demonstrate that our method could produce amazing continuous videos with fewer flickers., Comment: 8pages, 7 figures

A compact K\"ahler manifold $\left( M,\omega ,J\right) $ with $T$-symmetry admits a natural mixed polarization $\mathcal{P}_{\mathrm{mix}}$ whose real directions come from the $T$-action. In \cite{LW1}, we constructed a one-parameter family of K\"ahler structures $\left( \omega ,J_{t}\right) $'s with the same underlying K\"a hler form $\omega $ and $J_{0}=J$, such that (i) there is a $T$-equivariant biholomorphism between $\left( M,J_{0}\right) $ and $\left( M,J_{t}\right) $ and (ii) K\"ahler polarizations $\mathcal{P} _{t}$'s corresponding to $J_{t}$'s converge to $\mathcal{P}_{\mathrm{mix}}$ as $t$ goes to infinity. In this paper, we study the quantum analog of above results. Assume $L$ is a pre-quantum line bundle on $\left( M,\omega \right) $. Let $\mathcal{H}_{t}$ and $ \mathcal{H}_{\mathrm{mix}}$ be quantum spaces defined using polarizations $\mathcal{P}_{t}$ and $\mathcal{P}_{\mathrm{mix}}$ respectively. In particular, $\mathcal{H}_{t}=H_{\bar{\partial}_{t}}^{0}\left( M,L\right) $. They are both representations of $T$. We show that (i) there is a $T$-equivariant isomorphism between $\mathcal{H}_{0}$ and $\mathcal{H}_{\mathrm{mix}}$ and (ii) for regular $T$-weight $\lambda $, corresponding $\lambda $-weight spaces $ \mathcal{H}_{t,\lambda }$'s converge to $\mathcal{H}_{\mathrm{mix},\lambda }$ as $t$ goes to infinity., Comment: 24pages. Comments are welcome!