Search

Due to the scarcity of labeled samples in Image Quality Assessment (IQA) datasets, numerous recent studies have proposed multi-task based strategies, which explore feature information from other tasks or domains to boost the IQA task. Nevertheless, multi-task strategies based No-Reference Image Quality Assessment (NR-IQA) methods encounter several challenges. First, existing methods have not explicitly exploited texture details, which significantly influence the image quality. Second, multi-task methods conventionally integrate features through simple operations such as addition or concatenation, thereby diminishing the network's capacity to accurately represent distorted features. To tackle these challenges, we introduce a novel multi-task NR-IQA framework. Our framework consists of three key components: a high-frequency extraction network, a quality estimation network, and a distortion-aware network. The high-frequency extraction network is designed to guide the model's focus towards high-frequency information, which is highly related to the texture details. Meanwhile, the distortion-aware network extracts distortion-related features to distinguish different distortion types. To effectively integrate features from different tasks, a feature fusion module is developed based on an attention mechanism. Empirical results from five standard IQA databases confirm that our method not only achieves high performance but also exhibits robust generalization ability.

Neural Representations for Videos (NeRV) have simplified the video codec process and achieved swift decoding speeds by encoding video content into a neural network, presenting a promising solution for video compression. However, existing work overlooks the crucial issue that videos reconstructed by these methods lack high-frequency details. To address this problem, this paper introduces a High-Frequency Enhanced Hybrid Neural Representation Network. Our method focuses on leveraging high-frequency information to improve the synthesis of fine details by the network. Specifically, we design a wavelet high-frequency encoder that incorporates Wavelet Frequency Decomposer (WFD) blocks to generate high-frequency feature embeddings. Next, we design the High-Frequency Feature Modulation (HFM) block, which leverages the extracted high-frequency embeddings to enhance the fitting process of the decoder. Finally, with the refined Harmonic decoder block and a Dynamic Weighted Frequency Loss, we further reduce the potential loss of high-frequency information. Experiments on the Bunny and UVG datasets demonstrate that our method outperforms other methods, showing notable improvements in detail preservation and compression performance.

This paper studies a distributed estimation problem for time-varying/time-invariant large-scale interconnected systems (LISs). A fully distributed estimator is presented by recursively solving a distributed modified Riccati equation (DMRE) with decoupling variables. By partitioning the LIS based on the transition matrix's block structure, it turns out that the stability of the subsystem is independent of the global LIS if the decoupling variable is selected as the number of out-neighbors. Additionally, it is revealed that any LIS can be equivalently represented by a Markov system. Based on this insight, we show that the stability decoupling above can also be achieved if the decoupling variable equals the number of in-neighbors. Then, the distributed estimator is proved to be stable if the DMRE remains uniformly bounded. When the LIS is considered time-invariant, and by analyzing the spectral radius of a linear operator, it is proved that the DMRE is uniformly bounded if and only if a linear matrix inequality is feasible. Based on the boundedness result, we also show that the distributed estimator converges to a unique steady state for any initial condition. Finally, simulations verify the effectiveness of the proposed methods., Comment: 15 pages, 4 figures

6G is envisaged to provide multimodal sensing, pervasive intelligence, global coverage, global coverage, etc., which poses extreme intricacy and new challenges to the network design and optimization. As the core part of 6G, wireless channel is the carrier and enabler for the flourishing technologies and novel services, which intrinsically determines the ultimate system performance. However, how to describe and utilize the complicated and high-dynamic characteristics of wireless channel accurately and effectively still remains great hallenges. To tackle this, digital twin is envisioned as a powerful technology to migrate the physical entities to virtual and computational world. In this article, we propose a large model driven digital twin channel generator (ChannelGPT) embedded with environment intelligence (EI) to enable pervasive intelligence paradigm for 6G network. EI is an iterative and interactive procedure to boost the system performance with online environment adaptivity. Firstly, ChannelGPT is capable of utilization the multimodal data from wireless channel and corresponding physical environment with the equipped sensing ability. Then, based on the fine-tuned large model, ChannelGPT can generate multi-scenario channel parameters, associated map information and wireless knowledge simultaneously, in terms of each task requirement. Furthermore, with the support of online multidimensional channel and environment information, the network entity will make accurate and immediate decisions for each 6G system layer. In practice, we also establish a ChannelGPT prototype to generate high-fidelity channel data for varied scenarios to validate the accuracy and generalization ability based on environment intelligence.

In September 2024, Beijing University of Posts and Telecommunications and China Mobile Communications Group jointly releases a channel dataset for the sixth generation (6G) mobile communications, named BUPTCMCC-6G-DataAI+. BUPTCMCC-6G-DataAI+ is the update version of BUPTCMCC-6G-DataAI, which is already published in June 2023, aiming at extending 6G new technologies, frequency bands, and applications. BUPTCMCC-6G-DataAI+ provides deterministic data covering new mid-bands, millimeter wave (mmWave), and terahertz (THz), supports the features of XL-MIMO near-field, high mobility and provides multiple 6G scenarios such as reconfigurable intelligent surface (RIS) and industrial Internet. Configured with customized features according to different user needs, BUPTCMCC-6G-DataAI+ can adaptively generate scalable large-scale or small-scale parameters, providing data support for 6G research and development, and standardization., Comment: 2 pages, 1 figure

The air interface technology plays a crucial role in optimizing the communication quality for users. To address the challenges brought by the radio channel variations to air interface design, this article proposes a framework of wireless environment information-aided 6G AI-enabled air interface (WEI-6G AI$^{2}$), which actively acquires real-time environment details to facilitate channel fading prediction and communication technology optimization. Specifically, we first outline the role of WEI in supporting the 6G AI$^{2}$ in scenario adaptability, real-time inference, and proactive action. Then, WEI is delineated into four progressive steps: raw sensing data, features obtained by data dimensionality reduction, semantics tailored to tasks, and knowledge that quantifies the environmental impact on the channel. To validate the availability and compare the effect of different types of WEI, a path loss prediction use case is designed. The results demonstrate that leveraging environment knowledge requires only 2.2 ms of model inference time, which can effectively support real-time design for future 6G AI$^{2}$. Additionally, WEI can reduce the pilot overhead by 25\%. Finally, several open issues are pointed out, including multi-modal sensing data synchronization and information extraction method construction.

Recent progress of semantic point clouds analysis is largely driven by synthetic data (e.g., the ModelNet and the ShapeNet), which are typically complete, well-aligned and noisy free. Therefore, representations of those ideal synthetic point clouds have limited variations in the geometric perspective and can gain good performance on a number of 3D vision tasks such as point cloud classification. In the context of unsupervised domain adaptation (UDA), representation learning designed for synthetic point clouds can hardly capture domain invariant geometric patterns from incomplete and noisy point clouds. To address such a problem, we introduce a novel scheme for induced geometric invariance of point cloud representations across domains, via regularizing representation learning with two self-supervised geometric augmentation tasks. On one hand, a novel pretext task of predicting translation distances of augmented samples is proposed to alleviate centroid shift of point clouds due to occlusion and noises. On the other hand, we pioneer an integration of the relational self-supervised learning on geometrically-augmented point clouds in a cascade manner, utilizing the intrinsic relationship of augmented variants and other samples as extra constraints of cross-domain geometric features. Experiments on the PointDA-10 dataset demonstrate the effectiveness of the proposed method, achieving the state-of-the-art performance., Comment: 10 pages, 6 figures, 5 tables

The detailed study of the strong gravitational lensing of a Kerr black hole within Quantum Einstein Gravity (QEG) is performed. We calculate the photon sphere, the deflection angle of light, and observables on the equatorial plane under the strong deflection limit in a vacuum. The presence of quantum effects reduces the radius of the photon sphere, the magnification, the position of relativistic images, and the time delays on the same side of the lens. However, it increases the strong deflection angle, the separations, and the time delays on the opposite side of the lens. By modeling M87* and Sgr A* as the Kerr black hole within QEG, we find that the time delays are more significant in M87*, while other observables are more pronounced in Sgr A*. Furthermore, we consider the influence of plasma on the gravitational lensing effect. Plasma causes an additional deflection of light, increasing the magnification, images position and the time delays, but decreasing the separations. More importantly, we calculate the time delays under the strong deflection limit in the presence of plasma, and they increase with higher plasma concentrations. Our research may help to evaluate the observational imprints left by such quantum effects in the propagation of light and the impact of plasma around black holes on gravitational lensing., Comment: 15 pages, 15 figures

State Space Models (SSMs), especially Mamba, have shown great promise in medical image segmentation due to their ability to model long-range dependencies with linear computational complexity. However, accurate medical image segmentation requires the effective learning of both multi-scale detailed feature representations and global contextual dependencies. Although existing works have attempted to address this issue by integrating CNNs and SSMs to leverage their respective strengths, they have not designed specialized modules to effectively capture multi-scale feature representations, nor have they adequately addressed the directional sensitivity problem when applying Mamba to 2D image data. To overcome these limitations, we propose a Multi-Scale Vision Mamba UNet model for medical image segmentation, termed MSVM-UNet. Specifically, by introducing multi-scale convolutions in the VSS blocks, we can more effectively capture and aggregate multi-scale feature representations from the hierarchical features of the VMamba encoder and better handle 2D visual data. Additionally, the large kernel patch expanding (LKPE) layers achieve more efficient upsampling of feature maps by simultaneously integrating spatial and channel information. Extensive experiments on the Synapse and ACDC datasets demonstrate that our approach is more effective than some state-of-the-art methods in capturing and aggregating multi-scale feature representations and modeling long-range dependencies between pixels., Comment: 8 pages, 5 figures

Channel state information (CSI) is crucial for massive multi-input multi-output (MIMO) system. As the antenna scale increases, acquiring CSI results in significantly higher system overhead. In this letter, we propose a novel channel prediction method which utilizes wireless environmental information with pilot pattern optimization for CSI prediction (WEI-CSIP). Specifically, scatterers around the mobile station (MS) are abstracted from environmental information using multiview images. Then, an environmental feature map is extracted by a convolutional neural network (CNN). Additionally, the deep probabilistic subsampling (DPS) network acquires an optimal fixed pilot pattern. Finally, a CNN-based channel prediction network is designed to predict the complete CSI, using the environmental feature map and partial CSI. Simulation results show that the WEI-CSIP can reduce pilot overhead from 1/5 to 1/8, while improving prediction accuracy with normalized mean squared error reduced to 0.0113, an improvement of 83.2% compared to traditional channel prediction methods.

We determine which simplicial complexes have the maximum or minimum sum of Betti numbers and sum of bigraded Betti numbers with a given number of vertices in each dimension., Comment: 22 pages, 1 figure

Incomplete multi-modal image segmentation is a fundamental task in medical imaging to refine deployment efficiency when only partial modalities are available. However, the common practice that complete-modality data is visible during model training is far from realistic, as modalities can have imbalanced missing rates in clinical scenarios. In this paper, we, for the first time, formulate such a challenging setting and propose Preference-Aware Self-diStillatION (PASSION) for incomplete multi-modal medical image segmentation under imbalanced missing rates. Specifically, we first construct pixel-wise and semantic-wise self-distillation to balance the optimization objective of each modality. Then, we define relative preference to evaluate the dominance of each modality during training, based on which to design task-wise and gradient-wise regularization to balance the convergence rates of different modalities. Experimental results on two publicly available multi-modal datasets demonstrate the superiority of PASSION against existing approaches for modality balancing. More importantly, PASSION is validated to work as a plug-and-play module for consistent performance improvement across different backbones. Code is available at https://github.com/Jun-Jie-Shi/PASSION., Comment: Accepted by ACM MM 2024

Federated learning has emerged as a compelling paradigm for medical image segmentation, particularly in light of increasing privacy concerns. However, most of the existing research relies on relatively stringent assumptions regarding the uniformity and completeness of annotations across clients. Contrary to this, this paper highlights a prevalent challenge in medical practice: incomplete annotations. Such annotations can introduce incorrectly labeled pixels, potentially undermining the performance of neural networks in supervised learning. To tackle this issue, we introduce a novel solution, named FedIA. Our insight is to conceptualize incomplete annotations as noisy data (i.e., low-quality data), with a focus on mitigating their adverse effects. We begin by evaluating the completeness of annotations at the client level using a designed indicator. Subsequently, we enhance the influence of clients with more comprehensive annotations and implement corrections for incomplete ones, thereby ensuring that models are trained on accurate data. Our method's effectiveness is validated through its superior performance on two extensively used medical image segmentation datasets, outperforming existing solutions. The code is available at https://github.com/HUSTxyy/FedIA., Comment: Early accepted by MICCAI 2024

Cross-silo federated learning (FL) enables decentralized organizations to collaboratively train models while preserving data privacy and has made significant progress in medical image classification. One common assumption is task homogeneity where each client has access to all classes during training. However, in clinical practice, given a multi-label classification task, constrained by the level of medical knowledge and the prevalence of diseases, each institution may diagnose only partial categories, resulting in task heterogeneity. How to pursue effective multi-label medical image classification under task heterogeneity is under-explored. In this paper, we first formulate such a realistic label missing setting in the multi-label FL domain and propose a two-stage method FedMLP to combat class missing from two aspects: pseudo label tagging and global knowledge learning. The former utilizes a warmed-up model to generate class prototypes and select samples with high confidence to supplement missing labels, while the latter uses a global model as a teacher for consistency regularization to prevent forgetting missing class knowledge. Experiments on two publicly-available medical datasets validate the superiority of FedMLP against the state-of-the-art both federated semi-supervised and noisy label learning approaches under task heterogeneity. Code is available at https://github.com/szbonaldo/FedMLP., Comment: Early accepted by MICCAI 2024

In this paper, an anti-eavesdropping estimation problem is investigated. A linear encryption scheme is utilized, which first linearly transforms innovation via an encryption matrix and then encrypts some components of the transformed innovation. To reduce the computation and energy resources consumed by the linear encryption scheme, both stochastic and deterministic intermittent strategies which perform the linear encryption scheme only at partial moments are developed. When the system is stable, it is shown that the mean squared error (MSE) of the eavesdropper converges under any stochastic or deterministic intermittent strategy. Also, an analytical encryption matrix that maximizes the steady-state of the MSE is designed. When the system is unstable, the eavesdropper's MSE can be unbounded with arbitrary positive encryption probabilities and decision functions if encryption matrices are chosen appropriately. Then, the relationship between the aforementioned encryption parameters and the eavesdropper's MSE is analyzed. Moreover, a single intermittent strategy which only encrypts one message is discussed. This strategy can be unavailable for stable systems, but can make the eavesdropper's MSE unbounded in unstable systems for the encrypted message satisfies a linear matrix inequality (LMI) condition. The effectiveness of the proposed methods is verified in the simulation., Comment: 12 pages, 5 figures

We model the TESS light curve of the binary system RX Dra, and also first calculate a lot of theoretical models to fit the g-mode frequencies previously detected from the TESS series of this system. The mass ratio is determined to be $q$=0.9026$^{+0.0032}_{-0.0032}$. We newly found that there are 16 frequencies (F1-F7, F11-F20) identified as dipole g-modes, two frequencies (F21, F22) identified as quadrupole g-modes, and another two frequencies (F23, F24) identified as g-sextupole modes, based on these model fits. The primary star is newly determined to be a $\gamma$ Dor pulsator in the main-sequence stage with a rotation period of about 5.7$^{+0.7}_{-0.2}$ days, rotating slower than the orbital motion. The fundamental parameters of two components are firstly estimated as follows: $M_1$=1.53$^{+0.00}_{-0.17}$ M $_{\odot}$, $M_2$= 1.38$^{+0.18}_{-0.00}$ M $_{\odot}$, $T_1$=7240$^{+490}_{-44}$ K, $T_2$=6747$^{+201}_{-221}$ K, $R_1$=1.8288$^{+0.0260}_{-0.0959}$ R $_{\odot}$, $R_2$= 1.3075$^{+0.0450}_{-0.2543}$ R $_{\odot}$, $L_1$=8.2830$^{+1.8015}_{-0.6036}$ L $_{\odot}$ and $L_2$= 3.4145$^{+0.1320}_{-0.1843}$ L $_{\odot}$. Our results show that the secondary star is in the solar-like pulsator region of the H-R diagram, indicating that it could be a pulsating star of this type. Finally, the radius of the convective core of the primary star is estimated to be about 0.1403$^{+0.0206}_{-0.0000}$ R$_{\odot}$., Comment: 13 pages,7 figures

With the development of sixth generation (6G) networks toward digitalization and intelligentization of communications, rapid and precise channel prediction is crucial for the network potential release. Interestingly, a dynamic ray tracing (DRT) approach for channel prediction has recently been proposed, which utilizes the results of traditional RT to extrapolate the multipath geometry evolution. However, both the priori environmental data and the regularity in multipath evolution can be further utilized. In this work, an enhanced-dynamic ray tracing (E-DRT) algorithm architecture based on multipath bidirectional extrapolation has been proposed. In terms of accuracy, all available environment information is utilized to predict the birth and death processes of multipath components (MPCs) through bidirectional geometry extrapolation. In terms of efficiency, bidirectional electric field extrapolation is employed based on the evolution regularity of the MPCs' electric field. The results in a Vehicle-to-Vehicle (V2V) scenario show that E-DRT improves the accuracy of the channel prediction from 68.3% to 94.8% while reducing the runtime by 7.2% compared to DRT.

Reconstruction-based methods have been commonly used for unsupervised anomaly detection, in which a normal image is reconstructed and compared with the given test image to detect and locate anomalies. Recently, diffusion models have shown promising applications for anomaly detection due to their powerful generative ability. However, these models lack strict mathematical support for normal image reconstruction and unexpectedly suffer from low reconstruction quality. To address these issues, this paper proposes a novel and highly-interpretable method named Masked Diffusion Posterior Sampling (MDPS). In MDPS, the problem of normal image reconstruction is mathematically modeled as multiple diffusion posterior sampling for normal images based on the devised masked noisy observation model and the diffusion-based normal image prior under Bayesian framework. Using a metric designed from pixel-level and perceptual-level perspectives, MDPS can effectively compute the difference map between each normal posterior sample and the given test image. Anomaly scores are obtained by averaging all difference maps for multiple posterior samples. Exhaustive experiments on MVTec and BTAD datasets demonstrate that MDPS can achieve state-of-the-art performance in normal image reconstruction quality as well as anomaly detection and localization.

Due to escalating privacy concerns, federated learning has been recognized as a vital approach for training deep neural networks with decentralized medical data. In practice, it is challenging to ensure consistent imaging quality across various institutions, often attributed to equipment malfunctions affecting a minority of clients. This imbalance in image quality can cause the federated model to develop an inherent bias towards higher-quality images, thus posing a severe fairness issue. In this study, we pioneer the identification and formulation of this new fairness challenge within the context of the imaging quality shift. Traditional methods for promoting fairness in federated learning predominantly focus on balancing empirical risks across diverse client distributions. This strategy primarily facilitates fair optimization across different training data distributions, yet neglects the crucial aspect of generalization. To address this, we introduce a solution termed Federated learning with Inter-client Sharpness Matching (FedISM). FedISM enhances both local training and global aggregation by incorporating sharpness-awareness, aiming to harmonize the sharpness levels across clients for fair generalization. Our empirical evaluations, conducted using the widely-used ICH and ISIC 2019 datasets, establish FedISM's superiority over current state-of-the-art federated learning methods in promoting fairness. Code is available at https://github.com/wnn2000/FFL4MIA., Comment: This paper is accepted at IJCAI'24 (Main Track)

Designing new molecules is an important task in the field of pharmaceuticals. Due to the vast design space of molecules, generating molecules conditioned on a specific sub-structure relevant to a particular function or therapeutic target is a crucial task in computer-aided drug design. In this paper, we present ControlMol, which adds sub-structure control to molecule generation with diffusion models. Unlike previous methods which view this task as inpainting or conditional generation, we adopt the idea of ControlNet into conditional molecule generation and make adaptive adjustments to a pre-trained diffusion model. We apply our method to both 2D and 3D molecule generation tasks. Conditioned on randomly partitioned sub-structure data, our method outperforms previous methods by generating more valid and diverse molecules. The method is easy to implement and can be quickly applied to a variety of pre-trained molecule generation models., Comment: 9 pages,7 figures

Despite considerable progress being achieved in point cloud geometry compression, there still remains a challenge in effectively compressing large-scale scenes with sparse surfaces. Another key challenge lies in reducing decoding latency, a crucial requirement in real-world application. In this paper, we propose Pointsoup, an efficient learning-based geometry codec that attains high-performance and extremely low-decoding-latency simultaneously. Inspired by conventional Trisoup codec, a point model-based strategy is devised to characterize local surfaces. Specifically, skin features are embedded from local windows via an attention-based encoder, and dilated windows are introduced as cross-scale priors to infer the distribution of quantized features in parallel. During decoding, features undergo fast refinement, followed by a folding-based point generator that reconstructs point coordinates with fairly fast speed. Experiments show that Pointsoup achieves state-of-the-art performance on multiple benchmarks with significantly lower decoding complexity, i.e., up to 90$\sim$160$\times$ faster than the G-PCCv23 Trisoup decoder on a comparatively low-end platform (e.g., one RTX 2080Ti). Furthermore, it offers variable-rate control with a single neural model (2.9MB), which is attractive for industrial practitioners.

Label-free imaging through two-photon autofluorescence (2PAF) of NAD(P)H allows for non-destructive and high-resolution visualization of cellular activities in living systems. However, its application to thick tissues and organoids has been restricted by its limited penetration depth within 300 $\mu$m, largely due to tissue scattering at the typical excitation wavelength (~750 nm) required for NAD(P)H. Here, we demonstrate that the imaging depth for NAD(P)H can be extended to over 700 $\mu$m in living engineered human multicellular microtissues by adopting multimode fiber (MMF)-based low-repetition-rate high-peak-power three-photon (3P) excitation of NAD(P)H at 1100 nm. This is achieved by having over 0.5 MW peak power at the band of 1100$\pm$25 nm through adaptively modulating multimodal nonlinear pulse propagation with a compact fiber shaper. Moreover, the 8-fold increase in pulse energy at 1100 nm enables faster imaging of monocyte behaviors in the living multicellular models. These results represent a significant advance for deep and dynamic metabolic and structural imaging of intact living biosystems. The modular design (MMF with a slip-on fiber shaper) is anticipated to allow wide adoption of this methodology for demanding in vivo and in vitro imaging applications, including cancer research, autoimmune diseases, and tissue engineering., Comment: 20 pages, 5 figures, under review in Science Advances

Non-Abelian topological orders offer an intriguing path towards fault-tolerant quantum computation, where information can be encoded and manipulated in a topologically protected manner immune to arbitrary local noises and perturbations. However, realizing non-Abelian topologically ordered states is notoriously challenging in both condensed matter and programmable quantum systems, and it was not until recently that signatures of non-Abelian statistics were observed through digital quantum simulation approaches. Despite these exciting progresses, none of them has demonstrated the appropriate type of topological orders and associated non-Abelian anyons whose braidings alone support universal quantum computation. Here, we report the realization of non-Abelian topologically ordered states of the Fibonacci string-net model and demonstrate braidings of Fibonacci anyons featuring universal computational power, with a superconducting quantum processor. We exploit efficient quantum circuits to prepare the desired states and verify their nontrivial topological nature by measuring the topological entanglement entropy. In addition, we create two pairs of Fibonacci anyons and demonstrate their fusion rule and non-Abelian braiding statistics by applying unitary gates on the underlying physical qubits. Our results establish a versatile digital approach to exploring exotic non-Abelian topological states and their associated braiding statistics with current noisy intermediate-scale quantum processors.

Digital twin channel (DTC) is the real-time mapping of a wireless channel from the physical world to the digital world, which is expected to provide significant performance enhancements for the sixth-generation (6G) air-interface design. In this work, we first define five evolution levels of channel twins with the progression of wireless communication. The fifth level, autonomous DTC, is elaborated with multi-dimensional factors such as methodology, characterization precision, and data category. Then, we provide detailed insights into the requirements and architecture of a complete DTC for 6G. Subsequently, a sensing-enhanced real-time channel prediction platform and experimental validations are exhibited. Finally, drawing from the vision of the 6G network, we explore the potential applications and the open issues in future DTC research., Comment: 7 pages, 5 figures, 15 references. It is submitted to IEEE journal

Segment anything model (SAM), a foundation model with superior versatility and generalization across diverse segmentation tasks, has attracted widespread attention in medical imaging. However, it has been proved that SAM would encounter severe performance degradation due to the lack of medical knowledge in training and local feature encoding. Though several SAM-based models have been proposed for tuning SAM in medical imaging, they still suffer from insufficient feature extraction and highly rely on high-quality prompts. In this paper, we construct a large CT dataset consisting of 1.1M CT images and 5M masks from public datasets and propose a powerful foundation model SAMCT allowing labor-free prompts. Specifically, based on SAM, SAMCT is further equipped with a U-shaped CNN image encoder, a cross-branch interaction module, and a task-indicator prompt encoder. The U-shaped CNN image encoder works in parallel with the ViT image encoder in SAM to supplement local features. Cross-branch interaction enhances the feature expression capability of the CNN image encoder and the ViT image encoder by exchanging global perception and local features from one to the other. The task-indicator prompt encoder is a plug-and-play component to effortlessly encode task-related indicators into prompt embeddings. In this way, SAMCT can work in an automatic manner in addition to the semi-automatic interactive strategy in SAM. Extensive experiments demonstrate the superiority of SAMCT against the state-of-the-art task-specific and SAM-based medical foundation models on various tasks. The code, data, and models are released at https://github.com/xianlin7/SAMCT.

Noisy images are a challenge to image compression algorithms due to the inherent difficulty of compressing noise. As noise cannot easily be discerned from image details, such as high-frequency signals, its presence leads to extra bits needed for compression. Since the emerging learned image compression paradigm enables end-to-end optimization of codecs, recent efforts were made to integrate denoising into the compression model, relying on clean image features to guide denoising. However, these methods exhibit suboptimal performance under high noise levels, lacking the capability to generalize across diverse noise types. In this paper, we propose a novel method integrating a multi-scale denoiser comprising of Self Organizing Operational Neural Networks, for joint image compression and denoising. We employ contrastive learning to boost the network ability to differentiate noise from high frequency signal components, by emphasizing the correlation between noisy and clean counterparts. Experimental results demonstrate the effectiveness of the proposed method both in rate-distortion performance, and codec speed, outperforming the current state-of-the-art., Comment: Copyright 2024 IEEE - Submitted to IEEE ICIP 2024

Deep learning-based methods have demonstrated encouraging results in tackling the task of panoramic image inpainting. However, it is challenging for existing methods to distinguish valid pixels from invalid pixels and find suitable references for corrupted areas, thus leading to artifacts in the inpainted results. In response to these challenges, we propose a panoramic image inpainting framework that consists of a Face Generator, a Cube Generator, a side branch, and two discriminators. We use the Cubemap Projection (CMP) format as network input. The generator employs gated convolutions to distinguish valid pixels from invalid ones, while a side branch is designed utilizing contextual reconstruction (CR) loss to guide the generators to find the most suitable reference patch for inpainting the missing region. The proposed method is compared with state-of-the-art (SOTA) methods on SUN360 Street View dataset in terms of PSNR and SSIM. Experimental results and ablation study demonstrate that the proposed method outperforms SOTA both quantitatively and qualitatively., Comment: Copyright 2024 IEEE - to appear in IEEE ICASSP 2024

In this work, we investigate the problem of simultaneous blind demixing and super-resolution. Leveraging the subspace assumption regarding unknown point spread functions, this problem can be reformulated as a low-rank matrix demixing problem. We propose a convex recovery approach that utilizes the low-rank structure of each vectorized Hankel matrix associated with the target matrix. Our analysis reveals that for achieving exact recovery, the number of samples needs to satisfy the condition $n\gtrsim Ksr \log (sn)$. Empirical evaluations demonstrate the recovery capabilities and the computational efficiency of the convex method.

We study the physics of dilute magnetic impurities in a two-dimensional altermagnetic metal. We first discuss the Kondo problem. Although the spin degeneracy is broken in an altermagnetic metal, we show that the antiferromagnetic Kondo coupling still flows to the strong coupling regime at low energies in terms of the one-loop renormalization-group equation. To study the ground-state properties, we employ the variational wavefunction approach. We find that the impurity spin is completely screened at long distances, in contrast to the usual antiferromagnet. The $d$-wave nature of the Fermi surface in an altermagnetic metal is reflected in the correlation between the impurity and conduction electron spins, which exhibits the $C_{4z}$ symmetry of the altermagnet at long distances. The spin correlation decays as $1/r^3$ at long distances, and its amplitude oscillates with four different periods due to the anisotropic Fermi surfaces. Moreover, the values of these periods depend on the direction of observation. Next, we calculate the RKKY interaction which is of an anisotropic Heisenberg type. This anisotropic RKKY interaction also exhibits the $C_4$ symmetry at long distances due to the $d$-wave nature of the Fermi surface. In addition, the transverse and longitudinal components of the RKKY interactions posses distinct periods of oscillations., Comment: 10 pages, 5 figures

Federated learning (FL) has emerged as a promising paradigm for training segmentation models on decentralized medical data, owing to its privacy-preserving property. However, existing research overlooks the prevalent annotation noise encountered in real-world medical datasets, which limits the performance ceilings of FL. In this paper, we, for the first time, identify and tackle this problem. For problem formulation, we propose a contour evolution for modeling non-independent and identically distributed (Non-IID) noise across pixels within each client and then extend it to the case of multi-source data to form a heterogeneous noise model (i.e., Non-IID annotation noise across clients). For robust learning from annotations with such two-level Non-IID noise, we emphasize the importance of data quality in model aggregation, allowing high-quality clients to have a greater impact on FL. To achieve this, we propose Federated learning with Annotation quAlity-aware AggregatIon, named FedA3I, by introducing a quality factor based on client-wise noise estimation. Specifically, noise estimation at each client is accomplished through the Gaussian mixture model and then incorporated into model aggregation in a layer-wise manner to up-weight high-quality clients. Extensive experiments on two real-world medical image segmentation datasets demonstrate the superior performance of FedA$^3$I against the state-of-the-art approaches in dealing with cross-client annotation noise. The code is available at https://github.com/wnn2000/FedAAAI., Comment: Accepted at AAAI'24

Text prompts are crucial for generalizing pre-trained open-set object detection models to new categories. However, current methods for text prompts are limited as they require manual feedback when generalizing to new categories, which restricts their ability to model complex scenes, often leading to incorrect detection results. To address this limitation, we propose a novel visual prompt method that learns new category knowledge from a few labeled images, which generalizes the pre-trained detection model to the new category. To allow visual prompts to represent new categories adequately, we propose a statistical-based prompt construction module that is not limited by predefined vocabulary lengths, thus allowing more vectors to be used when representing categories. We further utilize the category dictionaries in the pre-training dataset to design task-specific similarity dictionaries, which make visual prompts more discriminative. We evaluate the method on the ODinW dataset and show that it outperforms existing prompt learning methods and performs more consistently in combinatorial inference., Comment: Accepted at ICASSP 2024