Search

We introduce an innovative system, AutoGameUI, for efficiently constructing cohesive user interfaces in game development. Our system is the first to address the coherence issue arising from integrating inconsistent UI and UX designs, typically leading to mismatches and inefficiencies. We propose a two-stage multimodal learning pipeline to obtain comprehensive representations of both UI and UX designs, and to establish their correspondences. Through the correspondences, a cohesive user interface is automatically constructed from pairwise designs. To achieve high-fidelity effects, we introduce a universal data protocol for precise design descriptions and cross-platform applications. We also develop an interactive web-based tool for game developers to facilitate the use of our system. We create a game UI dataset from actual game projects and combine it with a public dataset for training and evaluation. Our experimental results demonstrate the effectiveness of our system in maintaining coherence between the constructed interfaces and the original designs., Comment: 27 pages

The spontaneous conversion of muonium to antimuonium is one of the interesting charged lepton flavor violation phenomena, offering a sensitive probe of potential new physics and serving as a tool to constrain the parameter space beyond the Standard Model. Utilizing a high-intensity muon beam, a Michel electron magnetic spectrometer and a positron transport solenoid together with a positron detection system, MACE aims to discover or constrain this rare process at the conversion probability beyond the level of $10^{-13}$. This report provides an overview of the theoretical framework and detailed experimental design in the search for the muonium-to-antimuonium conversion., Comment: 115 pages, 59 figures

With the rapid advancement of large language models (LLMs), aligning policy models with human preferences has become increasingly critical. Direct Preference Optimization (DPO) has emerged as a promising approach for alignment, acting as an RL-free alternative to Reinforcement Learning from Human Feedback (RLHF). Despite DPO's various advancements and inherent limitations, an in-depth review of these aspects is currently lacking in the literature. In this work, we present a comprehensive review of the challenges and opportunities in DPO, covering theoretical analyses, variants, relevant preference datasets, and applications. Specifically, we categorize recent studies on DPO based on key research questions to provide a thorough understanding of DPO's current landscape. Additionally, we propose several future research directions to offer insights on model alignment for the research community.

Concept activation vector (CAV) has attracted broad research interest in explainable AI, by elegantly attributing model predictions to specific concepts. However, the training of CAV often necessitates a large number of high-quality images, which are expensive to curate and thus limited to a predefined set of concepts. To address this issue, we propose Language-Guided CAV (LG-CAV) to harness the abundant concept knowledge within the certain pre-trained vision-language models (e.g., CLIP). This method allows training any CAV without labeled data, by utilizing the corresponding concept descriptions as guidance. To bridge the gap between vision-language model and the target model, we calculate the activation values of concept descriptions on a common pool of images (probe images) with vision-language model and utilize them as language guidance to train the LG-CAV. Furthermore, after training high-quality LG-CAVs related to all the predicted classes in the target model, we propose the activation sample reweighting (ASR), serving as a model correction technique, to improve the performance of the target model in return. Experiments on four datasets across nine architectures demonstrate that LG-CAV achieves significantly superior quality to previous CAV methods given any concept, and our model correction method achieves state-of-the-art performance compared to existing concept-based methods. Our code is available at https://github.com/hqhQAQ/LG-CAV.

Large-scale multiple-input multiple-output (MIMO) holds great promise for the fifth-generation (5G) and future communication systems. In near-field scenarios, the spherical wavefront model is commonly utilized to accurately depict the propagation characteristics of large-scale MIMO communication channels. However, employing this modeling method necessitates the computation of angle and distance parameters for each antenna element, resulting in challenges regarding computational complexity. To solve this problem, we introduce a subarray decomposition scheme with the purpose of dividing the whole large-scale antenna array into several smaller subarrays. This scheme is implemented in the near-field channel modeling for large-scale MIMO communications between the base stations (BS) and the mobile receiver (MR). Essential channel propagation statistics, such as spatial cross-correlation functions (CCFs), temporal auto-correlation functions (ACFs), frequency correlation functions (CFs), and channel capacities, are derived and discussed. A comprehensive analysis is conducted to investigate the influence of the height of the BS, motion characteristics of the MR, and antenna configurations on the channel statistics. The proposed channel model criterions, such as the modeling precision and computational complexity, are also theoretically compared. Numerical results demonstrate the effectiveness of the presented communication model in obtaining a good tradeoff between modeling precision and computational complexity.

Large language models have been successfully applied to programming assistance tasks, such as code completion, code insertion, and instructional code editing. However, these applications remain insufficiently automated and struggle to effectively integrate various types of information during the programming process, including coding history, current code, and user instructions. In this work, we propose a new conversational framework that comprehensively integrates these information sources, collect data to train our models and evaluate their performance. Firstly, to thoroughly evaluate how well models align with different types of information and the quality of their outputs, we introduce a new benchmark, APEval (Assist Programming Eval), to comprehensively assess the performance of models in programming assistance tasks. Then, for data collection, we develop a data generation pipeline, Programming-Instruct, which synthesizes training data from diverse sources, such as GitHub and online judge platforms. This pipeline can automatically generate various types of messages throughout the programming process. Finally, using this pipeline, we generate 219K samples, fine-tune multiple models, and develop the CursorCore series. We show that CursorCore outperforms other models of comparable size. This framework unifies applications such as inline chat and automated editing, contributes to the advancement of coding assistants. Code, models and data are freely available at https://github.com/TechxGenus/CursorCore.

Video generation requires modeling a vast spatiotemporal space, which demands significant computational resources and data usage. To reduce the complexity, the prevailing approaches employ a cascaded architecture to avoid direct training with full resolution. Despite reducing computational demands, the separate optimization of each sub-stage hinders knowledge sharing and sacrifices flexibility. This work introduces a unified pyramidal flow matching algorithm. It reinterprets the original denoising trajectory as a series of pyramid stages, where only the final stage operates at the full resolution, thereby enabling more efficient video generative modeling. Through our sophisticated design, the flows of different pyramid stages can be interlinked to maintain continuity. Moreover, we craft autoregressive video generation with a temporal pyramid to compress the full-resolution history. The entire framework can be optimized in an end-to-end manner and with a single unified Diffusion Transformer (DiT). Extensive experiments demonstrate that our method supports generating high-quality 5-second (up to 10-second) videos at 768p resolution and 24 FPS within 20.7k A100 GPU training hours. All code and models will be open-sourced at https://pyramid-flow.github.io.

Large language models (LLMs) have increased the demand for personalized and stylish content generation. However, closed-source models like GPT-4 present limitations in optimization opportunities, while the substantial training costs and inflexibility of open-source alternatives, such as Qwen-72B, pose considerable challenges. Conversely, small language models (SLMs) struggle with understanding complex instructions and transferring learned capabilities to new contexts, often exhibiting more pronounced limitations. In this paper, we present a novel collaborative training framework that leverages the strengths of both LLMs and SLMs for style article generation, surpassing the performance of either model alone. We freeze the LLMs to harness their robust instruction-following capabilities and subsequently apply supervised fine-tuning on the SLM using style-specific data. Additionally, we introduce a self-improvement method to enhance style consistency. Our new benchmark, NoteBench, thoroughly evaluates style-aligned generation. Extensive experiments show that our approach achieves state-of-the-art performance, with improvements of 0.78 in ROUGE-L and 0.55 in BLEU-4 scores compared to GPT-4, while maintaining a low hallucination rate regarding factual and faithfulness.

Recent advances in Multi-modal Large Language Models (MLLMs), such as LLaVA-series models, are driven by massive machine-generated instruction-following data tuning. Such automatic instruction collection pipelines, however, inadvertently introduce significant variability in data quality. This paper introduces a novel instruction curation algorithm, derived from two unique perspectives, human and LLM preference alignment, to compress this vast corpus of machine-generated multimodal instructions to a compact and high-quality form: (i) For human preference alignment, we have collected a machine-generated multimodal instruction dataset and established a comprehensive set of both subjective and objective criteria to guide the data quality assessment critically from human experts. By doing so, a reward model was trained on the annotated dataset to internalize the nuanced human understanding of instruction alignment. (ii) For LLM preference alignment, given the instruction selected by the reward model, we propose leveraging the inner LLM used in MLLM to align the writing style of visual instructions with that of the inner LLM itself, resulting in LLM-aligned instruction improvement. Extensive experiments demonstrate that we can maintain or even improve model performance by compressing synthetic multimodal instructions by up to 90%. Impressively, by aggressively reducing the total training sample size from 158k to 14k (9$\times$ smaller), our model consistently outperforms its full-size dataset counterpart across various MLLM benchmarks. Our project is available at https://github.com/DCDmllm/Align2LLaVA.

Personalized text-to-image generation methods can generate customized images based on the reference images, which have garnered wide research interest. Recent methods propose a finetuning-free approach with a decoupled cross-attention mechanism to generate personalized images requiring no test-time finetuning. However, when multiple reference images are provided, the current decoupled cross-attention mechanism encounters the object confusion problem and fails to map each reference image to its corresponding object, thereby seriously limiting its scope of application. To address the object confusion problem, in this work we investigate the relevance of different positions of the latent image features to the target object in diffusion model, and accordingly propose a weighted-merge method to merge multiple reference image features into the corresponding objects. Next, we integrate this weighted-merge method into existing pre-trained models and continue to train the model on a multi-object dataset constructed from the open-sourced SA-1B dataset. To mitigate object confusion and reduce training costs, we propose an object quality score to estimate the image quality for the selection of high-quality training samples. Furthermore, our weighted-merge training framework can be employed on single-object generation when a single object has multiple reference images. The experiments verify that our method achieves superior performance to the state-of-the-arts on the Concept101 dataset and DreamBooth dataset of multi-object personalized image generation, and remarkably improves the performance on single-object personalized image generation. Our code is available at https://github.com/hqhQAQ/MIP-Adapter.

Instance segmentation plays a vital role in the morphological quantification of biomedical entities such as tissues and cells, enabling precise identification and delineation of different structures. Current methods often address the challenges of touching, overlapping or crossing instances through individual modeling, while neglecting the intrinsic interrelation between these conditions. In this work, we propose a Gradient Anomaly-aware Biomedical Instance Segmentation approach (GAInS), which leverages instance gradient information to perceive local gradient anomaly regions, thus modeling the spatial relationship between instances and refining local region segmentation. Specifically, GAInS is firstly built on a Gradient Anomaly Mapping Module (GAMM), which encodes the radial fields of instances through window sliding to obtain instance gradient anomaly maps. To efficiently refine boundaries and regions with gradient anomaly attention, we propose an Adaptive Local Refinement Module (ALRM) with a gradient anomaly-aware loss function. Extensive comparisons and ablation experiments in three biomedical scenarios demonstrate that our proposed GAInS outperforms other state-of-the-art (SOTA) instance segmentation methods. The code is available at https://github.com/DeepGAInS/GAInS., Comment: Accepted by BIBM2024

Cytology screening from Papanicolaou (Pap) smears is a common and effective tool for the preventive clinical management of cervical cancer, where abnormal cell detection from whole slide images serves as the foundation for reporting cervical cytology. However, cervical cell detection remains challenging due to 1) hazily-defined cell types (e.g., ASC-US) with subtle morphological discrepancies caused by the dynamic cancerization process, i.e., cell class ambiguity, and 2) imbalanced class distributions of clinical data may cause missed detection, especially for minor categories, i.e., cell class imbalance. To this end, we propose a holistic and historical instance comparison approach for cervical cell detection. Specifically, we first develop a holistic instance comparison scheme enforcing both RoI-level and class-level cell discrimination. This coarse-to-fine cell comparison encourages the model to learn foreground-distinguishable and class-wise representations. To emphatically improve the distinguishability of minor classes, we then introduce a historical instance comparison scheme with a confident sample selection-based memory bank, which involves comparing current embeddings with historical embeddings for better cell instance discrimination. Extensive experiments and analysis on two large-scale cytology datasets including 42,592 and 114,513 cervical cells demonstrate the effectiveness of our method. The code is available at https://github.com/hjiangaz/HERO., Comment: Accepted by BIBM2024

With the emergence of integrated sensing, communication, and computation (ISCC) in the upcoming 6G era, federated learning with ISCC (FL-ISCC), integrating sample collection, local training, and parameter exchange and aggregation, has garnered increasing interest for enhancing training efficiency. Currently, FL-ISCC primarily includes two algorithms: FedAVG-ISCC and FedSGD-ISCC. However, the theoretical understanding of the performance and advantages of these algorithms remains limited. To address this gap, we investigate a general FL-ISCC framework, implementing both FedAVG-ISCC and FedSGD-ISCC. We experimentally demonstrate the substantial potential of the ISCC framework in reducing latency and energy consumption in FL. Furthermore, we provide a theoretical analysis and comparison. The results reveal that:1) Both sample collection and communication errors negatively impact algorithm performance, highlighting the need for careful design to optimize FL-ISCC applications. 2) FedAVG-ISCC performs better than FedSGD-ISCC under IID data due to its advantage with multiple local updates. 3) FedSGD-ISCC is more robust than FedAVG-ISCC under non-IID data, where the multiple local updates in FedAVG-ISCC worsen performance as non-IID data increases. FedSGD-ISCC maintains performance levels similar to IID conditions. 4) FedSGD-ISCC is more resilient to communication errors than FedAVG-ISCC, which suffers from significant performance degradation as communication errors increase.Extensive simulations confirm the effectiveness of the FL-ISCC framework and validate our theoretical analysis., Comment: due to the limitation The abstract field cannot be longer than 1,920 characters", the abstract appearing here is slightly shorter than that in the PDF file

The primordial neutrino asymmetry leave profound imprints on the evolution history of the universe, which can be constrained by cosmological observations, including Big Bang Nucleosynthesis (BBN), Cosmic Microwave Background (CMB), and Large-Scale Structure (LSS). We present comprehensive analysis on implications and constraints of the primordial neutrino asymmetry $\xi_\nu$, based on a precise treatment of neutrino decoupling by solving the complete (anti)neutrino quantum kinetic equations in the Closed-Time-Path formalism. The effective number of neutrinos $ N_{\rm eff}$ and (anti)neutrino spectral distortions are calculated, and we find that the non-instantaneous decoupling correction is given by $\delta N_{\rm eff} = 0.0440 + 0.0102 \, \xi_\nu^2$. Then we perform the state-of-the-art calculation for the abundance of light elements including (anti)neutrino spectral distortions, which indicate a positive asymmetry $0.032 \leq \xi_\nu \leq 0.052$ from EMPRESS data. The implications of the neutrino asymmetry for the CMB and LSS are studied in detail, and we find that the Baryon Acoustic Oscillations (BAO) are also significantly affected by $\xi_\nu$ in addition to the sum of neutrino masses. A combined analysis with EMPRESS BBN, Planck CMB and BOSS BAO data yields a tighter constraint $\xi_\nu = 0.024 \pm 0.012$, which provides constraints on UV models capable of producing large asymmetries., Comment: 56 pages, 12 figures

We report a layered transition-metal-ordered oxypnictide Sr$_{2}$CrCoAsO$_{3}$. The new material was synthesized by solid-state reactions under vacuum. It has an intergrowth structure with a perovskite-like Sr$_3$Cr$_2$O$_6$ unit and ThCr$_2$Si$_2$-type SrCo$_2$As$_2$ block stacking coherently along the crystallographic $c$ axis. The measurements of electrical resistivity, magnetic susceptibility, and specific heat indicate metallic conductivity from the CoAs layers and short-range antiferromagnetic ordering in the CrO$_{2}$ planes. No itinerant-electron ferromagnetism expected in CoAs layers is observed. This result, combined with the first-principles calculations and the previous reports of other CoAs-layer-based materials, suggests that the Co$-$Co bondlength plays a crucial role in the emergence of itinerant ferromagnetism., Comment: 9 pages, 7 figures

We investigate constraints on neutrino non-standard interactions (NSIs) in the effective field theory framework, using data from the first measurement of solar $^8$B neutrinos via coherent elastic neutrino-nucleus scattering (CE$\nu$NS) in the PandaX-4T experiment and the COHERENT experiment. In the PandaX-4T experiment, due to relatively large statistical uncertainties and measured CE$\nu$NS counts that significantly differ from the Standard Model predictions, its sensitivities to the neutrino NSIs are currently limited, compared to the COHERENT experiment. However, the PandaX-4T experiment is uniquely sensitive to the neutrino NSIs for the $\tau$ flavor due to oscillation feature of the solar $^8$B neutrinos. We also assess how the experimental central value, exposure, and systematic uncertainties will affect the constraints on neutrino NSIs from various CE$\nu$NS measurements in the future., Comment: 9 pages, 5 figures, 3 tables

Robotic object singulation, where a robot must isolate, grasp, and retrieve a target object in a cluttered environment, is a fundamental challenge in robotic manipulation. This task is difficult due to occlusions and how other objects act as obstacles for manipulation. A robot must also reason about the effect of object-object interactions as it tries to singulate the target. Prior work has explored object singulation in scenarios where there is enough free space to perform relatively long pushes to separate objects, in contrast to when space is tight and objects have little separation from each other. In this paper, we propose the Singulating Objects in Packed Environments (SOPE) framework. We propose a novel method that involves a displacement-based state representation and a multi-phase reinforcement learning procedure that enables singulation using the 16-DOF Allegro Hand. We demonstrate extensive experiments in Isaac Gym simulation, showing the ability of our system to singulate a target object in clutter. We directly transfer the policy trained in simulation to the real world. Over 250 physical robot manipulation trials, our method obtains success rates of 79.2%, outperforming alternative learning and non-learning methods.

We introduce LLaVA-MoD, a novel framework designed to enable the efficient training of small-scale Multimodal Language Models (s-MLLM) by distilling knowledge from large-scale MLLM (l-MLLM). Our approach tackles two fundamental challenges in MLLM distillation. First, we optimize the network structure of s-MLLM by integrating a sparse Mixture of Experts (MoE) architecture into the language model, striking a balance between computational efficiency and model expressiveness. Second, we propose a progressive knowledge transfer strategy to ensure comprehensive knowledge migration. This strategy begins with mimic distillation, where we minimize the Kullback-Leibler (KL) divergence between output distributions to enable the student model to emulate the teacher network's understanding. Following this, we introduce preference distillation via Direct Preference Optimization (DPO), where the key lies in treating l-MLLM as the reference model. During this phase, the s-MLLM's ability to discriminate between superior and inferior examples is significantly enhanced beyond l-MLLM, leading to a better student that surpasses its teacher, particularly in hallucination benchmarks. Extensive experiments demonstrate that LLaVA-MoD outperforms existing models across various multimodal benchmarks while maintaining a minimal number of activated parameters and low computational costs. Remarkably, LLaVA-MoD, with only 2B activated parameters, surpasses Qwen-VL-Chat-7B by an average of 8.8% across benchmarks, using merely 0.3% of the training data and 23% trainable parameters. These results underscore LLaVA-MoD's ability to effectively distill comprehensive knowledge from its teacher model, paving the way for the development of more efficient MLLMs. The code will be available on: https://github.com/shufangxun/LLaVA-MoD.

Providing high-quality item recall for text queries is crucial in large-scale e-commerce search systems. Current Embedding-based Retrieval Systems (ERS) embed queries and items into a shared low-dimensional space, but uni-modality ERS rely too heavily on textual features, making them unreliable in complex contexts. While multi-modality ERS incorporate various data sources, they often overlook individual preferences for different modalities, leading to suboptimal results. To address these issues, we propose MRSE, a Multi-modality Retrieval System that integrates text, item images, and user preferences through lightweight mixture-of-expert (LMoE) modules to better align features across and within modalities. MRSE also builds user profiles at a multi-modality level and introduces a novel hybrid loss function that enhances consistency and robustness using hard negative sampling. Experiments on a large-scale dataset from Shopee and online A/B testing show that MRSE achieves an 18.9% improvement in offline relevance and a 3.7% gain in online core metrics compared to Shopee's state-of-the-art uni-modality system.

The gap between the trepidation of program reliability and the expense of repairs underscores the indispensability of Automated Program Repair (APR). APR is instrumental in transforming vulnerable programs into more robust ones, bolstering program reliability while simultaneously diminishing the financial burden of manual repairs. Commercial-scale language models (LM) have taken APR to unprecedented levels. However, the emergence reveals that for models fewer than 100B parameters, making single-step modifications may be difficult to achieve the desired effect. Moreover, humans interact with the LM through explicit prompts, which hinders the LM from receiving feedback from compiler and test cases to automatically optimize its repair policies. In this literature, we explore how small-scale LM (less than 20B) achieve excellent performance through process supervision and feedback. We start by constructing a dataset named CodeNet4Repair, replete with multiple repair records, which supervises the fine-tuning of a foundational model. Building upon the encouraging outcomes of reinforcement learning, we develop a reward model that serves as a critic, providing feedback for the fine-tuned LM's action, progressively optimizing its policy. During inference, we require the LM to generate solutions iteratively until the repair effect no longer improves or hits the maximum step limit. The results show that process-based not only outperforms larger outcome-based generation methods, but also nearly matches the performance of closed-source commercial large-scale LMs., Comment: 15 pages, 13 figures

While Parameter-Efficient Fine-Tuning (PEFT) methods like LoRA have effectively addressed GPU memory constraints during fine-tuning, their performance often falls short, especially in multidimensional task scenarios. To address this issue, one straightforward solution is to introduce task-specific LoRA modules as domain experts, leveraging the modeling of multiple experts' capabilities and thus enhancing the general capability of multi-task learning. Despite promising, these additional components often add complexity to the training and inference process, contravening the efficient characterization of PEFT designed for. Considering this, we introduce an innovative PEFT method, TeamLoRA, consisting of a collaboration and competition module for experts, and thus achieving the right balance of effectiveness and efficiency: (i) For collaboration, a novel knowledge-sharing and -organizing mechanism is devised to appropriately reduce the scale of matrix operations, thereby boosting the training and inference speed. (ii) For competition, we propose leveraging a game-theoretic interaction mechanism for experts, encouraging experts to transfer their domain-specific knowledge while facing diverse downstream tasks, and thus enhancing the performance. By doing so, TeamLoRA elegantly connects the experts as a "Team" with internal collaboration and competition, enabling a faster and more accurate PEFT paradigm for multi-task learning. To validate the superiority of TeamLoRA, we curate a comprehensive multi-task evaluation(CME) benchmark to thoroughly assess the capability of multi-task learning. Experiments conducted on our CME and other benchmarks indicate the effectiveness and efficiency of TeamLoRA. Our project is available at https://github.com/Lin-Tianwei/TeamLoRA.

A novel low-complexity wavenumber-domain method is proposed for near-field sensing (NISE). Specifically, the power-concentrated region of the wavenumber-domain channels is related to the target position in a non-linear manner. Based on this observation, a bi-directional convolutional neural network (BiCNN)-based approach is proposed to capture such a relationship, thereby facilitating low-complexity target localization. This method enables direct and gridless target localization using only a limited bandwidth and a single antenna array. Simulation results demonstrate that: 1) during the offline training phase, the proposed BiCNN method can learn to localize the target with fewer trainable parameters compared to the naive neural network architectures; and 2) during the online implementation phase, the BiCNN method can spend 100x less time while maintaining comparable performance to the conventional two-dimensional multiple signal classification (MUSIC) algorithms., Comment: This work has been submitted to the IEEE for possible publication

Persistent homology, a fundamental technique within Topological Data Analysis (TDA), captures structural and shape characteristics of graphs, yet encounters computational difficulties when applied to dynamic directed graphs. This paper introduces the Dynamic Neural Dowker Network (DNDN), a novel framework specifically designed to approximate the results of dynamic Dowker filtration, aiming to capture the high-order topological features of dynamic directed graphs. Our approach creatively uses line graph transformations to produce both source and sink line graphs, highlighting the shared neighbor structures that Dowker complexes focus on. The DNDN incorporates a Source-Sink Line Graph Neural Network (SSLGNN) layer to effectively capture the neighborhood relationships among dynamic edges. Additionally, we introduce an innovative duality edge fusion mechanism, ensuring that the results for both the sink and source line graphs adhere to the duality principle intrinsic to Dowker complexes. Our approach is validated through comprehensive experiments on real-world datasets, demonstrating DNDN's capability not only to effectively approximate dynamic Dowker filtration results but also to perform exceptionally in dynamic graph classification tasks., Comment: KDD 2024

While deep neural networks have achieved remarkable performance, they tend to lack transparency in prediction. The pursuit of greater interpretability in neural networks often results in a degradation of their original performance. Some works strive to improve both interpretability and performance, but they primarily depend on meticulously imposed conditions. In this paper, we propose a simple yet effective framework that acquires more explainable activation heatmaps and simultaneously increase the model performance, without the need for any extra supervision. Specifically, our concise framework introduces a new metric, i.e., explanation consistency, to reweight the training samples adaptively in model learning. The explanation consistency metric is utilized to measure the similarity between the model's visual explanations of the original samples and those of semantic-preserved adversarial samples, whose background regions are perturbed by using image adversarial attack techniques. Our framework then promotes the model learning by paying closer attention to those training samples with a high difference in explanations (i.e., low explanation consistency), for which the current model cannot provide robust interpretations. Comprehensive experimental results on various benchmarks demonstrate the superiority of our framework in multiple aspects, including higher recognition accuracy, greater data debiasing capability, stronger network robustness, and more precise localization ability on both regular networks and interpretable networks. We also provide extensive ablation studies and qualitative analyses to unveil the detailed contribution of each component., Comment: To appear in IEEE Transactions on Multimedia

With the wide application of biometrics, more and more attention has been paid to the security of biometric templates. However most of existing biometric template protection (BTP) methods have some security problems, e.g. the problem that protected templates leak part of the original biometric data (exists in Cancelable Biometrics (CB)), the use of error-correcting codes (ECC) leads to decodable attack, statistical attack (exists in Biometric Cryptosystems (BCS)), the inability to achieve revocability (exists in methods using Neural Network (NN) to learn pre-defined templates), the inability to use cryptographic hash to guarantee strong security (exists in CB and methods using NN to learn latent templates). In this paper, we propose a framework called BioDeepHash based on deep hashing and cryptographic hashing to address the above four problems, where different biometric data of the same user are mapped to a stable code using deep hashing instead of predefined binary codes thus avoiding the use of ECC. An application-specific binary string is employed to achieve revocability. Then cryptographic hashing is used to get the final protected template to ensure strong security. Ultimately our framework achieves not storing any data that would leak part of the original biometric data. We also conduct extensive experiments on facial and iris datasets. Our method achieves an improvement of 10.12$\%$ on the average Genuine Acceptance Rate (GAR) for iris data and 3.12$\%$ for facial data compared to existing methods. In addition, BioDeepHash achieves extremely low False Acceptance Rate (FAR), i.e. 0$\%$ FAR on the iris dataset and the highest FAR on the facial dataset is only 0.0002$\%$.

A near-field sensing (NISE) enabled predictive beamforming framework is proposed to facilitate wireless communications with high-mobility channels. Unlike conventional far-field sensing, which only captures the angle and the radial velocity of the user, NISE enables the estimation of the full motion state, including additional distance and transverse velocity information. Two full-motion state sensing approaches are proposed based on the concepts of estimation and tracking, respectively. 1)AGD-AO approach: The full motion state of the user is estimated within a single CPI. In particular, the gradient descent is adopted to estimate the transverse and radial velocities of the user based on the maximum likelihood criteria, while the distance and the angle are calculated by the kinematic model. In this process, moment estimations are leveraged to adaptively tune the step size, thereby leading to a smoother and faster gradient descent. 2) EKF approach: The full motion state of the user is tracked across multiple CPIs. Based on the noisy measurements in multiple CPIs, the EKF iteratively predicts and updates the current motion state to achieve a low tracking error. Based on the obtained full motion state, the beam prediction, and Doppler frequency compensation can be carried out with minimum pilot overhead. Numerical results are provided to validate the effectiveness and efficiency of the proposed approach compared to the conventional far-field predictive beamforming and feedback-based approaches. It is also revealed that: 1)the proposed AGD-AO can achieve stable descending with small gradients, thereby accelerating convergence; 2) compared to far-field predictive beamforming and feedback-based schemes, both of the proposed methods exhibit superior performance; and 3) by incorporating multiple CPIs, the EKF method exhibits greater robustness in low SNR regions., Comment: This work has been submitted to the IEEE for possible publication

Dynamic spectrum sharing (DSS) between satellite and terrestrial networks has increasingly engaged the academic and industrial sectors. Nevertheless, facilitating secure, efficient and scalable sharing continues to pose a pivotal challenge. Emerging as a promising technology to bridge the trust gap among multiple participants, blockchain has been envisioned to enable DSS in a decentralized manner. However, satellites with limited resources may struggle to support the frequent interactions required by blockchain networks. Additionally,given the extensive coverage of satellites, spectrum sharing needs vary by regions, challenging traditional blockchain approaches to accommodate differences. In this work, a partitioned, self-governed, and customized dynamic spectrum sharing approach (PSC-DSS) is proposed for spectrum sharing between satellite access networks and terrestrial access networks. This approach establishes a sharded and tiered architecture which allows various regions to manage spectrum autonomously while jointly maintaining a single blockchain ledger. Moreover, a spectrum-consensus integrated mechanism, which decouples DSS process and couples it with blockchain consensus protocol, is designed to enable regions to conduct DSS transactions in parallel and dynamically innovate spectrum sharing schemes without affecting others. Furthermore, a theoretical framework is derived to justify the stability performance of PSC-DSS. Finally, simulations and experiments are conducted to validate the advantageous performance of PSC-DSS in terms of low-overhead, high efficiency, and robust stability.

Attribution-based explanations are garnering increasing attention recently and have emerged as the predominant approach towards \textit{eXplanable Artificial Intelligence}~(XAI). However, the absence of consistent configurations and systematic investigations in prior literature impedes comprehensive evaluations of existing methodologies. In this work, we introduce {Meta-Rank}, an open platform for benchmarking attribution methods in the image domain. Presently, Meta-Rank assesses eight exemplary attribution methods using six renowned model architectures on four diverse datasets, employing both the \textit{Most Relevant First} (MoRF) and \textit{Least Relevant First} (LeRF) evaluation protocols. Through extensive experimentation, our benchmark reveals three insights in attribution evaluation endeavors: 1) evaluating attribution methods under disparate settings can yield divergent performance rankings; 2) although inconsistent across numerous cases, the performance rankings exhibit remarkable consistency across distinct checkpoints along the same training trajectory; 3) prior attempts at consistent evaluation fare no better than baselines when extended to more heterogeneous models and datasets. Our findings underscore the necessity for future research in this domain to conduct rigorous evaluations encompassing a broader range of models and datasets, and to reassess the assumptions underlying the empirical success of different attribution methods. Our code is publicly available at \url{https://github.com/TreeThree-R/Meta-Rank}., Comment: Accepted as a conference paper by ECCV 2024

Flat bands and electron correlation in moir\'e lattices give rise to many exotic phases, including Mott insulators, superconductivity, and topological states. Within the first moir\'e band, integer and fractional quantum anomalous Hall effects have been observed in twisted bilayer MoTe2 (tMoTe2) at one hole doping and fractional doping per moir\'e unit cell, respectively. When the second moir\'e band is fully hole doped, quantum spin Hall insulator has also been reported in tMoTe2 at a certain twist angle. Exotic topological states together with ferromagnetic (FM) states in the high moir\'e band can potentially exist as well. In this study, we report the observation of a FM phase in the second moir\'e band in tMoTe2. The FM phase can be tuned by both the doping level and displacement field. At filling around 2.58 holes per moir\'e unit cell, the FM phase reaches a Curie temperature of 3.5 K. A large displacement field can suppress the FM phase, like the FM phase at the filling of -1. Our results demonstrate the realization of time-reversal symmetry-breaking states in the higher moir\'e bands in tMoTe2., Comment: Main text: 13 pages, 5 figures. Supplementary: 11 pages, 15 figures

The brain is dynamic, associative and efficient. It reconfigures by associating the inputs with past experiences, with fused memory and processing. In contrast, AI models are static, unable to associate inputs with past experiences, and run on digital computers with physically separated memory and processing. We propose a hardware-software co-design, a semantic memory-based dynamic neural network (DNN) using memristor. The network associates incoming data with the past experience stored as semantic vectors. The network and the semantic memory are physically implemented on noise-robust ternary memristor-based Computing-In-Memory (CIM) and Content-Addressable Memory (CAM) circuits, respectively. We validate our co-designs, using a 40nm memristor macro, on ResNet and PointNet++ for classifying images and 3D points from the MNIST and ModelNet datasets, which not only achieves accuracy on par with software but also a 48.1% and 15.9% reduction in computational budget. Moreover, it delivers a 77.6% and 93.3% reduction in energy consumption., Comment: In press