Search

FPGA architectures have recently been enhanced to meet the substantial computational demands of modern deep neural networks (DNNs). To this end, both FPGA vendors and academic researchers have proposed in-fabric blocks that perform efficient tensor computations. However, these blocks are primarily optimized for dense computation, while most DNNs exhibit sparsity. To address this limitation, we propose incorporating structured sparsity support into FPGA architectures. We architect 2D systolic in-fabric blocks, named systolic sparse tensor (SST) slices, that support multiple degrees of sparsity to efficiently accelerate a wide variety of DNNs. SSTs support dense operation, 2:4 (50%) and 1:4 (75%) sparsity, as well as a new 1:3 (66.7%) sparsity level to further increase flexibility. When demonstrating on general matrix multiplication (GEMM) accelerators, which are the heart of most current DNN accelerators, our sparse SST-based designs attain up to 5x higher FPGA frequency and 10.9x lower area, compared to traditional FPGAs. Moreover, evaluation of the proposed SSTs on state-of-the-art sparse ViT and CNN models exhibits up to 3.52x speedup with minimal area increase of up to 13.3%, compared to dense in-fabric acceleration., Comment: Accepted as full paper at FPGA 2025

Accurate and efficient segmentation of brain tumors is critical for diagnosis, treatment planning, and monitoring in clinical practice. In this study, we present an enhanced ResUNet architecture for automatic brain tumor segmentation, integrating an EfficientNetB0 encoder, a channel attention mechanism, and an Atrous Spatial Pyramid Pooling (ASPP) module. The EfficientNetB0 encoder leverages pre-trained features to improve feature extraction efficiency, while the channel attention mechanism enhances the model's focus on tumor-relevant features. ASPP enables multiscale contextual learning, crucial for handling tumors of varying sizes and shapes. The proposed model was evaluated on two benchmark datasets: TCGA LGG and BraTS 2020. Experimental results demonstrate that our method consistently outperforms the baseline ResUNet and its EfficientNet variant, achieving Dice coefficients of 0.903 and 0.851 and HD95 scores of 9.43 and 3.54 for whole tumor and tumor core regions on the BraTS 2020 dataset, respectively. compared with state-of-the-art methods, our approach shows competitive performance, particularly in whole tumor and tumor core segmentation. These results indicate that combining a powerful encoder with attention mechanisms and ASPP can significantly enhance brain tumor segmentation performance. The proposed approach holds promise for further optimization and application in other medical image segmentation tasks., Comment: 13 pages, 1 figure

Spin coupling between magnetic metal atoms and organic radicals plays a pivotal role in high-performance magnetic materials. The complex interaction involving multi-spin centers in bulk materials makes it challenging to study spin coupling at the atomic scale. Here, we investigate the $d$-$\pi$-$d$ spin interaction in well-defined metal-organic coordinated structures composed of two iron (Fe) atoms and four all-trans retinoic acid (ReA) molecules, using low-temperature scanning tunneling microscopy and atomic force microscopy. The ReA molecule is turned into a spin-$1/2$ radical state by dehydrogenation, facilitating strong magnetic coupling with the coordinated Fe atoms. Comprehensive theoretical analysis, based on density functional theory and valence bond theory, further elucidates the intrinsic mechanism of ferrimagnetic spin coupling in the coordination structure. Specifically, simultaneous antiferromagnetic coupling of Fe dimer to ReA radicals parallelizes the dimer spin orientation. This work contributes to the fundamental understanding of spin interaction in metal-organic coordination structures and provides microscopic insights for designing advanced magnetic materials.

Real-world networks carry all kinds of noise, resulting in numerous challenges for community detection. Further improving the performance and robustness of community detection has attracted significant attention. This paper considers edge noise, which causes edges in the network to be added or removed. Existing methods achieve graph denoising through link prediction or robustness in low eigenvectors. However, they are either limited in application scenarios or not determined for effectiveness. We find that the noisy pixel in the adjacency matrix has a certain proportion in the loss function, which makes the optimization of the community detection model seriously deviate from the correct direction. Thus, we design an flexible framework to silence the contribution of noisy pixels to loss function, called Silencer. We take the nonnegative matrix factorization (NMF) and deep NMF methods as examples since they are the prime models for community detection. We first prove the convergence of Silencer in NMF. Compared with existing methods, Silencer show top performance in six real-world networks with random noise, adversarial perturbation, and mixed noise. Moreover, Silencer works on random (ER), scale-free (BA), and small-world (WS) networks, and the improvement of Silencer is gradually insignificant in the order ER, BA, and WS networks.

Vision transformers dominate image processing tasks due to their superior performance. However, the quadratic complexity of self-attention limits the scalability of these systems and their deployment on resource-constrained devices. State Space Models (SSMs) have emerged as a solution by introducing a linear recurrence mechanism, which reduces the complexity of sequence modeling from quadratic to linear. Recently, SSMs have been extended to high-resolution vision tasks. Nonetheless, the linear recurrence mechanism struggles to fully utilize matrix multiplication units on modern hardware, resulting in a computational bottleneck. We address this issue by introducing \textit{VMeanba}, a training-free compression method that eliminates the channel dimension in SSMs using mean operations. Our key observation is that the output activations of SSM blocks exhibit low variances across channels. Our \textit{VMeanba} leverages this property to optimize computation by averaging activation maps across the channel to reduce the computational overhead without compromising accuracy. Evaluations on image classification and semantic segmentation tasks demonstrate that \textit{VMeanba} achieves up to a 1.12x speedup with less than a 3\% accuracy loss. When combined with 40\% unstructured pruning, the accuracy drop remains under 3\%., Comment: Accepted by NeurIPS 2024 Machine Learning for Systems workshop

Multipartite entanglement is a crucial resource for a wide range of quantum information processing tasks, including quantum metrology, quantum computing, and quantum communication. The verification of multipartite entanglement, along with an understanding of its intrinsic structure, is of fundamental importance, both for the foundations of quantum mechanics and for the practical applications of quantum information technologies. Nonetheless, detecting entanglement structures remains a significant challenge, particularly for general states and large-scale quantum systems. To address this issue, we develop an efficient algorithm that combines semidefinite programming with a gradient descent method. This algorithm is designed to explore the entanglement structure by examining the inner polytope of the convex set that encompasses all states sharing the same entanglement properties. Through detailed examples, we demonstrate the superior performance of our approach compared to many of the best-known methods available today. Our method not only improves entanglement detection but also provides deeper insights into the complex structures of many-body quantum systems, which is essential for advancing quantum technologies, Comment: 7 pages, 3 figures, part of arXiv:2406.07274

Realizing universal fault-tolerant quantum computation is a key goal in quantum information science. By encoding quantum information into logical qubits utilizing quantum error correcting codes, physical errors can be detected and corrected, enabling substantial reduction in logical error rates. However, the set of logical operations that can be easily implemented on such encoded qubits is often constrained, necessitating the use of special resource states known as 'magic states' to implement universal, classically hard circuits. A key method to prepare high-fidelity magic states is to perform 'distillation', creating them from multiple lower fidelity inputs. Here we present the experimental realization of magic state distillation with logical qubits on a neutral-atom quantum computer. Our approach makes use of a dynamically reconfigurable architecture to encode and perform quantum operations on many logical qubits in parallel. We demonstrate the distillation of magic states encoded in d=3 and d=5 color codes, observing improvements of the logical fidelity of the output magic states compared to the input logical magic states. These experiments demonstrate a key building block of universal fault-tolerant quantum computation, and represent an important step towards large-scale logical quantum processors., Comment: 8+11 pages, 4+4 figures

In real-world applications, spectral Graph Neural Networks (GNNs) are powerful tools for processing diverse types of graphs. However, a single GNN often struggles to handle different graph types-such as homogeneous and heterogeneous graphs-simultaneously. This challenge has led to the manual design of GNNs tailored to specific graph types, but these approaches are limited by the high cost of labor and the constraints of expert knowledge, which cannot keep up with the rapid growth of graph data. To overcome these challenges, we propose AutoSGNN, an automated framework for discovering propagation mechanisms in spectral GNNs. AutoSGNN unifies the search space for spectral GNNs by integrating large language models with evolutionary strategies to automatically generate architectures that adapt to various graph types. Extensive experiments on nine widely-used datasets, encompassing both homophilic and heterophilic graphs, demonstrate that AutoSGNN outperforms state-of-the-art spectral GNNs and graph neural architecture search methods in both performance and efficiency.

Recent advancements in text-to-video (T2V) generation have leveraged diffusion models to enhance the visual coherence of videos generated from textual descriptions. However, most research has primarily focused on object motion, with limited attention given to cinematic language in videos, which is crucial for cinematographers to convey emotion and narrative pacing. To address this limitation, we propose a threefold approach to enhance the ability of T2V models to generate controllable cinematic language. Specifically, we introduce a cinematic language dataset that encompasses shot framing, angle, and camera movement, enabling models to learn diverse cinematic styles. Building on this, to facilitate robust cinematic alignment evaluation, we present CameraCLIP, a model fine-tuned on the proposed dataset that excels in understanding complex cinematic language in generated videos and can further provide valuable guidance in the multi-shot composition process. Finally, we propose CLIPLoRA, a cost-guided dynamic LoRA composition method that facilitates smooth transitions and realistic blending of cinematic language by dynamically fusing multiple pre-trained cinematic LoRAs within a single video. Our experiments demonstrate that CameraCLIP outperforms existing models in assessing the alignment between cinematic language and video, achieving an R@1 score of 0.81. Additionally, CLIPLoRA improves the ability for multi-shot composition, potentially bridging the gap between automatically generated videos and those shot by professional cinematographers., Comment: 13 pages

Image restoration (IR) aims to recover high-quality images from degraded inputs, with recent deep learning advancements significantly enhancing performance. However, existing methods lack a unified training benchmark for iterations and configurations. We also identify a bias in image complexity distributions between commonly used IR training and testing datasets, resulting in suboptimal restoration outcomes. To address this, we introduce a large-scale IR dataset called ReSyn, which employs a novel image filtering method based on image complexity to ensure a balanced distribution and includes both real and AIGC synthetic images. We establish a unified training standard that specifies iterations and configurations for image restoration models, focusing on measuring model convergence and restoration capability. Additionally, we enhance transformer-based image restoration models using linear attention mechanisms by proposing RWKV-IR, which integrates linear complexity RWKV into the transformer structure, allowing for both global and local receptive fields. Instead of directly using Vision-RWKV, we replace the original Q-Shift in RWKV with a Depth-wise Convolution shift to better model local dependencies, combined with Bi-directional attention for comprehensive linear attention. We also introduce a Cross-Bi-WKV module that merges two Bi-WKV modules with different scanning orders for balanced horizontal and vertical attention. Extensive experiments validate the effectiveness of our RWKV-IR model.

Precise breast cancer classification on histopathological images has the potential to greatly improve the diagnosis and patient outcome in oncology. The data imbalance problem largely stems from the inherent imbalance within medical image datasets, where certain tumor subtypes may appear much less frequently. This constitutes a considerable limitation in biased model predictions that can overlook critical but rare classes. In this work, we adopted EfficientNet, a state-of-the-art convolutional neural network (CNN) model that balances high accuracy with computational cost efficiency. To address data imbalance, we introduce an intensive data augmentation pipeline and cost-sensitive learning, improving representation and ensuring that the model does not overly favor majority classes. This approach provides the ability to learn effectively from rare tumor types, improving its robustness. Additionally, we fine-tuned the model using transfer learning, where weights in the beginning trained on a binary classification task were adopted to multi-class classification, improving the capability to detect complex patterns within the BreakHis dataset. Our results underscore significant improvements in the binary classification performance, achieving an exceptional recall increase for benign cases from 0.92 to 0.95, alongside an accuracy enhancement from 97.35 % to 98.23%. Our approach improved the performance of multi-class tasks from 91.27% with regular augmentation to 94.54% with intensive augmentation, reaching 95.04% with transfer learning. This framework demonstrated substantial gains in precision in the minority classes, such as Mucinous carcinoma and Papillary carcinoma, while maintaining high recall consistently across these critical subtypes, as further confirmed by confusion matrix analysis., Comment: 19 pages, 7 figures

Modern autonomous vehicle perception systems often struggle with occlusions and limited perception range. Previous studies have demonstrated the effectiveness of cooperative perception in extending the perception range and overcoming occlusions, thereby improving the safety of autonomous driving. In recent years, a series of cooperative perception datasets have emerged. However, these datasets only focus on camera and LiDAR, overlooking 4D Radar, a sensor employed in single-vehicle autonomous driving for robust perception in adverse weather conditions. In this paper, to bridge the gap of missing 4D Radar datasets in cooperative perception, we present V2X-Radar, the first large real-world multi-modal dataset featuring 4D Radar. Our V2X-Radar dataset is collected using a connected vehicle platform and an intelligent roadside unit equipped with 4D Radar, LiDAR, and multi-view cameras. The collected data includes sunny and rainy weather conditions, spanning daytime, dusk, and nighttime, as well as typical challenging scenarios. The dataset comprises 20K LiDAR frames, 40K camera images, and 20K 4D Radar data, with 350K annotated bounding boxes across five categories. To facilitate diverse research domains, we establish V2X-Radar-C for cooperative perception, V2X-Radar-I for roadside perception, and V2X-Radar-V for single-vehicle perception. We further provide comprehensive benchmarks of recent perception algorithms on the above three sub-datasets. The dataset and benchmark codebase will be available at \url{http://openmpd.com/column/V2X-Radar}., Comment: 11 pages, 5 figures

As a well-known open cluster, NGC 2323 (also called M50) has been widely investigated for over a hundred years and has always been considered a classical single cluster. In this work, with the help of Gaia DR3, we study the binary structure nature of this cluster. Although indistinguishable in the spatial space, the small but undeniable difference in the proper motion indicates that they may be two individual clusters. After investigating the properties of the two clusters, it is found that they have very close positions (three-dimensional $\Delta$pos = 12.3 pc, $\sigma_{\Delta \mathrm{pos}} = 3.4$ pc) and similar tangential velocities (two-dimensional $\Delta$V = 2.2 km s$^{-1}$, $\sigma_{\Delta \mathrm{V}} = 0.02$ km s$^{-1}$), indicating the existence of their physical association. Moreover, the best isochrone fitting ages of the two clusters are the same (158 Myr), further proving their possibly common origin. To comprehensively understand the formation and evolution of this binary cluster, we employ the PETAR $N$-body code to trace back their birthplace and deduce their dynamical evolutionary fate. With observational mean cluster properties, the simulations suggest that they may form together, and then orbit each other as a binary cluster for over 200 Myr. After that, because of their gradual mass loss, the two clusters will eventually separate and evolve into two independent clusters. Meanwhile, the numerical $N$-body simulation suggests that the less massive cluster is unlikely to be the cluster tidal tails created by the differential rotation of the Milky Way., Comment: 14 pages, 8 figures

State Space Models (SSMs) have emerged as an appealing alternative to Transformers for large language models, achieving state-of-the-art accuracy with constant memory complexity which allows for holding longer context lengths than attention-based networks. The superior computational efficiency of SSMs in long sequence modeling positions them favorably over Transformers in many scenarios. However, improving the efficiency of SSMs on request-intensive cloud-serving and resource-limited edge applications is still a formidable task. SSM quantization is a possible solution to this problem, making SSMs more suitable for wide deployment, while still maintaining their accuracy. Quantization is a common technique to reduce the model size and to utilize the low bit-width acceleration features on modern computing units, yet existing quantization techniques are poorly suited for SSMs. Most notably, SSMs have highly sensitive feature maps within the selective scan mechanism (i.e., linear recurrence) and massive outliers in the output activations which are not present in the output of token-mixing in the self-attention modules. To address this issue, we propose a static 8-bit per-tensor SSM quantization method which suppresses the maximum values of the input activations to the selective SSM for finer quantization precision and quantizes the output activations in an outlier-free space with Hadamard transform. Our 8-bit weight-activation quantized Mamba 2.8B SSM benefits from hardware acceleration and achieves a 1.72x lower generation latency on an Nvidia Orin Nano 8G, with only a 0.9% drop in average accuracy on zero-shot tasks. The experiments demonstrate the effectiveness and practical applicability of our approach for deploying SSM-based models of all sizes on both cloud and edge platforms.

Machine unlearning (MU) is becoming a promising paradigm to achieve the "right to be forgotten", where the training trace of any chosen data points could be eliminated, while maintaining the model utility on general testing samples after unlearning. With the advancement of forgetting research, many fundamental open questions remain unanswered: do different samples exhibit varying levels of difficulty in being forgotten? Further, does the sequence in which samples are forgotten, determined by their respective difficulty levels, influence the performance of forgetting algorithms? In this paper, we identify key factor affecting unlearning difficulty and the performance of unlearning algorithms. We find that samples with higher privacy risks are more likely to be unlearning, indicating that the unlearning difficulty varies among different samples which motives a more precise unlearning mode. Built upon this insight, we propose a general unlearning framework, dubbed RSU, which consists of Ranking module and SeqUnlearn module., Comment: The senior authors of the draft are not fully convinced that the novelty is significant enough for this submission compared to the latest research progress in this area. Additionally, the senior authors have identified writing issues. Based on these two reasons, we have decided to withdraw the draft from arXiv

$N{:}M$ sparsity is an emerging model compression method supported by more and more accelerators to speed up sparse matrix multiplication in deep neural networks. Most existing $N{:}M$ sparsity methods compress neural networks with a uniform setting for all layers in a network or heuristically determine the layer-wise configuration by considering the number of parameters in each layer. However, very few methods have been designed for obtaining a layer-wise customized $N{:}M$ sparse configuration for vision transformers (ViTs), which usually consist of transformer blocks involving the same number of parameters. In this work, to address the challenge of selecting suitable sparse configuration for ViTs on $N{:}M$ sparsity-supporting accelerators, we propose ELSA, Exploiting Layer-wise $N{:}M$ Sparsity for ViTs. Considering not only all $N{:}M$ sparsity levels supported by a given accelerator but also the expected throughput improvement, our methodology can reap the benefits of accelerators supporting mixed sparsity by trading off negligible accuracy loss with both memory usage and inference time reduction for ViT models. For instance, our approach achieves a noteworthy 2.9$\times$ reduction in FLOPs for both Swin-B and DeiT-B with only a marginal degradation of accuracy on ImageNet. Our code will be released upon paper acceptance.

We present numerical simulations of planetary systems in star clusters with different initial stellar densities, to investigate the impact of the density on debris disc dynamics. We use LPS+ to combine N-body codes NBODY6++GPU and REBOUND for simulations. We simulate debris discs with and without a Jupiter-mass planet at 50 au, in star clusters with N = 1k - 64k stars. The spatial range of the remaining planetary systems decreases with increasing N. As cluster density increases, the planet's influence range first increases and then decreases. For debris particles escaping from planetary systems, the probability of their direct ejection from the star cluster decreases as their initial semi-major axis (a0) or the cluster density increases. The eccentricity and inclination of surviving particles increase as cluster density increases. The presence of a planet leads to lower eccentricities and inclinations of surviving particles. The radial density distribution of the remaining discs decays exponentially in sparse clusters. We derive a general expression of the gravitational encounter rate. Our results are unable to directly explain the scarcity of debris discs in star clusters. Nevertheless, given that many planetary systems have multiple planets, the mechanism of the planet-cluster combined gravitational influence on the disc remains appealing as a potential explanation., Comment: 15 pages, 15 figures, and 4 tables. Accepted for publication in MNRAS

Object detectors have shown outstanding performance on various public datasets. However, annotating a new dataset for a new task is usually unavoidable in real, since 1) a single existing dataset usually does not contain all object categories needed; 2) using multiple datasets usually suffers from annotation incompletion and heterogeneous features. We propose a novel problem as "Annotation-incomplete Multi-dataset Detection", and develop an end-to-end multi-task learning architecture which can accurately detect all the object categories with multiple partially annotated datasets. Specifically, we propose an attention feature extractor which helps to mine the relations among different datasets. Besides, a knowledge amalgamation training strategy is incorporated to accommodate heterogeneous features from different sources. Extensive experiments on different object detection datasets demonstrate the effectiveness of our methods and an improvement of 2.17%, 2.10% in mAP can be achieved on COCO and VOC respectively., Comment: 12 pages, 9 figures

Although diffusion-based image virtual try-on has made considerable progress, emerging approaches still struggle to effectively address the issue of hand occlusion (i.e., clothing regions occluded by the hand part), leading to a notable degradation of the try-on performance. To tackle this issue widely existing in real-world scenarios, we propose VTON-HandFit, leveraging the power of hand priors to reconstruct the appearance and structure for hand occlusion cases. Firstly, we tailor a Handpose Aggregation Net using the ControlNet-based structure explicitly and adaptively encoding the global hand and pose priors. Besides, to fully exploit the hand-related structure and appearance information, we propose Hand-feature Disentanglement Embedding module to disentangle the hand priors into the hand structure-parametric and visual-appearance features, and customize a masked cross attention for further decoupled feature embedding. Lastly, we customize a hand-canny constraint loss to better learn the structure edge knowledge from the hand template of model image. VTON-HandFit outperforms the baselines in qualitative and quantitative evaluations on the public dataset and our self-collected hand-occlusion Handfit-3K dataset particularly for the arbitrary hand pose occlusion cases in real-world scenarios. The Code and dataset will be available at \url{https://github.com/VTON-HandFit/VTON-HandFit}., Comment: The project page is \url{https://vton-handfit.github.io}

Wireless communications are significantly impacted by the propagation environment, particularly in doubly selective channels with variations in both time and frequency domains. Orthogonal Time Frequency Space (OTFS) modulation has emerged as a promising solution; however, its high equalization complexity, if performed in the delay-Doppler domain, limits its universal application. This article explores domain-adaptive system design, with an emphasis on adaptive equalization, while also discussing modulation and pilot placement strategies. It investigates the dynamic selection of best-fit domains based on channel conditions to enhance performance across diverse environments. We examine channel domain connections, signal designs, and equalization techniques with domain adaptivity, and highlight future research opportunities., Comment: Magazine article, 7 pages, 4 figures, 2 tables

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.

Post-training KV-Cache compression methods typically either sample a subset of effectual tokens or quantize the data into lower numerical bit width. However, these methods cannot exploit redundancy in the hidden dimension of the KV tensors. This paper presents a hidden dimension compression approach called Palu, a KV-Cache compression framework that utilizes low-rank projection to reduce inference-time LLM memory usage. Palu decomposes the linear layers into low-rank matrices, caches compressed intermediate states, and reconstructs the full keys and values on the fly. To improve accuracy, compression rate, and efficiency, Palu further encompasses (1) a medium-grained low-rank decomposition scheme, (2) an efficient rank search algorithm, (3) low-rank-aware quantization compatibility enhancements, and (4) optimized GPU kernels with operators fusion. Extensive experiments with popular LLMs show that Palu compresses KV-Cache by 50% while maintaining strong accuracy and delivering up to 1.89x on the RoPE-based attention module. When combined with quantization, Palu's inherent quantization-friendly design yields small to negligible extra accuracy degradation while saving additional memory than quantization-only methods and achieving up to 2.91x speedup for the RoPE-based attention. Moreover, it maintains comparable or even better accuracy (up to 1.19 lower perplexity) compared to quantization-only methods. These results demonstrate Palu's superior capability to effectively address the efficiency and memory challenges of LLM inference posed by KV-Cache. Our code is publicly available at: https://github.com/shadowpa0327/Palu

The field of evolutionary many-task optimization (EMaTO) is increasingly recognized for its ability to streamline the resolution of optimization challenges with repetitive characteristics, thereby conserving computational resources. This paper tackles the challenge of crafting efficient knowledge transfer mechanisms within EMaTO, a task complicated by the computational demands of individual task evaluations. We introduce a novel framework that employs a complex network to comprehensively analyze the dynamics of knowledge transfer between tasks within EMaTO. By extracting and scrutinizing the knowledge transfer network from existing EMaTO algorithms, we evaluate the influence of network modifications on overall algorithmic efficacy. Our findings indicate that these networks are diverse, displaying community-structured directed graph characteristics, with their network density adapting to different task sets. This research underscores the viability of integrating complex network concepts into EMaTO to refine knowledge transfer processes, paving the way for future advancements in the domain., Comment: 9 pages, accepted by GECCO 2024 poster

Quantum machine learning has gained considerable attention as quantum technology advances, presenting a promising approach for efficiently learning complex data patterns. Despite this promise, most contemporary quantum methods require significant resources for variational parameter optimization and face issues with vanishing gradients, leading to experiments that are either limited in scale or lack potential for quantum advantage. To address this, we develop a general-purpose, gradient-free, and scalable quantum reservoir learning algorithm that harnesses the quantum dynamics of neutral-atom analog quantum computers to process data. We experimentally implement the algorithm, achieving competitive performance across various categories of machine learning tasks, including binary and multi-class classification, as well as timeseries prediction. Effective and improving learning is observed with increasing system sizes of up to 108 qubits, demonstrating the largest quantum machine learning experiment to date. We further observe comparative quantum kernel advantage in learning tasks by constructing synthetic datasets based on the geometric differences between generated quantum and classical data kernels. Our findings demonstrate the potential of utilizing classically intractable quantum correlations for effective machine learning. We expect these results to stimulate further extensions to different quantum hardware and machine learning paradigms, including early fault-tolerant hardware and generative machine learning tasks., Comment: 10 + 14 pages, 4 + 7 figures

The interaction of excitons with lattice vibrations underlies the scattering from bright to dark excitons as well as the coherent modulation of the exciton energy. Unlike the former mechanism, which involves phonons with finite momentum, the latter can be exclusively attributed to {\it coherent phonons} with zero momentum. We here lay down the microscopic theory of coherent phonons interacting with resonantly pumped bright excitons and provide the explicit expression of the corresponding coupling. The coupling notably resembles the exciton-phonon one, but with a crucial distinction: it contains the bare electron-phonon matrix elements rather than the screened ones. Our theory predicts that the exciton energy features a polaronic-like red-shift and monochromatic oscillations or beatings, depending on the number of coupled optical modes. Both the red-shift and the amplitude of the oscillations are proportional to the excitation density and to the square of the exciton-coherent-phonon coupling. We validate our analytical findings through comparisons with numerical simulations of time-resolved optical absorbance in resonantly pumped MoS$_{2}$ monolayers.

Recent advances in diffusion-based text-to-image models have simplified creating high-fidelity images, but preserving the identity (ID) of specific elements, like a personal dog, is still challenging. Object customization, using reference images and textual descriptions, is key to addressing this issue. Current object customization methods are either object-specific, requiring extensive fine-tuning, or object-agnostic, offering zero-shot customization but limited to specialized domains. The primary issue of promoting zero-shot object customization from specific domains to the general domain is to establish a large-scale general ID dataset for model pre-training, which is time-consuming and labor-intensive. In this paper, we propose a novel pipeline to construct a large dataset of general objects and build the Multi-Category ID-Consistent (MC-IDC) dataset, featuring 315k text-image samples across 10k categories. With the help of MC-IDC, we introduce Customizing Anything (CustAny), a zero-shot framework that maintains ID fidelity and supports flexible text editing for general objects. CustAny features three key components: a general ID extraction module, a dual-level ID injection module, and an ID-aware decoupling module, allowing it to customize any object from a single reference image and text prompt. Experiments demonstrate that CustAny outperforms existing methods in both general object customization and specialized domains like human customization and virtual try-on. Our contributions include a large-scale dataset, the CustAny framework and novel ID processing to advance this field. Code and dataset will be released soon in https://github.com/LingjieKong-fdu/CustAny.

Multipartite entanglement is one of the crucial resources in quantum information processing tasks such as quantum metrology, quantum computing and quantum communications. It is essential to verify not only the multipartite entanglement, but also the entanglement structure in both fundamental theories and the applications of quantum information technologies. However, it is proved to be challenging to detect the entanglement structures, including entanglement depth, entanglement intactness and entanglement stretchability, especially for general states and large-scale quantum systems. By using the partitions of the tensor product space we propose a systematic method to construct powerful entanglement witnesses which identify better the multipartite entanglement structures. Besides, an efficient algorithm using semi-definite programming and a gradient descent algorithm are designed to detect entanglement structure from the inner polytope of the convex set containing all the states with the same entanglement structure. We demonstrate by detailed examples that our criteria perform better than other known ones. Our results may be applied to many quantum information processing tasks.

In this paper, we prove that the oscillation operator, variation operator and maximal differential transform associated with the approximate identities are bounded from ${\rm BMO}({\mathbb R}^n)$ to its subspace ${\rm BLO}({\mathbb R}^n)$.

Multimodal large language models (MLLMs) contribute a powerful mechanism to understanding visual information building on large language models. However, MLLMs are notorious for suffering from hallucinations, especially when generating lengthy, detailed descriptions for images. Our analysis reveals that hallucinations stem from the inherent summarization mechanism of large language models, leading to excessive dependence on linguistic tokens while neglecting vision information. In this paper, we propose NoiseBoost, a broadly applicable and simple method for alleviating hallucinations for MLLMs through the integration of noise feature perturbations. Noise perturbation acts as a regularizer, facilitating a balanced distribution of attention weights among visual and linguistic tokens. Despite its simplicity, NoiseBoost consistently enhances the performance of MLLMs across common training strategies, including supervised fine-tuning and reinforcement learning. Further, NoiseBoost pioneerly enables semi-supervised learning for MLLMs, unleashing the power of unlabeled data. Comprehensive experiments demonstrate that NoiseBoost improves dense caption accuracy by 8.1% with human evaluation and achieves comparable results with 50% of the data by mining unlabeled data. Code and models are available at https://kaiwu5.github.io/noiseboost., Comment: 14 pages, 5 figures with supplementary material

Class-imbalanced node classification tasks are prevalent in real-world scenarios. Due to the uneven distribution of nodes across different classes, learning high-quality node representations remains a challenging endeavor. The engineering of loss functions has shown promising potential in addressing this issue. It involves the meticulous design of loss functions, utilizing information about the quantities of nodes in different categories and the network's topology to learn unbiased node representations. However, the design of these loss functions heavily relies on human expert knowledge and exhibits limited adaptability to specific target tasks. In this paper, we introduce a high-performance, flexible, and generalizable automated loss function search framework to tackle this challenge. Across 15 combinations of graph neural networks and datasets, our framework achieves a significant improvement in performance compared to state-of-the-art methods. Additionally, we observe that homophily in graph-structured data significantly contributes to the transferability of the proposed framework., Comment: ICML 2024

In the past year, Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance in tasks such as visual question answering, visual understanding and reasoning. However, the extensive model size and high training and inference costs have hindered the widespread application of MLLMs in academia and industry. Thus, studying efficient and lightweight MLLMs has enormous potential, especially in edge computing scenarios. In this survey, we provide a comprehensive and systematic review of the current state of efficient MLLMs. Specifically, we summarize the timeline of representative efficient MLLMs, research state of efficient structures and strategies, and the applications. Finally, we discuss the limitations of current efficient MLLM research and promising future directions. Please refer to our GitHub repository for more details: https://github.com/lijiannuist/Efficient-Multimodal-LLMs-Survey.

Zero-shot optimization involves optimizing a target task that was not seen during training, aiming to provide the optimal solution without or with minimal adjustments to the optimizer. It is crucial to ensure reliable and robust performance in various applications. Current optimizers often struggle with zero-shot optimization and require intricate hyperparameter tuning to adapt to new tasks. To address this, we propose a Pretrained Optimization Model (POM) that leverages knowledge gained from optimizing diverse tasks, offering efficient solutions to zero-shot optimization through direct application or fine-tuning with few-shot samples. Evaluation on the BBOB benchmark and two robot control tasks demonstrates that POM outperforms state-of-the-art black-box optimization methods, especially for high-dimensional tasks. Fine-tuning POM with a small number of samples and budget yields significant performance improvements. Moreover, POM demonstrates robust generalization across diverse task distributions, dimensions, population sizes, and optimization horizons. For code implementation, see https://github.com/ninja-wm/POM/.

In additive manufacturing, the fabrication sequence has a large influence on the quality of manufactured components. While planning of the fabrication sequence is typically performed after the component has been designed, recent developments have demonstrated the possibility and benefits of simultaneous optimization of both the structural layout and the corresponding fabrication sequence. The simultaneous optimization approach, called space-time topology optimization, introduces a pseudo-time field to encode the manufacturing process order, alongside a pseudo-density field representing the structural layout. To comply with manufacturing principles, the pseudo-time field needs to be monotonic, i.e., free of local minima. However, explicitly formulated constraints are not always effective, particularly for complex structural layouts. In this paper, we introduce a novel method to regularize the pseudo-time field in space-time topology optimization. We conceptualize the monotonic additive manufacturing process as a virtual heat conduction process starting from the surface upon which a component is constructed layer by layer. The virtual temperature field, which shall not be confused with the actual temperature field during manufacturing, serves as an analogy for encoding the fabrication sequence. In this new formulation, we use local virtual heat conductivity coefficients as optimization variables to steer the temperature field and, consequently, the fabrication sequence. The virtual temperature field is inherently free of local minima due to the physics it resembles. We numerically validate the effectiveness of this regularization in space-time topology optimization under process-dependent loads, including gravity and thermomechanical loads., Comment: 23 pages, 14 figures

Despite significant advancements in image customization with diffusion models, current methods still have several limitations: 1) unintended changes in non-target areas when regenerating the entire image; 2) guidance solely by a reference image or text descriptions; and 3) time-consuming fine-tuning, which limits their practical application. In response, we introduce a tuning-free framework for simultaneous text-image-guided image customization, enabling precise editing of specific image regions within seconds. Our approach preserves the semantic features of the reference image subject while allowing modification of detailed attributes based on text descriptions. To achieve this, we propose an innovative attention blending strategy that blends self-attention features in the UNet decoder during the denoising process. To our knowledge, this is the first tuning-free method that concurrently utilizes text and image guidance for image customization in specific regions. Our approach outperforms previous methods in both human and quantitative evaluations, providing an efficient solution for various practical applications, such as image synthesis, design, and creative photography., Comment: 17 pages, 8 figures

Deep learning-based image matching methods play a crucial role in computer vision, yet they often suffer from substantial computational demands. To tackle this challenge, we present HCPM, an efficient and detector-free local feature-matching method that employs hierarchical pruning to optimize the matching pipeline. In contrast to recent detector-free methods that depend on an exhaustive set of coarse-level candidates for matching, HCPM selectively concentrates on a concise subset of informative candidates, resulting in fewer computational candidates and enhanced matching efficiency. The method comprises a self-pruning stage for selecting reliable candidates and an interactive-pruning stage that identifies correlated patches at the coarse level. Our results reveal that HCPM significantly surpasses existing methods in terms of speed while maintaining high accuracy. The source code will be made available upon publication.