Search

Information engines produce mechanical work through measurement and adaptive control. For information engines, the principal challenge lies in how to store the generated work for subsequent utilization. Here, we report an experimental demonstration where quantized mechanical motion serves as a quantum battery and gets charged in repeated cycles by a single trapped-ion information engine. This is enabled by a key technological advancement in rapid state discrimination, allowing us to suppress measurement-induced disturbances. Consequently, we were able to obtain a charging efficiency over 50\% of the theoretical limit at the optimal temperature. The experimental results substantiate that this approach can render trapped ions a promising platform for microscopic information engines with potential applications in the future upon scaling up.

In response to the challenges posed by the extensive parameter updates required for full fine-tuning of large-scale pre-trained models, parameter-efficient fine-tuning (PEFT) methods, exemplified by Low-Rank Adaptation (LoRA), have emerged. LoRA simplifies the fine-tuning process but may still struggle with a certain level of redundancy in low-rank matrices and limited effectiveness from merely increasing their rank. To address these issues, a natural idea is to enhance the independence and diversity of the learning process for the low-rank matrices. Therefore, we propose Masked LoRA Experts (MLAE), an innovative approach that applies the concept of masking to visual PEFT. Our method incorporates a cellular decomposition strategy that transforms a low-rank matrix into independent rank-1 submatrices, or "experts", thus enhancing independence. Additionally, we introduce a binary mask matrix that selectively activates these experts during training to promote more diverse and anisotropic learning, based on expert-level dropout strategies. Our investigations reveal that this selective activation not only enhances performance but also fosters a more diverse acquisition of knowledge with a marked decrease in parameter similarity among MLAE, significantly boosting the quality of the model. Remarkably, MLAE achieves new state-of-the-art (SOTA) performance with an average accuracy score of 78.8% on the VTAB-1k benchmark and 90.9% on the FGVC benchmark, surpassing the previous SOTA result by an average of 0.8% on both benchmarks with approximately half parameters. Our code is available at https://github.com/jie040109/MLAE., Comment: Tech report

Text-to-image generation models have seen considerable advancement, catering to the increasing interest in personalized image creation. Current customization techniques often necessitate users to provide multiple images (typically 3-5) for each customized object, along with the classification of these objects and descriptive textual prompts for scenes. This paper questions whether the process can be made more user-friendly and the customization more intricate. We propose a method where users need only provide images along with text for each customization topic, and necessitates only a single image per visual concept. We introduce the concept of a ``multi-modal prompt'', a novel integration of text and images tailored to each customization concept, which simplifies user interaction and facilitates precise customization of both objects and scenes. Our proposed paradigm for customized text-to-image generation surpasses existing finetune-based methods in user-friendliness and the ability to customize complex objects with user-friendly inputs. Our code is available at $\href{https://github.com/zhongzero/Multi-Modal-Prompt}{https://github.com/zhongzero/Multi-Modal-Prompt}$., Comment: 11 pages, 8 figures

Dataset distillation, a pragmatic approach in machine learning, aims to create a smaller synthetic dataset from a larger existing dataset. However, existing distillation methods primarily adopt a model-based paradigm, where the synthetic dataset inherits model-specific biases, limiting its generalizability to alternative models. In response to this constraint, we propose a novel methodology termed "model pool". This approach involves selecting models from a diverse model pool based on a specific probability distribution during the data distillation process. Additionally, we integrate our model pool with the established knowledge distillation approach and apply knowledge distillation to the test process of the distilled dataset. Our experimental results validate the effectiveness of the model pool approach across a range of existing models while testing, demonstrating superior performance compared to existing methodologies.

[$^{18}$F]fluorodeoxyglucose (FDG) positron emission tomography (PET) has emerged as a crucial tool in identifying the epileptic focus, especially in cases where magnetic resonance imaging (MRI) diagnosis yields indeterminate results. FDG PET can provide the metabolic information of glucose and help identify abnormal areas that are not easily found through MRI. However, the effectiveness of FDG PET-based assessment and diagnosis depends on the selection of a healthy control group. The healthy control group typically consists of healthy individuals similar to epilepsy patients in terms of age, gender, and other aspects for providing normal FDG PET data, which will be used as a reference for enhancing the accuracy and reliability of the epilepsy diagnosis. However, significant challenges arise when a healthy PET control group is unattainable. Yaakub \emph{et al.} have previously introduced a Pix2PixGAN-based method for MRI to PET translation. This method used paired MRI and FDG PET scans from healthy individuals for training, and produced pseudo normal FDG PET images from patient MRIs that are subsequently used for lesion detection. However, this approach requires a large amount of high-quality, paired MRI and PET images from healthy control subjects, which may not always be available. In this study, we investigated unsupervised learning methods for unpaired MRI to PET translation for generating pseudo normal FDG PET for epileptic focus localization. Two deep learning methods, CycleGAN and SynDiff, were employed, and we found that diffusion-based method achieved improved performance in accurately localizing the epileptic focus., Comment: SPIE Medical Imaging 2024

Image-to-image translation is a common task in computer vision and has been rapidly increasing the impact on the field of medical imaging. Deep learning-based methods that employ conditional generative adversarial networks (cGANs), such as Pix2PixGAN, have been extensively explored to perform image-to-image translation tasks. However, when noisy medical image data are considered, such methods cannot be directly applied to produce clean images. Recently, an augmented GAN architecture named AmbientGAN has been proposed that can be trained on noisy measurement data to synthesize high-quality clean medical images. Inspired by AmbientGAN, in this work, we propose a new cGAN architecture, Ambient-Pix2PixGAN, for performing medical image-to-image translation tasks by use of noisy measurement data. Numerical studies that consider MRI-to-PET translation are conducted. Both traditional image quality metrics and task-based image quality metrics are employed to assess the proposed Ambient-Pix2PixGAN. It is demonstrated that our proposed Ambient-Pix2PixGAN can be successfully trained on noisy measurement data to produce high-quality translated images in target imaging modality., Comment: SPIE Medical Imaging 2024

Medical imaging systems that are designed for producing diagnostically informative images should be objectively assessed via task-based measures of image quality (IQ). Ideally, computation of task-based measures of IQ needs to account for all sources of randomness in the measurement data, including the variability in the ensemble of objects to be imaged. To address this need, stochastic object models (SOMs) that can generate an ensemble of synthesized objects or phantoms can be employed. Various mathematical SOMs or phantoms were developed that can interpretably synthesize objects, such as lumpy object models and parameterized torso phantoms. However, such SOMs that are purely mathematically defined may not be able to comprehensively capture realistic object variations. To establish realistic SOMs, it is desirable to use experimental data. An augmented generative adversarial network (GAN), AmbientGAN, was recently proposed for establishing SOMs from medical imaging measurements. However, it remains unclear to which extent the AmbientGAN-produced objects can be interpretably controlled. This work introduces a novel approach called AmbientCycleGAN that translates mathematical SOMs to realistic SOMs by use of noisy measurement data. Numerical studies that consider clustered lumpy background (CLB) models and real mammograms are conducted. It is demonstrated that our proposed method can stably establish SOMs based on mathematical models and noisy measurement data. Moreover, the ability of the proposed AmbientCycleGAN to interpretably control image features in the synthesized objects is investigated., Comment: SPIE Medical Imaging 2024

While modern biotechnologies allow synthesizing new proteins and function measurements at scale, efficiently exploring a protein sequence space and engineering it remains a daunting task due to the vast sequence space of any given protein. Protein engineering is typically conducted through an iterative process of adding mutations to the wild-type or lead sequences, recombination of mutations, and running new rounds of screening. To enhance the efficiency of such a process, we propose a tree search-based bandit learning method, which expands a tree starting from the initial sequence with the guidance of a bandit machine learning model. Under simplified assumptions and a Gaussian Process prior, we provide theoretical analysis and a Bayesian regret bound, demonstrating that the combination of local search and bandit learning method can efficiently discover a near-optimal design. The full algorithm is compatible with a suite of randomized tree search heuristics, machine learning models, pre-trained embeddings, and bandit techniques. We test various instances of the algorithm across benchmark protein datasets using simulated screens. Experiment results demonstrate that the algorithm is both sample-efficient and able to find top designs using reasonably small mutation counts., Comment: AAAI 2024

Multi-modal Large Language Models (MLLMs) are increasingly prominent in the field of artificial intelligence. These models not only excel in traditional vision-language tasks but also demonstrate impressive performance in contemporary multi-modal benchmarks. Although many of these benchmarks attempt to holistically evaluate MLLMs, they typically concentrate on basic reasoning tasks, often yielding only simple yes/no or multi-choice responses. These methods naturally lead to confusion and difficulties in conclusively determining the reasoning capabilities of MLLMs. To mitigate this issue, we manually curate a benchmark dataset specifically designed for MLLMs, with a focus on complex reasoning tasks. Our benchmark comprises three key reasoning categories: deductive, abductive, and analogical reasoning. The queries in our dataset are intentionally constructed to engage the reasoning capabilities of MLLMs in the process of generating answers. For a fair comparison across various MLLMs, we incorporate intermediate reasoning steps into our evaluation criteria. In instances where an MLLM is unable to produce a definitive answer, its reasoning ability is evaluated by requesting intermediate reasoning steps. If these steps align with our manual annotations, appropriate scores are assigned. This evaluation scheme resembles methods commonly used in human assessments, such as exams or assignments, and represents what we consider a more effective assessment technique compared with existing benchmarks. We evaluate a selection of representative MLLMs using this rigorously developed open-ended multi-step elaborate reasoning benchmark, designed to challenge and accurately measure their reasoning capabilities. The code and data will be released at https://infimm.github.io/InfiMM-Eval/

A promising paradigm of quantum computing for achieving practical quantum advantages is quantum annealing or quantum approximate optimization algorithm, where the classical problems are encoded in Ising interactions. However, it is challenging to build a quantum system that can efficiently map any structured problems. Here, we present a programmable trapped-ion quantum simulator of an Ising model with all-to-all connectivity with up to four spins. We implement the spin-spin interactions by using the coupling of trapped ions to multiple collective motional modes and realize the programmability through phase modulation of the Raman laser beams that are individually addressed on ions. As an example, we realize several Ising lattices with different interaction connectivities, where the interactions can be ferromagnetic or anti-ferromagnetic. We confirm the programmed interaction geometry by observing the ground states of the corresponding models through quantum state tomography. Our experimental demonstrations serve as an important basis for realizing practical quantum advantages with trapped ions., Comment: 11 pages, 7 figures

Few-shot learning is a challenging problem since only a few examples are provided to recognize a new class. Several recent studies exploit additional semantic information, e.g. text embeddings of class names, to address the issue of rare samples through combining semantic prototypes with visual prototypes. However, these methods still suffer from the spurious visual features learned from the rare support samples, resulting in limited benefits. In this paper, we propose a novel Semantic Prompt (SP) approach for few-shot learning. Instead of the naive exploitation of semantic information for remedying classifiers, we explore leveraging semantic information as prompts to tune the visual feature extraction network adaptively. Specifically, we design two complementary mechanisms to insert semantic prompts into the feature extractor: one is to enable the interaction between semantic prompts and patch embeddings along the spatial dimension via self-attention, another is to supplement visual features with the transformed semantic prompts along the channel dimension. By combining these two mechanisms, the feature extractor presents a better ability to attend to the class-specific features and obtains more generalized image representations with merely a few support samples. Through extensive experiments on four datasets, the proposed approach achieves promising results, improving the 1-shot learning accuracy by 3.67% on average., Comment: Accepted by CVPR 2023

Segmentation of COVID-19 lesions can assist physicians in better diagnosis and treatment of COVID-19. However, there are few relevant studies due to the lack of detailed information and high-quality annotation in the COVID-19 dataset. To solve the above problem, we propose C2FVL, a Coarse-to-Fine segmentation framework via Vision-Language alignment to merge text information containing the number of lesions and specific locations of image information. The introduction of text information allows the network to achieve better prediction results on challenging datasets. We conduct extensive experiments on two COVID-19 datasets including chest X-ray and CT, and the results demonstrate that our proposed method outperforms other state-of-the-art segmentation methods., Comment: Accepted by ICASSP 2023

Human skin can accurately sense the self-decoupled normal and shear forces when in contact with objects of different sizes. Although there exist many soft and conformable tactile sensors on robotic applications able to decouple the normal force and shear forces, the impact of the size of object in contact on the force calibration model has been commonly ignored. Here, using the principle that contact force can be derived from the light power loss in the soft optical fiber core, we present a soft tactile sensor that decouples normal and shear forces and calibrates the measurement results based on the object size, by designing a two-layered weaved polymer-based optical fiber anisotropic structure embedded in a soft elastomer. Based on the anisotropic response of optical fibers, we developed a linear calibration algorithm to simultaneously measure the size of the contact object and the decoupled normal and shear forces calibrated the object size. By calibrating the sensor at the robotic arm tip, we show that robots can reconstruct the force vector at an average accuracy of 0.15N for normal forces, 0.17N for shear forces in X-axis , and 0.18N for shear forces in Y-axis, within the sensing range of 0-2N in all directions, and the average accuracy of object size measurement of 0.4mm, within the test indenter diameter range of 5-12mm., Comment: submitted to IROS 2023

Quantum error mitigation has been extensively explored to increase the accuracy of the quantum circuits in noisy-intermediate-scale-quantum (NISQ) computation, where quantum error correction requiring additional quantum resources is not adopted. Among various error-mitigation schemes, probabilistic error cancellation (PEC) has been proposed as a general and systematic protocol that can be applied to numerous hardware platforms and quantum algorithms. However, PEC has only been tested in two-qubit systems and a superconducting multi-qubit system by learning a sparse error model. Here, we benchmark PEC using up to four trapped-ion qubits. For the benchmark, we simulate the dynamics of interacting fermions with or without spins by applying multiple Trotter steps. By tomographically reconstructing the error model and incorporating other mitigation methods such as positive probability and symmetry constraints, we are able to increase the fidelity of simulation and faithfully observe the dynamics of the Fermi-Hubbard model, including the different behavior of charge and spin of fermions. Our demonstrations can be an essential step for further extending systematic error-mitigation schemes toward practical quantum advantages., Comment: 15 pages, 11 figures

As a major component of online crime, email-based fraud is a threat that causes substantial economic losses every year. To counteract these scammers, volunteers called scam-baiters play the roles of victims, reply to scammers, and try to waste their time and attention with long and unproductive conversations. To curb email fraud and magnify the effectiveness of scam-baiting, we developed and deployed an expandable scam-baiting mailserver that can conduct scam-baiting activities automatically. We implemented three reply strategies using three different models and conducted a one-month-long experiment during which we elicited 150 messages from 130 different scammers. We compare the performance of each strategy at attracting and holding the attention of scammers, finding tradeoffs between human-written and automatically-generated response strategies. We also demonstrate that scammers can be engaged concurrently by multiple servers deploying these strategies in a second experiment, which used two server instances to contact 92 different scammers over 12 days. We release both our platform and a dataset containing conversations between our automatic scam-baiters and real human scammers, to support future work in preventing online fraud.

Few-shot learning (FSL) aims to recognize novel queries with only a few support samples through leveraging prior knowledge from a base dataset. In this paper, we consider the domain shift problem in FSL and aim to address the domain gap between the support set and the query set. Different from previous cross-domain FSL work (CD-FSL) that considers the domain shift between base and novel classes, the new problem, termed cross-domain cross-set FSL (CDSC-FSL), requires few-shot learners not only to adapt to the new domain, but also to be consistent between different domains within each novel class. To this end, we propose a novel approach, namely stabPA, to learn prototypical compact and cross-domain aligned representations, so that the domain shift and few-shot learning can be addressed simultaneously. We evaluate our approach on two new CDCS-FSL benchmarks built from the DomainNet and Office-Home datasets respectively. Remarkably, our approach outperforms multiple elaborated baselines by a large margin, e.g., improving 5-shot accuracy by 6.0 points on average on DomainNet. Code is available at https://github.com/WentaoChen0813/CDCS-FSL, Comment: Accepted by ECCV 2022

Controllable bosonic systems can provide post-classical computational power with sub-universal quantum computational capability. A network that consists of a number of bosons evolving through beam-splitters and phase-shifters between different modes, has been proposed and applied to demonstrate quantum advantages. While the network has been implemented mostly in optical systems with photons, recently alternative realizations have been explored, where major limitations in photonic systems such as photon loss, and probabilistic manipulation can be addressed. Phonons, the quantized excitations of vibrational modes, of trapped ions can be a promising candidate to realize the bosonic network. Here, we experimentally demonstrate a minimal-loss phononic network that can be programmed and in which any phononic states are deterministically prepared and detected. We realize the network with up to four collective-vibrational modes, which can be straightforwardly extended to reveal quantum advantage. We benchmark the performance of the network with an exemplary algorithm of tomography for arbitrary multi-mode states with a fixed total phonon number. We obtain reconstruction fidelities of 94.5 $\pm$ 1.95 % and 93.4 $\pm$ 3.15 % for single-phonon and two-phonon states, respectively. Our experiment demonstrates a clear and novel pathway to scale up a phononic network for various quantum information processing beyond the limitations of classical and other quantum systems.

It is challenging to obtain quantum statistics of multiple time points due to the principle of quantum mechanics that a measurement disturbs the quantum state. We propose an ancilla-assisted measurement scheme that does not suffer from the measurement-induced back-action and experimentally demonstrate it using dual-species trapped ions. By ensemble averaging the ancilla-measurement outcomes with properly chosen weights, quantum statistics, such as quantum correlation functions and quasi-probability distributions can be reconstructed. We employ $^{171}\rm{Yb}^+$-$^{138}\rm{Ba}^+$ ions as the system and the ancilla to perform multi-time measurements that consist of repeated initialization and detection of the ancilla state without effecting the system state. The two- and three-time quantum correlation functions and quasi-probability distributions are clearly revealed from experimental data. We successfully verify that the marginal distribution is unaffected by the measurement at each time and identify the nonclassicality of the reconstructed distribution. Our scheme can be applied for any $N$-time measurements of a general quantum process, which will be an essential tool for exploring properties of various quantum systems., Comment: 9 pages, 6 figures

Supervised learning has been widely used for attack categorization, requiring high-quality data and labels. However, the data is often imbalanced and it is difficult to obtain sufficient annotations. Moreover, supervised models are subject to real-world deployment issues, such as defending against unseen artificial attacks. To tackle the challenges, we propose a semi-supervised fine-grained attack categorization framework consisting of an encoder and a two-branch structure and this framework can be generalized to different supervised models. The multilayer perceptron with residual connection is used as the encoder to extract features and reduce the complexity. The Recurrent Prototype Module (RPM) is proposed to train the encoder effectively in a semi-supervised manner. To alleviate the data imbalance problem, we introduce the Weight-Task Consistency (WTC) into the iterative process of RPM by assigning larger weights to classes with fewer samples in the loss function. In addition, to cope with new attacks in real-world deployment, we propose an Active Adaption Resampling (AAR) method, which can better discover the distribution of unseen sample data and adapt the parameters of encoder. Experimental results show that our model outperforms the state-of-the-art semi-supervised attack detection methods with a 3% improvement in classification accuracy and a 90% reduction in training time., Comment: Tech report

Two-dimensional (2D) quantum magnetism is a paradigm in strongly correlated many-body physics. The understanding of 2D quantum magnetism can be expedited by employing a controllable quantum simulator that faithfully maps 2D-spin Hamiltonians. The 2D quantum simulators can exhibit exotic phenomena such as frustrated quantum magnetism and topological order and can be used to show quantum computational advantages. Many experimental platforms are being developed, including Rydberg atoms and superconducting annealers. However, with trapped-ion systems, which showed the most advanced controllability and quantum coherence, quantum magnetism was explored in one-dimensional chains. Here, we report simulations of frustrated quantum magnetism with 2D ion crystals. We create a variety of spin-spin interactions for quantum magnets, including those that exhibit frustration by driving different vibrational modes and adiabatically prepare the corresponding ground states. The experimentally measured ground states are consistent with the theoretical predictions and are highly degenerate for geometrically frustrated spin models in two dimensions. Quantum coherence of the ground states is probed by reversing the time evolution of the B-field to the initial value and then measuring the extent to which the remaining state coincides with the initial state. Our results open the door for quantum simulations with 2D ion crystals., Comment: 11 pages, 9 figures