Search

Evaluating image captions typically relies on reference captions, which are costly to obtain and exhibit significant diversity and subjectivity. While reference-free evaluation metrics have been proposed, most focus on cross-modal evaluation between captions and images. Recent research has revealed that the modality gap generally exists in the representation of contrastive learning-based multi-modal systems, undermining the reliability of cross-modality metrics like CLIPScore. In this paper, we propose CAMScore, a cyclic reference-free automatic evaluation metric for image captioning models. To circumvent the aforementioned modality gap, CAMScore utilizes a text-to-image model to generate images from captions and subsequently evaluates these generated images against the original images. Furthermore, to provide fine-grained information for a more comprehensive evaluation, we design a three-level evaluation framework for CAMScore that encompasses pixel-level, semantic-level, and objective-level perspectives. Extensive experiment results across multiple benchmark datasets show that CAMScore achieves a superior correlation with human judgments compared to existing reference-based and reference-free metrics, demonstrating the effectiveness of the framework.

This study addresses the issue of leveraging federated learning to improve data privacy and performance in IVF embryo selection. The EM (Expectation-Maximization) algorithm is incorporated into deep learning models to form a federated learning framework for quality evaluation of blastomere cleavage using two-dimensional images. The framework comprises a server site and several client sites characterized in that each is locally trained with an EM algorithm. Upon the completion of the local EM training, a separate 5-mode mixture distribution is generated for each client, the clients' distribution statics are then uploaded to the server site and aggregated therein to produce a global (sharing) 5-mode distribution. During the inference phase, each client uses image classifiers and an instance segmentor, assisted by the global 5-mode distribution acting as a calibrator to (1) identify the absolute cleavage timing of blastomere, i.e., tPNa, tPNf, t2, t3, t4, t5, t6, t7, and t8, (2) track the cleavage process of blastomeres to detect the irregular cleavage patterns, and (3) assess the symmetry degree of blastomeres. Experimental results show that the proposed method outperforms commercial Time-Lapse Incubators in reducing the average error of timing prediction by twofold. The proposed facilitate frameworks the adaptability and scalability of classifiers and segmentor to data variability associated with patients in different locations or countries., Comment: 6 pages, 8 figures. Accepted by the 1st Workshop on Federated Learning for Unbounded and Intelligent Decentralization (FLUID). in conjunction with AAAI 2025 in Philadelphia, Pennsylvania, USA

Data collaboration via Data Clean Room offers value but raises privacy concerns, which can be addressed through synthetic data and multi-table synthesizers. Common multi-table synthesizers fail to perform when subjects occur repeatedly in both tables. This is an urgent yet unresolved problem, since having both tables with repeating subjects is common. To improve performance in this scenario, we present the DEREC 3-step pre-processing pipeline to generalize adaptability of multi-table synthesizers. We also introduce the SIMPRO 3-aspect evaluation metrics, which leverage conditional distribution and large-scale simultaneous hypothesis testing to provide comprehensive feedback on synthetic data fidelity at both column and table levels. Results show that using DEREC improves fidelity, and multi-table synthesizers outperform single-table counterparts in collaboration settings. Together, the DEREC-SIMPRO pipeline offers a robust solution for generalizing data collaboration, promoting a more efficient, data-driven society.

In the current era of big data and machine learning, it's essential to find ways to shrink the size of training dataset while preserving the training performance to improve efficiency. However, the challenge behind it includes providing practical ways to find points that can be deleted without significantly harming the training result and suffering from problems like underfitting. We therefore present the perfect deleted point problem for 1-step noisy SGD in the classical linear regression task, which aims to find the perfect deleted point in the training dataset such that the model resulted from the deleted dataset will be identical to the one trained without deleting it. We apply the so-called signal-to-noise ratio and suggest that its value is closely related to the selection of the perfect deleted point. We also implement an algorithm based on this and empirically show the effectiveness of it in a synthetic dataset. Finally we analyze the consequences of the perfect deleted point, specifically how it affects the training performance and privacy budget, therefore highlighting its potential. This research underscores the importance of data deletion and calls for urgent need for more studies in this field.

Adverse weather image restoration aims to remove unwanted degraded artifacts, such as haze, rain, and snow, caused by adverse weather conditions. Existing methods achieve remarkable results for addressing single-weather conditions. However, they face challenges when encountering unpredictable weather conditions, which often happen in real-world scenarios. Although different weather conditions exhibit different degradation patterns, they share common characteristics that are highly related and complementary, such as occlusions caused by degradation patterns, color distortion, and contrast attenuation due to the scattering of atmospheric particles. Therefore, we focus on leveraging common knowledge across multiple weather conditions to restore images in a unified manner. In this paper, we propose a Triplet Attention Network (TANet) to efficiently and effectively address all-in-one adverse weather image restoration. TANet consists of Triplet Attention Block (TAB) that incorporates three types of attention mechanisms: Local Pixel-wise Attention (LPA) and Global Strip-wise Attention (GSA) to address occlusions caused by non-uniform degradation patterns, and Global Distribution Attention (GDA) to address color distortion and contrast attenuation caused by atmospheric phenomena. By leveraging common knowledge shared across different weather conditions, TANet successfully addresses multiple weather conditions in a unified manner. Experimental results show that TANet efficiently and effectively achieves state-of-the-art performance in all-in-one adverse weather image restoration. The source code is available at https://github.com/xhuachris/TANet-ACCV-2024., Comment: 17 pages (ACCV 2024)

In order to overcome the challenge of lacking polarization encoding in integrated quantum photonic circuits, we propose a scheme to realize arbitrary polarization manipulation of a single photon by integrating a single quantum emitter in a photonic waveguide. In our scheme, one transition path of the three-level emitter is designed to simultaneously couples with two orthogonal polarization degenerate modes in the waveguide with adjustable coupling strengths, and the other transition path of the three-level emitter is driven by an external coherent field. The proposed polarization converter has several advantages, including arbitrary polarization conversion for any input polarization, tunable working frequency, excellent anti-dissipation ability with high conversion efficiency, and atomic-scale size. Our work provides an effective solution to enable the polarization encoding of photons which can be applied in the integrated quantum photonic circuits, and will boost quantum photonic chip.

The recent experiment shows the superconducting transition temperature in the Ruddlesden-Popper bilayer La$_3$Ni$_2$O$_{7}$ decreases monotonically with increasing pressure above 14 GPa. In order to unravel the underlying mechanism for this unusual dependence, we performed theoretical investigations by combining the density functional theory (DFT) and the unbiased functional renormalization group (FRG). Our DFT calculations show that the Fermi pockets are essentially unchanged with increasing pressure (above 14 GPa), but the bandwidth is enlarged, and particularly the interlayer hopping integral between the nickel $3d_{3z^2-r^2}$ orbitals is enhanced. From the DFT band structure, we construct the bilayer tight-binding model in terms of the nickel $3d_{3z^2-r^2}$ and $3d_{x^2-y^2}$ orbitals. On this basis, we investigate the superconductivity induced by correlation effects by FRG calculations. We find consistently $s_\pm$-wave pairing triggered by spin fluctuations, but the latter are weakened by pressure and lead to a decreasing transition temperature versus pressure, in qualitatively agreement with the experiment. We emphasize that the itinerancy of the $d$-orbitals is important and captured naturally in our FRG calculations, and we argue that the unusual pressure dependence would be unnatural, if not impossible, in the otherwise local-moment picture of the nickel $d$-orbitals. This sheds lights on the pertinent microscopic description of, and more importantly the mechanism of superconductivity in La$_3$Ni$_2$O$_{7}$., Comment: 8 pages, 4 figures

In this work, we report a pronounced light upconversion in few-layer transition metal dichalcogenides. Our joint theory-experiment study attributes the upconversion photoluminescence to a resonant exciton-exciton annihilation involving a pair of dark excitons with opposite momenta, followed by the spontaneous emission of upconverted bright excitons, which can have a high upconversion efficiency. Additionally, the upconversion photoluminescence is generic in MoS2, MoSe2, WS2, and WSe2, showing a high tuneability from green to ultraviolet light.

Detecting mid-infrared (MIR) radiation has significant astronomical applications, although limited by unsatisfactory MIR detectors. Here we reported on the realization of a MIR up-conversion interferometer based on synthetic long base-line (SLBL) in the laboratory. The experimental system consisted of an interferometer and subsequent up-conversion detection part of mid-infrared signal, which streamlined the structure and enhanced the reliability of the system. By using a tungsten filament lamp as an imitated star, we not only achieved the single target angle resolution of 1.10 times 10^(-4) rad, but also obtained the field angle resolution of 3.0 times 10^(-4) rad of double star targets. The angular resolution is in inverse proportion to the length of baseline. The maximum length of simulated baseline in the laboratory is about 3cm. In a Keck Interferometer (KI) liked program, the base line can reach up to 85m leading to a corresponding angular resolution of 3.0 times 10^(-9) rad (about 1.8mas). The study will offer potential benefits in extending the usage of mid-infrared light in astronomical exploration., Comment: 11 pages, 4 figures. Accepted by Physics Review D

Due to the high noise caused by solar background radiation, the existing satellite-based free-space quantum key distribution (QKD) experiments are mainly carried out at night, hindering the establishment of a practical all-day real-time global-scale quantum network. Given that the 3-5 {\mu}m mid-infrared (MIR) band has extremely low solar background radiation and strong scattering resistance, it is one of the ideal bands for free-space quantum communication. Here, firstly, we report on the preparation of a high-quality MIR (3370 nm) and telecom band (1555 nm) two-color polarization-entangled photon source, then we use this source to realize a principle QKD based on free-space and fiber hybrid channels in a laboratory. The theoretical analysis clearly shows that a long-distance QKD over 500 km of free-space and 96 km of fiber hybrid channels can be reached simultaneously. This work represents a significant step toward developing all-day global-scale quantum communication networks., Comment: 24 pages, 9 figures

Polarimetric imaging, a technique that captures the invisible polarization-related properties of given materials, has broad applications from fundamental physics to advanced fields such as target recognition, stress detection, biomedical diagnosis and remote sensing. The introduction of quantum sources into classical imaging systems has demonstrated distinct advantages, yet few studies have explored their combination with polarimetric imaging. In this study, we present a quantum polarimetric imaging system that integrates polarization-entangled photon pairs into a polarizer-sample-compensator-analyzer (PSRA)-type polarimeter. Our system visualizes the birefringence properties of a periodical-distributed anisotropic material under decreasing illumination levels and diverse disturbing light sources. Compared to the classical system, the quantum approach reveals the superior sensitivity and robustness in low-light conditions, particularly useful in biomedical studies where the low illumination and non-destructive detection are urgently needed. The study also highlights the nonlocality of entangled photons in birefringence measurement, indicating the potential of quantum polarimetric system in the remote sensing domain.

Pair density wave (PDW) is an intriguing quantum matter proposed in the frontier of condensed matter physics. However, the existence of PDW in microscopic models has been rare. In this work, we obtain, by Ginzburg-Landau arguments and self-consistent mean field theory, novel $2a_0\times2a_0$ PDW on the kagome lattice arising from attractive on-bond pairing interactions and the distinct Bloch wave functions near the p-type van Hove singularity. The PDW state carrying three independent wave-vectors, the so-called 3Q PDW, is nodeless and falls into two topological classes characterized by the Chern number $C = 0$ or $C = \pm2$. The chiral ($C=\pm2$) PDW state presents a rare case of interaction driven topological quantum state without the requirement of spin-orbit coupling. Finally, we analyze the stabilities and properties of these PDWs intertwining with charge orders, and discuss the relevance of our minimal model to recent experimental observations in kagome superconductors. Our theory not only elucidates the driving force of the chiral PDW, but also predicts strongly anisotropic superconducting gap structure in the momentum space and quantized transverse thermal conductivity that can be tested in future experiments., Comment: 7 pages, 4 figures

Vortex is a topological defect in the superconducting condensate when a magnetic field is applied to a type-II superconductor, as elucidated by the Ginzburg-Landau theory. Due to the confinement of the quasiparticles by a vortex, it exhibits a circular shaped pattern of bound states with discrete energy levels, as predicted by the Caroli-de Gennes-Matricon theory in 1964. Here, however, we report a completely new type of vortex pattern which is necklace-like in an iron-based superconductor KCa2Fe4As4F2. Our theoretical analysis shows that this necklace-like vortex pattern arises from selective off-shell interference between vortex bound states of opposite angular momenta in the presence of rotational symmetry breaking due to disorders. This fascinating effect can be observed in a system with a small Fermi energy and wave vector, conditions fortuitously met in our samples. Our results not only disclose a novel vortex structure but also provide insights into comprehending the physics of the superconducting condensate., Comment: 29 pages total; 16 pages of main text with 5 figures, 13 pages of supplementary materials with 10 figures

M-type stars are the ones that flare most frequently, but how big their maximum flare energy can reach is still unknown. We present 163 flares from 162 individual M2 through L1-type stars that triggered the GWAC, with flare energies ranging from $10^{32.2}$ to $10^{36.4}$ erg . The flare amplitudes range from $\triangle G = 0.84$ to $\sim 10$ mag. Flare energy increases with stellar surface temperature ($T_{\rm eff}$) but both $\triangle G$ and equivalent duration $\log_{10}(ED)$ seem to be independent of $T_{\rm eff}$. Combining periods detected from light curves of TESS and K2, spectra from LAMOST, SDSS and the 2.16 m Telescope, and the Gaia DR3 data, we found that these GWAC flare stars are young. For the stars that have spectra, we found that these stars are in or very near to the saturation region, and $\log_{10}(L_{\rm H\alpha}/L_{\rm bol})$ is lower for M7-L1 stars than for M2-M6 stars. We also studied the relation between GWAC flare bolometric energy $E_{\rm bol}$ and stellar hemispherical area $S$, and found that $\log_{10}E_{\rm bol}$ (in erg) increases with increasing $S$ (in cm$^2$), and the maximum flare energy $\log_{10}E_{\rm bol, max} \geqslant \log_{10}S + 14.25$. For M7-L1 stars, there seem to be other factors limiting their maximum flare energies in addition to stellar hemispherical area., Comment: 18 pages, 11 figures, 4 tables

The evaluation of synthetic data generation is crucial, especially in the retail sector where data accuracy is paramount. This paper introduces a comprehensive framework for assessing synthetic retail data, focusing on fidelity, utility, and privacy. Our approach differentiates between continuous and discrete data attributes, providing precise evaluation criteria. Fidelity is measured through stability and generalizability. Stability ensures synthetic data accurately replicates known data distributions, while generalizability confirms its robustness in novel scenarios. Utility is demonstrated through the synthetic data's effectiveness in critical retail tasks such as demand forecasting and dynamic pricing, proving its value in predictive analytics and strategic planning. Privacy is safeguarded using Differential Privacy, ensuring synthetic data maintains a perfect balance between resembling training and holdout datasets without compromising security. Our findings validate that this framework provides reliable and scalable evaluation for synthetic retail data. It ensures high fidelity, utility, and privacy, making it an essential tool for advancing retail data science. This framework meets the evolving needs of the retail industry with precision and confidence, paving the way for future advancements in synthetic data methodologies.

The promise of tabular generative models is to produce realistic synthetic data that can be shared and safely used without dangerous leakage of information from the training set. In evaluating these models, a variety of methods have been proposed to measure the tendency to copy data from the training dataset when generating a sample. However, these methods suffer from either not considering data-copying from a privacy threat perspective, not being motivated by recent results in the data-copying literature or being difficult to make compatible with the high dimensional, mixed type nature of tabular data. This paper proposes a new similarity metric and Membership Inference Attack called Data Plagiarism Index (DPI) for tabular data. We show that DPI evaluates a new intuitive definition of data-copying and characterizes the corresponding privacy risk. We show that the data-copying identified by DPI poses both privacy and fairness threats to common, high performing architectures; underscoring the necessity for more sophisticated generative modeling techniques to mitigate this issue.

Isotope effect with a large coefficient $\alpha=-\partial \ln T_c/\partial \ln M$ is usually taken as an evidence of phonon mediated superconductors in the Bardeen-Cooper-Schrieffer (BCS) theory. However, in cuprates which are now widely believed to be strong correlation induced d-wave superconductors, $\alpha$ is experimentally observed to be quite small at optimal doping, but keeps growing up with decreasing $T_c$ upon doping, even after exceeding the BCS value $1/2$. Such an anomalous isotope effect seems to challenge the non-phonon picture and still leave room for the phonon-dominated mechanism. In this work, we show that the anomalous dependence of $\alpha$ on $T_c$ can actually be obtained in spin fluctuation induced d-wave superconductors, by studying the Hubbard model on square lattices with functional renormalization group. We have considered two types of electron-phonon couplings (EPCs). The first type couples to electron densities, including the Holstein, breathing and buckling phonons, called Holstein-like. For all these EPCs, $\alpha$ is negative and drops down towards $-\infty$ with decreasing $T_c$ upon doping. On the opposite, for the other type of Peierls-like EPC coupling to electron hoppings on the nearest bonds, also called Su-Schrieffer-Heeger phonon, $\alpha$ is positive, grows up with decreasing $T_c$ and tends to diverge as $T_c\to0$, in qualitative agreement with the experiments. The difference between these two types of EPCs can be understood by their isotope effects on spin fluctuations. From this study, we conclude that the SSH phonon can explain the anomalous isotope effect in cuprates, although it is not the leading pairing mechanism.

The bound states around a vortex in anisotropic superconductors is a longstanding yet important issue. In this work, we develop a variational theory on the basis of the Andreev approximation to obtain the energy levels and wave functions of the low-energy quantized bound states in superconductors with anisotropic pairing on arbitrary Fermi surface. In the case of circular Fermi surface, the effective Schr\"odinger equation yielding the bound state energies gets back to the theory proposed by Volovik and Kopnin many years ago. Our generalization here enables us to prove the equidistant energy spectrum inside a vortex in a broader class of superconductors. More importantly, we are now able to obtain the wave functions of these bound states by projecting the quasiclassical wave function on the eigenmodes of the effective Schr\"odinger equation, going beyond the quasiclassical Eilenberger results, which, as we find, are sensitive to the scattering rate. For the case of isotropic Fermi surface, the spatial profile of the low-energy local density of states is dominated near the vortex center and elongates along the gap antinode directions, in addition to the ubiquitous Friedel oscillation arising from the quantum inteference neglected in the Eilenberger theory. Moreover, as a consequence of the pairing anisotropy, the quantized wave functions develop a peculiar distribution of winding number, which reduces stepwise towards the vortex center. Our work provides a flexible way to study the vortex bound states in the future.

We present BadGD, a unified theoretical framework that exposes the vulnerabilities of gradient descent algorithms through strategic backdoor attacks. Backdoor attacks involve embedding malicious triggers into a training dataset to disrupt the model's learning process. Our framework introduces three novel constructs: Max RiskWarp Trigger, Max GradWarp Trigger, and Max GradDistWarp Trigger, each designed to exploit specific aspects of gradient descent by distorting empirical risk, deterministic gradients, and stochastic gradients respectively. We rigorously define clean and backdoored datasets and provide mathematical formulations for assessing the distortions caused by these malicious backdoor triggers. By measuring the impact of these triggers on the model training procedure, our framework bridges existing empirical findings with theoretical insights, demonstrating how a malicious party can exploit gradient descent hyperparameters to maximize attack effectiveness. In particular, we show that these exploitations can significantly alter the loss landscape and gradient calculations, leading to compromised model integrity and performance. This research underscores the severe threats posed by such data-centric attacks and highlights the urgent need for robust defenses in machine learning. BadGD sets a new standard for understanding and mitigating adversarial manipulations, ensuring the reliability and security of AI systems., Comment: 25 pages, 1 figure

With the proliferation of generative AI and the increasing volume of generative data (also called as synthetic data), assessing the fidelity of generative data has become a critical concern. In this paper, we propose a discriminative approach to estimate the total variation (TV) distance between two distributions as an effective measure of generative data fidelity. Our method quantitatively characterizes the relation between the Bayes risk in classifying two distributions and their TV distance. Therefore, the estimation of total variation distance reduces to that of the Bayes risk. In particular, this paper establishes theoretical results regarding the convergence rate of the estimation error of TV distance between two Gaussian distributions. We demonstrate that, with a specific choice of hypothesis class in classification, a fast convergence rate in estimating the TV distance can be achieved. Specifically, the estimation accuracy of the TV distance is proven to inherently depend on the separation of two Gaussian distributions: smaller estimation errors are achieved when the two Gaussian distributions are farther apart. This phenomenon is also validated empirically through extensive simulations. In the end, we apply this discriminative estimation method to rank fidelity of synthetic image data using the MNIST dataset.

Flat lenses have shown promising applications in miniaturized and ultracompact lightweight optical systems. However, it has been a great challenge in simultaneously achieving broadband achromatism and high numerical aperture. Here, we demonstrate that this long-term dilemma can be broken through by the zone division multiplex of the meta-atoms on a composite substrate possessing stepwise optical thickness. The aperture size can be freely expanded by increasing the optical thickness difference between the central and marginal zones of the substrate, free from achromatic bandwidth. The achromatic flat lens with both 0.9 numerical aperture and bandwidth of 650-1000 nm is experimentally achieved. A microscopic imaging with 1.1 {\mu}m resolution has also demonstrated. These unprecedented performances mark a substantial step toward practical applications of the flat lenses.

Few-shot model compression aims to compress a large model into a more compact one with only a tiny training set (even without labels). Block-level pruning has recently emerged as a leading technique in achieving high accuracy and low latency in few-shot CNN compression. But, few-shot compression for Vision Transformers (ViT) remains largely unexplored, which presents a new challenge. In particular, the issue of sparse compression exists in traditional CNN few-shot methods, which can only produce very few compressed models of different model sizes. This paper proposes a novel framework for few-shot ViT compression named DC-ViT. Instead of dropping the entire block, DC-ViT selectively eliminates the attention module while retaining and reusing portions of the MLP module. DC-ViT enables dense compression, which outputs numerous compressed models that densely populate the range of model complexity. DC-ViT outperforms state-of-the-art few-shot compression methods by a significant margin of 10 percentage points, along with lower latency in the compression of ViT and its variants., Comment: Accepted to CVPR 2024. Note: Jianxin Wu is a contributing author for the arXiv version of this paper but is not listed as an author in the CVPR version due to his role as Program Chair

In this paper, we investigate the scaling limit of heavy-tailed unstable cumulative INAR($\infty$) processes. These processes exhibit a power-law tail of the form $n^{-(1+\alpha)}$ for $\alpha \in (\frac{1}{2}, 1)$, and the $\ell^1$ norm of the kernel vector approaches $1$. Our results contrast with the scaling limits of continuous-time heavy-tailed unstable Hawkes processes and those of INAR($p$) processes. We show that the discrete-time scaling limit also has a long-memory property and can be seen as an integrated fractional Cox-Ingersoll-Ross process.

Metal-oxide interfaces with poor coherency have unique properties comparing to the bulk materials and offer broad applications in the fields of heterogeneous catalysis, battery, and electronics. However, current understanding of the three-dimensional (3D) atomic metal-oxide interfaces remains limited because of their inherent structural complexity and limitations of conventional two-dimensional imaging techniques. Here, we determine the 3D atomic structure of metal-oxide interfaces in zirconium-zirconia nanoparticles using atomic-resolution electron tomography. We quantitatively analyze the atomic concentration and the degree of oxidation, and find the coherency and translational symmetry of the interfaces are broken. Moreover, we observe porous structures such as Zr vacancies and nano-pores and investigate their distribution. Our findings provide a clear 3D atomic picture of metal-oxide interface with direct experimental evidence. We anticipate this work could encourage future studies on fundamental problems of oxides such as interfacial structures in semiconductor and atomic motion during oxidation process., Comment: 35 pages, 4 main figures, 17 Supplementary figures