Search

Wavefront shaping has become a powerful tool for manipulating light propagation in various complex media undergoing linear scattering. Controlling nonlinear optical interactions with spatial degrees of freedom is a relatively recent but growing area of research. A wavefront-shaping-based approach can be used to suppress nonlinear stimulated Brillouin scattering (SBS) and transverse mode instability (TMI), which are the two main limitations to power scaling in high-power narrowband fiber amplifiers. Here we formulate both SBS and TMI suppression as optimization problems with respect to coherent multimode input excitation in a given multimode fiber. We develop an efficient method for finding the globally optimal input excitation for SBS and TMI suppression using linear programming. We theoretically show that optimally exciting a standard multimode fiber leads to roughly an order of magnitude enhancement in output power limited by SBS and TMI, compared to fundamental-mode-only excitation. We find that the optimal mode content is robust to small perturbations and our approach works even in the presence of mode dependent loss and gain. Optimal mode content can be excited in real experiments using spatial light modulators, creating a novel platform for instability-free ultrahigh-power fiber lasers., Comment: 13 pages, 7 figures

We demonstrate sub-recoil Sisyphus cooling using the long-lived $^{3}\mathrm{P}_{0}$ clock state in alkaline-earth-like ytterbium. A 1388 nm optical standing wave nearly resonant with the $^{3}\textrm{P}_{0}$$\,\rightarrow$$\,^{3}\textrm{D}_{1}$ transition creates a spatially periodic light shift of the $^{3}\textrm{P}_{0}$ clock state. Following excitation on the ultranarrow clock transition, we observe Sisyphus cooling in this potential, as the light shift is correlated with excitation to $^{3}\textrm{D}_{1}$ and subsequent spontaneous decay to the $^{1}\textrm{S}_{0}$ ground state. We observe that cooling enhances the loading efficiency of atoms into a 759 nm magic-wavelength one-dimensional (1D) optical lattice, as compared to standard Doppler cooling on the $^{1}\textrm{S}_{0}$$\,\rightarrow\,$$^{3}\textrm{P}_{1}$ transition. Sisyphus cooling yields temperatures below 200 nK in the weakly confined, transverse dimensions of the 1D optical lattice. These lower temperatures improve optical lattice clocks by facilitating the use of shallow lattices with reduced light shifts, while retaining large atom numbers to reduce the quantum projection noise. This Sisyphus cooling can be pulsed or continuous and is applicable to a range of quantum metrology applications., Comment: 8 pages, 6 figures

To mitigate the suboptimal nature of graph structure, Graph Structure Learning (GSL) has emerged as a promising approach to improve graph structure and boost performance in downstream tasks. Despite the proposal of numerous GSL methods, the progresses in this field mostly concentrated on node-level tasks, while graph-level tasks (e.g., graph classification) remain largely unexplored. Notably, applying node-level GSL to graph classification is non-trivial due to the lack of find-grained guidance for intricate structure learning. Inspired by the vital role of subgraph in graph classification, in this paper we explore the potential of subgraph structure learning for graph classification by tackling the challenges of key subgraph selection and structure optimization. We propose a novel Motif-driven Subgraph Structure Learning method for Graph Classification (MOSGSL). Specifically, MOSGSL incorporates a subgraph structure learning module which can adaptively select important subgraphs. A motif-driven structure guidance module is further introduced to capture key subgraph-level structural patterns (motifs) and facilitate personalized structure learning. Extensive experiments demonstrate a significant and consistent improvement over baselines, as well as its flexibility and generalizability for various backbones and learning procedures., Comment: 16 pages, 8 figures

The Fermi Gamma-Ray Space Telescope reveals two large bubbles in the Galaxy, extending nearly symmetrically $\sim50^{\circ}$ above and below the Galactic center (GC). Previous simulations of bubble formation invoking active galactic nucleus (AGN) jets have assumed that the jets are vertical to the Galactic disk; however, in general, the jet orientation does not necessarily correlate with the rotational axis of the Galactic disk. Using three-dimensional special relativistic hydrodynamic simulations including cosmic rays (CRs) and thermal gas, we show that the dense clumpy gas within the Galactic disk disrupts jet collimation ("failed jets" hereafter), which causes the failed jets to form hot bubbles. Subsequent buoyancy in the stratified atmosphere renders them vertical to form the symmetric Fermi and eROSITA bubbles (collectively, Galactic bubbles). We find that (1) despite the relativistic jets emanated from the GC are at various angles $\le45^{\circ}$ with respect to the rotational axis of the Galaxy, the Galactic bubbles nonetheless appear aligned with the axis; (2) the edge of the eROSITA bubbles corresponds to a forward shock driven by the hot bubbles; (3) followed by the forward shock is a tangling contact discontinuity corresponding to the edge of the Fermi bubbles; (4) assuming a leptonic model we find that the observed gamma-ray bubbles and microwave haze can be reproduced with a best-fit CR power-law spectral index of 2.4; The agreements between the simulated and the observed multi-wavelength features suggest that forming the Galactic bubbles by oblique AGN failed jets is a plausible scenario., Comment: 29 pages, 18 figures

The recent rapid advancements in both sensing and machine learning technologies have given rise to the universal collection and utilization of people's biometrics, such as fingerprints, voices, retina/facial scans, or gait/motion/gestures data, enabling a wide range of applications including authentication, health monitoring, or much more sophisticated analytics. While providing better user experiences and deeper business insights, the use of biometrics has raised serious privacy concerns due to their intrinsic sensitive nature and the accompanying high risk of leaking sensitive information such as identity or medical conditions. In this paper, we propose a novel modality-agnostic data transformation framework that is capable of anonymizing biometric data by suppressing its sensitive attributes and retaining features relevant to downstream machine learning-based analyses that are of research and business values. We carried out a thorough experimental evaluation using publicly available facial, voice, and motion datasets. Results show that our proposed framework can achieve a \highlight{high suppression level for sensitive information}, while at the same time retain underlying data utility such that subsequent analyses on the anonymized biometric data could still be carried out to yield satisfactory accuracy., Comment: Preprint of IJIS version, https://link.springer.com/article/10.1007/s10207-024-00862-8

The growing richness of large-scale datasets has been crucial in driving the rapid advancement and wide adoption of machine learning technologies. The massive collection and usage of data, however, pose an increasing risk for people's private and sensitive information due to either inadvertent mishandling or malicious exploitation. Besides legislative solutions, many technical approaches have been proposed towards data privacy protection. However, they bear various limitations such as leading to degraded data availability and utility, or relying on heuristics and lacking solid theoretical bases. To overcome these limitations, we propose a formal information-theoretic definition for this utility-preserving privacy protection problem, and design a data-driven learnable data transformation framework that is capable of selectively suppressing sensitive attributes from target datasets while preserving the other useful attributes, regardless of whether or not they are known in advance or explicitly annotated for preservation. We provide rigorous theoretical analyses on the operational bounds for our framework, and carry out comprehensive experimental evaluations using datasets of a variety of modalities, including facial images, voice audio clips, and human activity motion sensor signals. Results demonstrate the effectiveness and generalizability of our method under various configurations on a multitude of tasks. Our code is available at https://github.com/jpmorganchase/MaSS., Comment: ICML 2024, GitHub: https://github.com/jpmorganchase/MaSS

Recommendation Systems (RS) are often plagued by popularity bias. When training a recommendation model on a typically long-tailed dataset, the model tends to not only inherit this bias but often exacerbate it, resulting in over-representation of popular items in the recommendation lists. This study conducts comprehensive empirical and theoretical analyses to expose the root causes of this phenomenon, yielding two core insights: 1) Item popularity is memorized in the principal spectrum of the score matrix predicted by the recommendation model; 2) The dimension collapse phenomenon amplifies the relative prominence of the principal spectrum, thereby intensifying the popularity bias. Building on these insights, we propose a novel debiasing strategy that leverages a spectral norm regularizer to penalize the magnitude of the principal singular value. We have developed an efficient algorithm to expedite the calculation of the spectral norm by exploiting the spectral property of the score matrix. Extensive experiments across seven real-world datasets and three testing paradigms have been conducted to validate the superiority of the proposed method., Comment: 23 pages, 9 figures

This paper addresses text-supervised semantic segmentation, aiming to learn a model capable of segmenting arbitrary visual concepts within images by using only image-text pairs without dense annotations. Existing methods have demonstrated that contrastive learning on image-text pairs effectively aligns visual segments with the meanings of texts. We notice that there is a discrepancy between text alignment and semantic segmentation: A text often consists of multiple semantic concepts, whereas semantic segmentation strives to create semantically homogeneous segments. To address this issue, we propose a novel framework, Image-Text Co-Decomposition (CoDe), where the paired image and text are jointly decomposed into a set of image regions and a set of word segments, respectively, and contrastive learning is developed to enforce region-word alignment. To work with a vision-language model, we present a prompt learning mechanism that derives an extra representation to highlight an image segment or a word segment of interest, with which more effective features can be extracted from that segment. Comprehensive experimental results demonstrate that our method performs favorably against existing text-supervised semantic segmentation methods on six benchmark datasets., Comment: CVPR 2024

With the capacity to capture high-order collaborative signals, Graph Neural Networks (GNNs) have emerged as powerful methods in Recommender Systems (RS). However, their efficacy often hinges on the assumption that training and testing data share the same distribution (a.k.a. IID assumption), and exhibits significant declines under distribution shifts. Distribution shifts commonly arises in RS, often attributed to the dynamic nature of user preferences or ubiquitous biases during data collection in RS. Despite its significance, researches on GNN-based recommendation against distribution shift are still sparse. To bridge this gap, we propose Distributionally Robust GNN (DR-GNN) that incorporates Distributional Robust Optimization (DRO) into the GNN-based recommendation. DR-GNN addresses two core challenges: 1) To enable DRO to cater to graph data intertwined with GNN, we reinterpret GNN as a graph smoothing regularizer, thereby facilitating the nuanced application of DRO; 2) Given the typically sparse nature of recommendation data, which might impede robust optimization, we introduce slight perturbations in the training distribution to expand its support. Notably, while DR-GNN involves complex optimization, it can be implemented easily and efficiently. Our extensive experiments validate the effectiveness of DR-GNN against three typical distribution shifts. The code is available at https://github.com/WANGBohaO-jpg/DR-GNN., Comment: Accepted by WWW2024

Time-reversal symmetry enables shaping input waves to control output waves in many linear and nonlinear systems; however energy dissipation violates such symmetry. We consider a saturated multimode fiber amplifier in which light generates heat flow and suffers nonlinear thermo-optical scattering, breaking time-reversal symmetry. We identify a spacetime symmetry which maps the target output back to an input field. This mapping employs phase conjugation, gain and absorption substitution but not time reversal, and holds in steady-state and for slowly varying inputs. Our results open the possibility of output control of a saturated multimode fiber amplifier.

We implement coherent delocalization as a tool for improving the two primary metrics of atomic clock performance: systematic uncertainty and instability. By decreasing atomic density with coherent delocalization, we suppress cold-collision shifts and two-body losses. Atom loss attributed to Landau-Zener tunneling in the ground lattice band would compromise coherent delocalization at low trap depths for our $^{171}$Yb atoms; hence, we implement for the first time delocalization in excited lattice bands. Doing so increases the spatial distribution of atoms trapped in the vertically-oriented optical lattice by $\sim7$ times. At the same time we observe a reduction of the cold-collision shift by 6.5(8) times, while also making inelastic two-body loss negligible. With these advantages, we measure the trap-light-induced quenching rate and natural lifetime of the ${}^3$P${}_0$ excited-state as $5.7(7)\times10^{-4}$ $E_r^{-1}s^{-1}$ and 19(2) s, respectively.

Recent works have shown that by using large pre-trained models along with learnable prompts, rehearsal-free methods for class-incremental learning (CIL) settings can achieve superior performance to prominent rehearsal-based ones. Rehearsal-free CIL methods struggle with distinguishing classes from different tasks, as those are not trained together. In this work we propose a regularization method based on virtual outliers to tighten decision boundaries of the classifier, such that confusion of classes among different tasks is mitigated. Recent prompt-based methods often require a pool of task-specific prompts, in order to prevent overwriting knowledge of previous tasks with that of the new task, leading to extra computation in querying and composing an appropriate prompt from the pool. This additional cost can be eliminated, without sacrificing accuracy, as we reveal in the paper. We illustrate that a simplified prompt-based method can achieve results comparable to previous state-of-the-art (SOTA) methods equipped with a prompt pool, using much less learnable parameters and lower inference cost. Our regularization method has demonstrated its compatibility with different prompt-based methods, boosting those previous SOTA rehearsal-free CIL methods' accuracy on the ImageNet-R and CIFAR-100 benchmarks. Our source code is available at https://github.com/jpmorganchase/ovor., Comment: Accepted by ICLR 2024

Machine unlearning has emerged as a new paradigm to deliberately forget data samples from a given model in order to adhere to stringent regulations. However, existing machine unlearning methods have been primarily focused on classification models, leaving the landscape of unlearning for generative models relatively unexplored. This paper serves as a bridge, addressing the gap by providing a unifying framework of machine unlearning for image-to-image generative models. Within this framework, we propose a computationally-efficient algorithm, underpinned by rigorous theoretical analysis, that demonstrates negligible performance degradation on the retain samples, while effectively removing the information from the forget samples. Empirical studies on two large-scale datasets, ImageNet-1K and Places-365, further show that our algorithm does not rely on the availability of the retain samples, which further complies with data retention policy. To our best knowledge, this work is the first that represents systemic, theoretical, empirical explorations of machine unlearning specifically tailored for image-to-image generative models. Our code is available at https://github.com/jpmorganchase/l2l-generator-unlearning., Comment: ICLR 2024

We treat disordered Heisenberg model in 1D as the "standard model" of many-body localization (MBL). Two independent order parameters stemming purely from the half-chain von Neumann entanglement entropy $S_{\textrm{vN}}$ are introduced to probe its eigenstate transition. From symmetry-endowed entropy decomposition, they are probability distribution deviation $|d(p_n)|$ and von Neumann entropy $S_{\textrm{vN}}^{n}(D_n\!=\!\mbox{max})$ of the maximum-dimensional symmetry subdivision. Finite-size analyses reveal that $\{p_n\}$ drives the localization transition, preceded by a thermalization breakdown transition governed by $\{S_{\textrm{vN}}^{n}\}$. For noninteracting case, these transitions coincide, but in interacting situation they separate. Such separability creates an intermediate phase region and may help discriminate between the Anderson and MBL transitions. An obstacle whose solution eludes community to date is the violation of Harris criterion in nearly all numeric investigations of MBL so far. Upon elucidating the mutually independent components in $S_{\textrm{vN}}$, it is clear that previous studies of eigenspectra, $S_{\textrm{vN}}$, and the like lack resolution to pinpoint (thus completely overlook) the crucial internal structures of the transition. We show, for the first time, that after this necessary decoupling, the universal critical exponents for both transitions of $|d(p_n)|$ and $S_{\textrm{vN}}^{n}(D_n\!=\!\mbox{max})$ fulfill the Harris criterion: $\nu\approx2.0\ (\nu\approx1.5)$ for quench (quasirandom) disorder. Our work puts forth "symmetry combined with entanglement" as the missing organization principle for the generic eigenstate matter and transition.

Deep learning has witnessed significant advancements in recent years at the cost of increasing training, inference, and model storage overhead. While existing model compression methods strive to reduce the number of model parameters while maintaining high accuracy, they inevitably necessitate the re-training of the compressed model or impose architectural constraints. To overcome these limitations, this paper presents a novel framework, termed \textbf{K}nowledge \textbf{T}ranslation (KT), wherein a ``translation'' model is trained to receive the parameters of a larger model and generate compressed parameters. The concept of KT draws inspiration from language translation, which effectively employs neural networks to convert different languages, maintaining identical meaning. Accordingly, we explore the potential of neural networks to convert models of disparate sizes, while preserving their functionality. We propose a comprehensive framework for KT, introduce data augmentation strategies to enhance model performance despite restricted training data, and successfully demonstrate the feasibility of KT on the MNIST dataset. Code is available at \url{https://github.com/zju-SWJ/KT}.

The rapid growth of machine learning has spurred legislative initiatives such as ``the Right to be Forgotten,'' allowing users to request data removal. In response, ``machine unlearning'' proposes the selective removal of unwanted data without the need for retraining from scratch. While the Neural-Tangent-Kernel-based (NTK-based) unlearning method excels in performance, it suffers from significant computational complexity, especially for large-scale models and datasets. Our work introduces ``Fast-NTK,'' a novel NTK-based unlearning algorithm that significantly reduces the computational complexity by incorporating parameter-efficient fine-tuning methods, such as fine-tuning batch normalization layers in a CNN or visual prompts in a vision transformer. Our experimental results demonstrate scalability to much larger neural networks and datasets (e.g., 88M parameters; 5k images), surpassing the limitations of previous full-model NTK-based approaches designed for smaller cases (e.g., 8M parameters; 500 images). Notably, our approach maintains a performance comparable to the traditional method of retraining on the retain set alone. Fast-NTK can thus enable for practical and scalable NTK-based unlearning in deep neural networks., Comment: 6 pages, 1 figure

Sampling from diffusion models can be treated as solving the corresponding ordinary differential equations (ODEs), with the aim of obtaining an accurate solution with as few number of function evaluations (NFE) as possible. Recently, various fast samplers utilizing higher-order ODE solvers have emerged and achieved better performance than the initial first-order one. However, these numerical methods inherently result in certain approximation errors, which significantly degrades sample quality with extremely small NFE (e.g., around 5). In contrast, based on the geometric observation that each sampling trajectory almost lies in a two-dimensional subspace embedded in the ambient space, we propose Approximate MEan-Direction Solver (AMED-Solver) that eliminates truncation errors by directly learning the mean direction for fast diffusion sampling. Besides, our method can be easily used as a plugin to further improve existing ODE-based samplers. Extensive experiments on image synthesis with the resolution ranging from 32 to 512 demonstrate the effectiveness of our method. With only 5 NFE, we achieve 6.61 FID on CIFAR-10, 10.74 FID on ImageNet 64$\times$64, and 13.20 FID on LSUN Bedroom. Our code is available at https://github.com/zju-pi/diff-sampler., Comment: Accepted by CVPR 2024

We present a faster repumping scheme for strontium magneto-optical traps operating on the broad ${\mathrm{5s^2} ^1\mathrm{S}_0} - {\mathrm{5s5p} ^1\mathrm{P}_1}$ laser cooling transition. Contrary to existing repumping schemes, we directly address lost atoms that spontaneously decayed to the ${\mathrm{5s4d} ^1\mathrm{D}_2}$ state, sending them back into the laser cooling cycle by optical pumping on the ${\mathrm{5s4d} ^1\mathrm{D}_2} - {\mathrm{5s8p} ^1\mathrm{P}_1}$ transition. We thus avoid the $\sim 100 \, \mathrm{\mu s}$-slow decay path from ${\mathrm{5s4d} ^1\mathrm{D}_2}$ to the ${\mathrm{5s5p} ^3\mathrm{P}_{1,2}}$ states that is part of other repumping schemes. Using one low-cost external-cavity diode laser emitting at $448 \, \mathrm{nm}$, we show our scheme increases the flux out of a 2D magneto-optical trap by $60 \, \%$ compared to without repumping. Furthermore, we perform spectroscopy on the ${\mathrm{5s4d} ^1\mathrm{D}_2} - {\mathrm{5s8p} ^1\mathrm{P}_1}$ transition and measure its frequency $\nu_{\mathrm{^{88}Sr}} = (668917515.3 \pm 4.0 \pm 25) \, \mathrm{MHz}$. We also measure the frequency shifts between the four stable isotopes of strontium and infer the specific mass and field shift factors, $\delta \nu_\text{SMS} ^{88,86} = -267(45) \, \mathrm{MHz}$ and $\delta \nu_\text{FS} ^{88,86} = 2(42) \, \mathrm{MHz}$. Finally, we measure the hyperfine splitting of the ${\mathrm{5s8p} ^1\mathrm{P}_1}$ state in fermionic strontium, and deduce the magnetic dipole and electric quadrupole coupling coefficients $A = -4(5) \, \mathrm{MHz}$ and $B = 5(35) \, \mathrm{MHz}$. Our experimental demonstration shows that this simple and very fast scheme could improve the laser cooling and imaging performance of cold strontium atom devices, such as quantum computers based on strontium atoms in arrays of optical tweezers., Comment: 10 pages, 5 figures