Search

We identify sufficient conditions to avoid known failure modes, including representation, dimensional, cluster and intracluster collapses, occurring in non-contrastive self-supervised learning. Based on these findings, we propose a principled design for the projector and loss function. We theoretically demonstrate that this design introduces an inductive bias that promotes learning representations that are both decorrelated and clustered without explicit enforcing these properties and leading to improved generalization. To the best of our knowledge, this is the first solution that achieves robust training with respect to these failure modes while guaranteeing enhanced generalization performance in downstream tasks. We validate our theoretical findings on image datasets including SVHN, CIFAR10, CIFAR100 and ImageNet-100, and show that our solution, dubbed FALCON, outperforms existing feature decorrelation and cluster-based self-supervised learning methods in terms of generalization to clustering and linear classification tasks.

Attosecond photoelectron interferometry based on the combination of an attosecond pulse train and a synchronized infrared field is a fundamental technique for the temporal characterization of attosecond waveforms and for the investigation of electron dynamics in the photoionization process. In this approach, the comb of extreme ultraviolet harmonics typically lies above the ionization threshold of the target under investigation, thus releasing a photoelectron by single-photon absorption. The interaction of the outgoing photoelectron with the infrared pulse results in the absorption or emission of infrared photons, thereby creating additional peaks in the photoelectron spectrum, referred to as sidebands. While, in the absence of resonances in the first ionization step, the phases imparted on the photoionization process evolve smoothly with the photon energy, the presence of intermediate resonances imprints a large additional phase on the outgoing photoelectron wave packet. In this work, using a comb of harmonics below and above the ionization threshold of neon, we investigate the effect of intermediate bound excited states on attosecond photoelectron interferometry. We show that the phase of the oscillations of the sidebands and their angular distributions are strongly affected by such resonances. By slightly tuning the photon energies of the extreme ultraviolet harmonics, we show how the contributions of selected resonances can be enhanced or suppressed., Comment: 10 pages, 8 figures

Neural probabilistic logic systems follow the neuro-symbolic (NeSy) paradigm by combining the perceptive and learning capabilities of neural networks with the robustness of probabilistic logic. Learning corresponds to likelihood optimization of the neural networks. However, to obtain the likelihood exactly, expensive probabilistic logic inference is required. To scale learning to more complex systems, we therefore propose to instead optimize a sampling based objective. We prove that the objective has a bounded error with respect to the likelihood, which vanishes when increasing the sample count. Furthermore, the error vanishes faster by exploiting a new concept of sample diversity. We then develop the EXPLAIN, AGREE, LEARN (EXAL) method that uses this objective. EXPLAIN samples explanations for the data. AGREE reweighs each explanation in concordance with the neural component. LEARN uses the reweighed explanations as a signal for learning. In contrast to previous NeSy methods, EXAL can scale to larger problem sizes while retaining theoretical guarantees on the error. Experimentally, our theoretical claims are verified and EXAL outperforms recent NeSy methods when scaling up the MNIST addition and Warcraft pathfinding problems.

Ultrafast all-optical logic devices based on nonlinear light-matter interactions hold the promise to overcome the speed limitations of conventional electronic devices. Strong coupling of excitons and photons inside an optical resonator enhances such interactions and generates new polariton states which give access to unique nonlinear phenomena, such as Bose-Einstein condensation, used for all-optical ultrafast polariton transistors. However, the pulse energies required to pump such devices range from tens to hundreds of pJ, making them not competitive with electronic transistors. Here we introduce a new paradigm for all-optical switching based on the ultrafast transition from the strong to the weak coupling regime in microcavities embedding atomically thin transition metal dichalcogenides. Employing single and double stacks of hBN-encapsulated MoS$_2$ homobilayers with high optical nonlinearities and fast exciton relaxation times, we observe a collapse of the 55-meV polariton gap and its revival in less than one picosecond, lowering the threshold for optical switching below 4 pJ per pulse, while retaining ultrahigh switching frequencies. As an additional degree of freedom, the switching can be triggered pumping either the intra- or the interlayer excitons of the bilayers at different wavelengths, speeding up the polariton dynamics, owing to unique interspecies excitonic interactions. Our approach will enable the development of compact ultrafast all-optical logical circuits and neural networks, showcasing a new platform for polaritonic information processing based on manipulating the light-matter coupling.

In this work, we propose to study the global geometrical properties of generative models. We introduce a new Riemannian metric to assess the similarity between any two data points. Importantly, our metric is agnostic to the parametrization of the generative model and requires only the evaluation of its data likelihood. Moreover, the metric leads to the conceptual definition of generative distances and generative geodesics, whose computation can be done efficiently in the data space. Their approximations are proven to converge to their true values under mild conditions. We showcase three proof-of-concept applications of this global metric, including clustering, data visualization, and data interpolation, thus providing new tools to support the geometrical understanding of generative models., Comment: 10 pages, 9 figures

In recent years, twisted bilayer systems such as bilayer graphene have attracted a great deal of attention as the twist angle introduces a degree of freedom which can be used to non-trivially modify system properties. This idea has been picked up in the cold atom community, first with a theoretical proposal to simulate twisted bilayers in state-dependent optical lattices, and, more recently, with an experimental realization of twisted bilayers with bosonic atoms in two different spin states. In this manuscript, we theoretically investigate dipolar bosons in a twisted bilayer geometry. The interplay between dipolar interaction and the twist between the layers results in the emergence of quantum states not observed in the absence of twist. We study how system properties vary as we change the twist angle at fixed distance between the layers and fixed dipolar interaction. We find that at a twist angle $\theta=0.1^{\circ}$, the observed quantum phases are consistent with those seen in the absence of twist angle, i.e. paired superfluid, paired supersolid, and paired solid phases. However, a slight increase in the twist angle to $\theta=0.2^{\circ}$ disrupts these paired phases in favor of a phase separation between checkerboard solid and superfluid regions. Notably, at a twist angle of $\theta=5.21^{\circ}$, the local occupation number follows the moir\'e pattern of the underlying moir\'e bilayers so that a periodic structure of insulating islands is formed. These insulating islands are surrounded by a superfluid., Comment: 6 pages, 4 figures

Tailored light-matter interactions in the strong coupling regime enable the manipulation and control of quantum systems with up to unit efficiency, with applications ranging from quantum information to photochemistry. While strong light-matter interactions are readily induced at the valence electron level using long-wavelength radiation, comparable phenomena have been only recently observed with short wavelengths, accessing highly-excited multi-electron and inner-shell electron states. However, the quantum control of strong-field processes at short wavelengths has not been possible, so far, due to the lack of pulse shaping technologies in the extreme ultraviolet (XUV) and X-ray domain. Here, exploiting pulse shaping of the seeded free-electron laser (FEL) FERMI, we demonstrate the strong-field quantum control of ultrafast Rabi dynamics in helium atoms with high fidelity. Our approach unravels a strong dressing of the ionization continuum, otherwise elusive to experimental observables. The latter is exploited to achieve control of the total ionization rate, with prospective applications in many XUV and soft X-ray experiments. Leveraging recent advances in intense few-femtosecond to attosecond XUV to soft X-ray light sources, our results open an avenue to the efficient manipulation and selective control of core electron processes and electron correlation phenomena in real time.

Amid efforts to address the ongoing crisis of high costs, high debt burdens, and decreasing confidence in the value of higher education, TICAS [The Institute for College Access & Success] commissioned leading academics to write a series of papers to inform the policy conversation about how to implement effective, equitable, and sustainable solutions to the college affordability crisis. Collectively, the papers outline the current college financing landscape, discuss potential policy options for reducing reliance on debt, and examine specific components that could be addressed as part of a larger federal-state funding partnership to achieve debt-free college for all.

Double-core-hole (DCH) states in isolated water and heavy water molecules, resulting from the sequential absorption of two x-ray photons, have been investigated. A comparison of the subsequent Auger emission spectra from the two isotopes provides direct evidence of ultrafast nuclear motion during the 1.5 fs lifetime of these DCH states. Our numerical results align well with the experimental data, providing for various DCH states an in-depth study of the dynamics responsible of the observed isotope effect.

Investigating finite temperature effects on quantum phases is key to their experimental realization. Finite temperature, and the interplay between quantum and thermal fluctuations can undermine properties and/or key features of quantum systems but they can also bring upon interesting phenomena. In this paper, we present a comprehensive investigation of the finite temperature phase diagram of two-dimensional lattice dipolar bosons coupled to a high-finesse optical cavity. Interestingly, we observe that checkerboard density-density correlations are enhanced at finite temperature. Indeed, we found that finite temperature drives a superfluid ground state into a normal state which will then develop checkerboard order at higher temperatures. We show that this effect is solely due to the cavity-mediated interactions. We also confirm that the supersolid checkerboard phase survives for a wide range of filling factors up to temperature scale of the order of half hopping amplitude, while the checkerboard diagonal order can survive up to temperatures of a few hopping amplitudes., Comment: 5 pages, 4 figures

We present a combined theoretical and experimental work investigating the angle-resolved phases of the photoionization process driven by a two-color field consisting of an attosecond pulse train and an infrared pulse in an ensemble of randomly oriented molecules. We derive a general form for the two-color photoelectron (and time-delay) angular distribution valid also in the case of chiral molecules and when relative polarizations of the photons contributing to the attosecond photoelectron interferometer differ. We show a comparison between the experimental data and the theoretical predictions in an ensemble of methane and deuteromethane molecules, discussing the effect of nuclear dynamics on the photoionization phases. Finally, we demonstrate that the oscillating component and the phase of the two-color signal can be fitted using complex asymmetry parameters, in perfect analogy with the atomic case.

Self-supervised learning is a popular and powerful method for utilizing large amounts of unlabeled data, for which a wide variety of training objectives have been proposed in the literature. In this study, we perform a Bayesian analysis of state-of-the-art self-supervised learning objectives, elucidating the underlying probabilistic graphical models in each class and presenting a standardized methodology for their derivation from first principles. The analysis also indicates a natural means of integrating self-supervised learning with likelihood-based generative models. We instantiate this concept within the realm of cluster-based self-supervised learning and energy models, introducing a novel lower bound which is proven to reliably penalize the most important failure modes. Furthermore, this newly proposed lower bound enables the training of a standard backbone architecture without the necessity for asymmetric elements such as stop gradients, momentum encoders, or specialized clustering layers - typically introduced to avoid learning trivial solutions. Our theoretical findings are substantiated through experiments on synthetic and real-world data, including SVHN, CIFAR10, and CIFAR100, thus showing that our objective function allows to outperform existing self-supervised learning strategies in terms of clustering, generation and out-of-distribution detection performance by a wide margin. We also demonstrate that GEDI can be integrated into a neuro-symbolic framework to mitigate the reasoning shortcut problem and to learn higher quality symbolic representations thanks to the enhanced classification performance., Comment: Changes from previous version: added mean and standard deviations in experiments. Integral version of workshop paper arXiv:2309.15420. Improved GEDI version (from two stages to single stage training) arxiv:2212.13425

Categorical random variables can faithfully represent the discrete and uncertain aspects of data as part of a discrete latent variable model. Learning in such models necessitates taking gradients with respect to the parameters of the categorical probability distributions, which is often intractable due to their combinatorial nature. A popular technique to estimate these otherwise intractable gradients is the Log-Derivative trick. This trick forms the basis of the well-known REINFORCE gradient estimator and its many extensions. While the Log-Derivative trick allows us to differentiate through samples drawn from categorical distributions, it does not take into account the discrete nature of the distribution itself. Our first contribution addresses this shortcoming by introducing the CatLog-Derivative trick - a variation of the Log-Derivative trick tailored towards categorical distributions. Secondly, we use the CatLog-Derivative trick to introduce IndeCateR, a novel and unbiased gradient estimator for the important case of products of independent categorical distributions with provably lower variance than REINFORCE. Thirdly, we empirically show that IndeCateR can be efficiently implemented and that its gradient estimates have significantly lower bias and variance for the same number of samples compared to the state of the art.

We present a novel objective function for cluster-based self-supervised learning (SSL) that is designed to circumvent the triad of failure modes, namely representation collapse, cluster collapse, and the problem of invariance to permutations of cluster assignments. This objective consists of three key components: (i) A generative term that penalizes representation collapse, (ii) a term that promotes invariance to data augmentations, thereby addressing the issue of label permutations and (ii) a uniformity term that penalizes cluster collapse. Additionally, our proposed objective possesses two notable advantages. Firstly, it can be interpreted from a Bayesian perspective as a lower bound on the data log-likelihood. Secondly, it enables the training of a standard backbone architecture without the need for asymmetric elements like stop gradients, momentum encoders, or specialized clustering layers. Due to its simplicity and theoretical foundation, our proposed objective is well-suited for optimization. Experiments on both toy and real world data demonstrate its effectiveness, Comment: Some sentences in the Background Section are overlapping with Section 2 in arXiv:2304.11357 However, the main technical content and all other sections are different

We introduce GEDI, a Bayesian framework that combines existing self-supervised learning objectives with likelihood-based generative models. This framework leverages the benefits of both GEnerative and DIscriminative approaches, resulting in improved symbolic representations over standalone solutions. Additionally, GEDI can be easily integrated and trained jointly with existing neuro-symbolic frameworks without the need for additional supervision or costly pre-training steps. We demonstrate through experiments on real-world data, including SVHN, CIFAR10, and CIFAR100, that GEDI outperforms existing self-supervised learning strategies in terms of clustering performance by a significant margin. The symbolic component further allows it to leverage knowledge in the form of logical constraints to improve performance in the small data regime., Comment: ICLR 2023 Workshop NeSy-GeMs. arXiv admin note: substantial text overlap with arXiv:2212.13425

Interval graphs and proper interval graphs are well known graph classes, for which several generalizations have been proposed in the literature. In this work, we study the (proper) thinness, and several variations, for the classes of cographs, crowns graphs and grid graphs. We provide the exact values for several variants of thinness (proper, independent, complete, precedence, and combinations of them) for the crown graphs $CR_n$. For cographs, we prove that the precedence thinness can be determined in polynomial time. We also improve known bounds for the thinness of $n \times n$ grids $GR_n$ and $m \times n$ grids $GR_{m,n}$, proving that $\left \lceil \frac{n-1}{3} \right \rceil \leq \mbox{thin}(GR_n) \leq \left \lceil \frac{n+1}{2} \right \rceil$. Regarding the precedence thinness, we prove that $\mbox{prec-thin}(GR_{n,2}) = \left \lceil \frac{n+1}{2} \right \rceil$ and that $\left \lceil \frac{n-1}{3} \right \rceil \left \lceil\frac{n-1}{2} \right \rceil + 1 \leq \mbox{prec-thin}(GR_n) \leq \left \lceil\frac{n-1}{2} \right \rceil^2+1$. As applications, we show that the $k$-coloring problem is NP-complete for precedence $2$-thin graphs and for proper $2$-thin graphs, when $k$ is part of the input. On the positive side, it is polynomially solvable for precedence proper $2$-thin graphs, given the order and partition.

Isotopic substitution in molecular systems can affect fundamental molecular properties including the energy position and spacing of electronic, vibrational and rotational levels, thus modifying the dynamics associated to their coherent superposition. In extreme ultraviolet spectroscopy, the photoelectron leaving the molecule after the absorption of a single photon can trigger an ultrafast nuclear motion in the cation, which can lead, eventually, to molecular fragmentation. This dynamics depends on the mass of the constituents of the cation, thus showing, in general, a significant isotopic dependence. In time-resolved attosecond photoelectron interferometry, the absorption of the extreme ultraviolet photon is accompanied by the exchange of an additional quantum of energy (typically in the infrared spectral range) with the photoelectron-photoion system, offering the opportunity to investigate in time the influence of isotopic substitution on the characteristics of the photoionisation dynamics. Here we show that attosecond photoelectron interferometry is sensitive to isotopic substitution by investigating the two-color photoionisation spectra measured in a mixture of methane (CH$_4$) and deuteromethane (CD$_4$). The isotopic dependence manifests itself in the modification of the amplitude and contrast of the oscillations of the photoelectron peaks generated in the two-color field with the two isotopologues. The observed effects are interpreted considering the differences in the time evolution of the nuclear autocorrelation functions in the two molecules.

Two types of long range interactions, dipolar interaction and cavity-mediated interaction lead to exotic quantum phases. Both interactions have been realized and observed in optical lattice setups. Here, we study quantum phases of dipolar bosons trapped in optical lattices and coupled to a high-finesse cavity where both dipolar interaction and cavity-mediated interaction coexist. We perform quantum Monte Carlo simulations, and find that the checkerboard solid is enhanced and the checkerboard supersolid phase can exist in a wide range of densities (e.g. $ 0.27\lesssim n\lesssim0.73 $). Our unbiased numerical results suggest that both solid and supersolid phases can be achieved experimentally with magnetic atoms coupled to a cavity., Comment: 8 pages, 7 figures

Self-supervised learning is a popular and powerful method for utilizing large amounts of unlabeled data, for which a wide variety of training objectives have been proposed in the literature. In this study, we perform a Bayesian analysis of state-of-the-art self-supervised learning objectives and propose a unified formulation based on likelihood learning. Our analysis suggests a simple method for integrating self-supervised learning with generative models, allowing for the joint training of these two seemingly distinct approaches. We refer to this combined framework as GEDI, which stands for GEnerative and DIscriminative training. Additionally, we demonstrate an instantiation of the GEDI framework by integrating an energy-based model with a cluster-based self-supervised learning model. Through experiments on synthetic and real-world data, including SVHN, CIFAR10, and CIFAR100, we show that GEDI outperforms existing self-supervised learning strategies in terms of clustering performance by a wide margin. We also demonstrate that GEDI can be integrated into a neural-symbolic framework to address tasks in the small data regime, where it can use logical constraints to further improve clustering and classification performance., Comment: Fixed typos/cleaned the experimental section

The implementation of attosecond photoelectron-photoion coincidence spectroscopy for the investigation of atomic and molecular dynamics calls for a high-repetition-rate driving source combined with experimental setups characterized by excellent stability for data acquisition over time intervals ranging from a few hours up to a few days. This requirement is crucial for the investigation of processes characterized by low cross sections and for the characterization of fully differential photoelectron(s) and photoion(s) angular and energy distributions. We demonstrate that the implementation of industrial-grade lasers, combined with a careful design of the delay line implemented in the pump-probe setup, allows one to reach ultrastable experimental conditions leading to an error in the estimation of the time delays of only 12 as. This result opens new possibilities for the investigation of attosecond dynamics in simple quantum systems.