Search

Large Language Models (LLMs) have revolutionized AI systems by enabling communication with machines using natural language. Recent developments in Generative AI (GenAI) like Vision-Language Models (GPT-4V) and Gemini have shown great promise in using LLMs as multimodal systems. This new research line results in building Generative AI systems, GenAISys for short, that are capable of multimodal processing and content creation, as well as decision-making. GenAISys use natural language as a communication means and modality encoders as I/O interfaces for processing various data sources. They are also equipped with databases and external specialized tools, communicating with the system through a module for information retrieval and storage. This paper aims to explore and state new research directions in Generative AI Systems, including how to design GenAISys (compositionality, reliability, verifiability), build and train them, and what can be learned from the system-based perspective. Cross-disciplinary approaches are needed to answer open questions about the inner workings of GenAI systems.

Experimental studies have found unusual transport properties in Ce$_3$Bi$_4$Pd$_3$ which are potentially a consequence of the interplay between band-structure topology and electronic correlations. Based on these measurements, the existence of Weyl points in strongly renormalized, flat quasiparticle bands has been postulated. However, so far, there has been neither a direct spectroscopic observation of these, nor a calculation from first principles that would confirm their existence close to the Fermi energy. Here, we present density functional theory (DFT) and dynamical mean field theory (DMFT) calculations and study the low-energy excitations and their topological properties. We find that the Kondo effect promotes two out of the six angular momentum $J=5/2$ states, with the other four pushed to higher energies. Further, we find Weyl nodes close to the Fermi energy as previously suggested for explaining the observed giant spontaneous Hall effect in Ce$_3$Bi$_4$Pd$_3$, as well as nodal lines.

Optimizing deep neural networks is one of the main tasks in successful deep learning. Current state-of-the-art optimizers are adaptive gradient-based optimization methods such as Adam. Recently, there has been an increasing interest in formulating gradient-based optimizers in a probabilistic framework for better estimation of gradients and modeling uncertainties. Here, we propose to combine both approaches, resulting in the Variational Stochastic Gradient Descent (VSGD) optimizer. We model gradient updates as a probabilistic model and utilize stochastic variational inference (SVI) to derive an efficient and effective update rule. Further, we show how our VSGD method relates to other adaptive gradient-based optimizers like Adam. Lastly, we carry out experiments on two image classification datasets and four deep neural network architectures, where we show that VSGD outperforms Adam and SGD.

De novo drug design requires simultaneously generating novel molecules outside of training data and predicting their target properties, making it a hard task for generative models. To address this, we propose Joint Transformer that combines a Transformer decoder, Transformer encoder, and a predictor in a joint generative model with shared weights. We formulate a probabilistic black-box optimization algorithm that employs Joint Transformer to generate novel molecules with improved target properties and outperforms other SMILES-based optimization methods in de novo drug design., Comment: Accepted to NeurIPS 2023 Generative AI and Biology Workshop

Running large-scale computer codes for huge fluid flow problems requires not only large supercomputers but also efficient and well-optimized computer codes that save the resources of those supercomputers. This paper evaluates the high-performance implementation of the recently proposed Lattice Boltzmann Method (LBM) algorithm with a fixed viscosity and relaxation time $\tau=1$ called Tau1. We show that the performance of the Tau1 algorithm is almost $4\times$ higher than other state-of-the-art standard LBM implementations. We support this finding by detailed complexity analysis and performance study based on the code with several optimizations, including multithreading, vector processing, significantly decreased number of divisions, and dedicated memory layout. We studied its performance in porous media flow in the three-dimensional model of porosity-varying porous medium to make it sound in the physical context. We find performance drops with porosity, which we link to the fact that memory access patterns change dramatically with the increased complexity of the pore space. In contrast to standard LBM implementations, where the performance drops with the number of lattice links, the processing speed of the new algorithm measured in fluid node updates per second is almost constant regardless of lattice arrangements., Comment: 20 pages, 8 figures

Infinite-layer nickelates show high-temperature superconductivity, and the experimental phase diagram agrees well with the one simulated within the dynamical vertex approximation (D$\Gamma$A). Here, we compare the spin-fluctuation spectrum behind these calculations to resonant inelastic X-ray scattering experiments. The overall agreement is good. This independent cross-validation of the strength of spin fluctuations strongly supports the scenario, advanced by D$\Gamma$A, that spin-fluctuations are the mediator of the superconductivity observed in nickelates., Comment: 13 pages, 8 figures

High-temperature unconventional superconductivity quite generically emerges from doping a strongly correlated parent compound, often (close to) an antiferromagnetic insulator. The recently developed dynamical vertex approximation is a state-of-the-art technique that has quantitatively predicted the superconducting dome of nickelates. Here, we apply it to study the effect of pressure in the infinite-layer nickelate Sr$_x$Pr$_ {1-x}$NiO$_2$. We reproduce the increase of the critical temperature ($T_c$) under pressure found in experiment up to 12 GPa. According to our results, $T_c$ can be further increased with higher pressures. Even without Sr-doping the parent compound, PrNiO$_2$, will become a high-temperature superconductor thanks to a strongly enhanced self-doping of the \nidxsqysq{} orbital under pressure. With a maximal \Tc{} of 100\,K around 100\,GPa, nickelate superconductors can reach that of the best cuprates., Comment: Main text: 6 pages, 4 figures. Supplementary information: 18 pages, 16 figures

Predicting patient features from single-cell data can help identify cellular states implicated in health and disease. Linear models and average cell type expressions are typically favored for this task for their efficiency and robustness, but they overlook the rich cell heterogeneity inherent in single-cell data. To address this gap, we introduce MixMIL, a framework integrating Generalized Linear Mixed Models (GLMM) and Multiple Instance Learning (MIL), upholding the advantages of linear models while modeling cell state heterogeneity. By leveraging predefined cell embeddings, MixMIL enhances computational efficiency and aligns with recent advancements in single-cell representation learning. Our empirical results reveal that MixMIL outperforms existing MIL models in single-cell datasets, uncovering new associations and elucidating biological mechanisms across different domains., Comment: AISTATS 2024 Oral, Code: https://github.com/AIH-SGML/MixMIL

We present an algorithm for the determination of the local symmetry group for arbitrary k-points in 3D Brillouin zones. First, we test our implementation against tabulated results available for standard high-symmetry points (given by universal fractional coordinates). Then, to showcase the general applicability of our methodology, we produce the irreducible representations for the ``non-universal high-symmetry" points, first reported by Setyawan and Curtarolo [Comput. Mater. Sci. 49, 299 (2010)]. The present method can be regarded as a first step for the determination of elementary band decompositions and symmetry-enforced constraints in crystalline topological materials., Comment: 34 pages, 10 figures, 43 tables

The main question this paper addresses is: What combination of a robot controller and a learning method should be used, if the morphology of the learning robot is not known in advance? Our interest is rooted in the context of morphologically evolving modular robots, but the question is also relevant in general, for system designers interested in widely applicable solutions. We perform an experimental comparison of three controller-and-learner combinations: one approach where controllers are based on modelling animal locomotion (Central Pattern Generators, CPG) and the learner is an evolutionary algorithm, a completely different method using Reinforcement Learning (RL) with a neural network controller architecture, and a combination `in-between' where controllers are neural networks and the learner is an evolutionary algorithm. We apply these three combinations to a test suite of modular robots and compare their efficacy, efficiency, and robustness. Surprisingly, the usual CPG-based and RL-based options are outperformed by the in-between combination that is more robust and efficient than the other two setups.

Evolutionary robot systems offer two principal advantages: an advanced way of developing robots through evolutionary optimization and a special research platform to conduct what-if experiments regarding questions about evolution. Our study sits at the intersection of these. We investigate the question ``What if the 18th-century biologist Lamarck was not completely wrong and individual traits learned during a lifetime could be passed on to offspring through inheritance?'' We research this issue through simulations with an evolutionary robot framework where morphologies (bodies) and controllers (brains) of robots are evolvable and robots also can improve their controllers through learning during their lifetime. Within this framework, we compare a Lamarckian system, where learned bits of the brain are inheritable, with a Darwinian system, where they are not. Analyzing simulations based on these systems, we obtain new insights about Lamarckian evolution dynamics and the interaction between evolution and learning. Specifically, we show that Lamarckism amplifies the emergence of `morphological intelligence', the ability of a given robot body to acquire a good brain by learning, and identify the source of this success: `newborn' robots have a higher fitness because their inherited brains match their bodies better than those in a Darwinian system., Comment: preprint-nature scientific report. arXiv admin note: text overlap with arXiv:2303.12594

Ce$_{3}$Bi$_{4}$Au$_{3}$ crystallizes in the same non-centrosymmetric cubic structure as the prototypical Kondo insulator Ce$_{3}$Bi$_{4}$Pt$_{3}$. Here we report the physical properties of Ce$_{3}$Bi$_{4}$Au$_{3}$ single crystals using magnetization, thermodynamic, and electrical-transport measurements. Magnetic-susceptibility and heat-capacity data reveal antiferromagnetic (AFM) order below $T_N=3.2$ K. The magnetic entropy $S_{\rm mag}$ reaches $R$ln2 slightly above $T_N$, which suggests localized $4f$-moments in a doublet ground state. Multiple field-induced magnetic transitions are observed at temperatures below $T_N$, which indicate a complex spin structure with competing interactions. Ce$_{3}$Bi$_{4}$Au$_{3}$ shows semimetallic behavior in electrical resistivity measurements in contrast to the majority of reported Cerium-based 343 compounds. Electrical-resistivity measurements under hydrostatic pressure reveal a slight enhancement of $T_N$ under pressures up to 2.3 GPa, which supports a scenario wherein Ce$_{3}$Bi$_{4}$Au$_{3}$ belongs to the far left of the Doniach phase diagram dominated by Ruderman-Kittel-Kasuya-Yosida (RKKY) interactions. Using realistic many-body simulations, we confirm the semi-metallic electronic structure of Ce$_{3}$Bi$_{4}$Au$_{3}$ and quantitatively reproduce its local moment behavior in the paramagnetic state., Comment: 10 pages, 10 figures

Finding a material that turns superconducting under ambient conditions has been the goal of over a century of research, and recently Pb$_{10-x}$Cu$_x$(PO$_4$)$_6$O aka LK-99 has been put forward as a possible contestant. In this work, we study the possibility of electronically driven superconductivity in LK-99 also allowing for electron or hole doping. We use an $\textit{ab initio}$ derived two-band model of the Cu $e_g$ orbitals for which we determine interaction values from the constrained random phase approximation (cRPA). For this two-band model we perform calculations in the fluctuation exchange (FLEX) approach to assess the strength of orbital and spin fluctuations. We scan over a broad range of parameters and enforce no magnetic or orbital symmetry breaking. Even under optimized conditions for superconductivity, spin and orbital fluctuations turn out to be too weak for superconductivity anywhere near to room-temperature. We contrast this finding to non-self-consistent RPA, where it is possible to induce spin-singlet $d$-wave superconductivity at $T_{\mathrm{c}}\geq300$ K if the system is put close enough to a magnetic instability., Comment: 14 pages, 3 figures; revised submission to SciPost Physics

We briefly review the status quo of research on the putative superconductor Pb$_9$Cu(PO$_4$)$_6$O also known as LK-99. Further, we provide {\em ab initio} derived tight-binding parameters for a two- and five-band model, and solve these in dynamical-mean-field theory. The ratio interaction-to-bandwidth makes LK-99 a Mott or charge transfer insulator. Electron or hole doping (which is different from substituting Pb by Cu and thus differs from LK-99) is required to make it metallic and potentially superconducting., Comment: 7 figures, 9 pages, 2 tables. Version 2 is the same as version 1 except for additional DMFT and DFT+U results in Secs. IV and V

As a follow-up to our previous work on particle acceleration simulation near neutron stars, in this paper, we discuss the impact of radiation reaction on test particles injected into their magnetosphere. We therefore neglect the interaction between particles through the electromagnetic field as well as gravitation. We integrate numerically the reduced Landau-Lifshitz equation for electrons and protons in the vacuum field of a rotating magnetic dipole based on analytical solutions in a constant electromagnetic field. These expressions are simple in a frame where the electric and magnetic field are parallel. Lorentz transforms are used to switch back and forth between this frame and the observer frame. We found that, though due solely to the Lorentz force, electrons reach Lorentz factors up to $\gamma=10^{14}$ and protons reach them up to $\gamma=10^{10.7}$. When radiation reaction is enabled, electrons reach energies up to $\gamma=10^{10.5}$ and protons reach energies up to $\gamma=10^{8.3}$. The second set of values are more realistic since the radiation reaction feedback is predominant within the magnetosphere. Moreover, as expected, symmetrical behaviours between the north and south hemispheres are highlighted, either with respect to the location around the neutron star or with respect to particles of opposite charge to mass ratio~$(q/m)$. The study of the influence of the magnetic dipolar moment inclination shows similar behaviours regardless of whether radiation reaction is enabled. Protons (respectively electrons) impact the surface of the neutron star less as the inclination angle increases (decreases for electrons), while if the rotation and magnetic axes are aligned, all the protons impact the neutron star, and all the electrons impact the surface if the rotation and magnetic axes are anti-aligned., Comment: Accepted for publication in A&A; revised version after language editing

Magnetic frustration is a route that can lead to the emergence of novel ground states, including spin liquids and spin ices. Such frustration can be introduced through either the geometry of lattice structures or by incompatible exchange interactions. Identifying suitable strategies to control the degree of magnetic frustration in real systems is an active field of research. In this study, we devise a design principle for the tuning of frustrated magnetism on the square lattice through the manipulation of nearest (NN) and next-nearest neighbor (NNN) antiferromagnetic (AF) exchange interactions. By studying the magnon excitations in epitaxially-strained La$_2$NiO$_4$ films using resonant inelastic x-ray scattering (RIXS) we show that, in contrast to the cuprates, the dispersion peaks at the AF zone boundary. This indicates the presence of an AF-NNN spin interaction. Using first principles simulations and an effective spin-model, we demonstrate the AF-NNN coupling to be a consequence of the two-orbital nature of La$_2$NiO$_4$. Our results demonstrate that compressive strain can enhance this coupling, providing a design principle for the tunability of frustrated magnetism on a square lattice.

Adam Smith developed a version of moral philosophy where better decisions are made by interrogating an impartial spectator within us. We discuss the possibility of using an external non-human-based substitute tool that would augment our internal mental processes and play the role of the impartial spectator. Such tool would have more knowledge about the world, be more impartial, and would provide a more encompassing perspective on moral assessment.

Diffusion models have achieved remarkable success in generating high-quality images thanks to their novel training procedures applied to unprecedented amounts of data. However, training a diffusion model from scratch is computationally expensive. This highlights the need to investigate the possibility of training these models iteratively, reusing computation while the data distribution changes. In this study, we take the first step in this direction and evaluate the continual learning (CL) properties of diffusion models. We begin by benchmarking the most common CL methods applied to Denoising Diffusion Probabilistic Models (DDPMs), where we note the strong performance of the experience replay with the reduced rehearsal coefficient. Furthermore, we provide insights into the dynamics of forgetting, which exhibit diverse behavior across diffusion timesteps. We also uncover certain pitfalls of using the bits-per-dimension metric for evaluating CL.

Hierarchical Variational Autoencoders (VAEs) are among the most popular likelihood-based generative models. There is a consensus that the top-down hierarchical VAEs allow effective learning of deep latent structures and avoid problems like posterior collapse. Here, we show that this is not necessarily the case, and the problem of collapsing posteriors remains. To discourage this issue, we propose a deep hierarchical VAE with a context on top. Specifically, we use a Discrete Cosine Transform to obtain the last latent variable. In a series of experiments, we observe that the proposed modification allows us to achieve better utilization of the latent space and does not harm the model's generative abilities., Comment: Code: https://github.com/AKuzina/dct_vae