Search

We develop both first and second order numerical optimization methods to solve non-smooth optimization problems featuring a shared sparsity penalty, constrained by differential equations with uncertainty. To alleviate the curse of dimensionality we use tensor product approximations. To handle the non-smoothness of the objective function we introduce a smoothed version of the shared sparsity objective. We consider both a benchmark elliptic PDE constraint, and a more realistic topology optimization problem. We demonstrate that the error converges linearly in iterations and the smoothing parameter, and faster than algebraically in the number of degrees of freedom, consisting of the number of quadrature points in one variable and tensor ranks. Moreover, in the topology optimization problem, the smoothed shared sparsity penalty actually reduces the tensor ranks compared to the unpenalised solution. This enables us to find a sparse high-resolution design under a high-dimensional uncertainty.

The origin of the ultra high energy cosmic rays via annihilation of heavy stable, fermions "f", of the cosmological dark matter (DM) is studied. The particles in question are supposed to be created by the scalaron decays in $R^2$ modified gravity. Novel part of our approach is the assumption that the mass of these carriers of DM is slightly below than a half of the scalaron mass. In such a case the phase space volume becomes tiny. It leads to sufficiently low probability of "f" production, so their average cosmological energy density could be equal to the observed energy density of dark matter. Several regions of the universe, where the annihilation could take place, are studied. They include the whole universe under assumption of homogeneous energy density, the high density DM clump in the galactic centre, the cloud of DM in the Galaxy with realistic density distribution, and high density clusters of DM in the Galaxy. Possible resonance annihilation of $f \bar f$ into energetic light particle is considered. We have shown that the proposed scenario can successfully explain the origin of the ultrahigh energy flux of cosmic rays where the canonical astrophysical mechanisms are not operative., Comment: 18 pages, 3 figures

Running quantum algorithms often involves implementing complex quantum circuits with such a large number of multi-qubit gates that the challenge of tackling practical applications appears daunting. To date, no experiments have successfully demonstrated a quantum advantage due to the ease with which the results can be adequately replicated on classical computers through the use of tensor network algorithms. Additionally, it remains unclear even in theory where exactly these advantages are rooted within quantum systems because the logarithmic complexity commonly associated with quantum algorithms is also present in algorithms based on tensor networks. In this article, we propose a novel approach called Tensor Quantum Programming, which leverages tensor networks for hybrid quantum computing. Our key insight is that the primary challenge of algorithms based on tensor networks lies in their high ranks (bond dimensions). Quantum computing offers a potential solution to this challenge, as an ideal quantum computer can represent tensors with arbitrarily high ranks in contrast to classical counterparts, which indicates the way towards quantum advantage. While tensor-based vector-encoding and state-readout are known procedures, the matrix-encoding required for performing matrix-vector multiplications directly on quantum devices remained unsolved. Here, we developed an algorithm that encodes Matrix Product Operators into quantum circuits with a depth that depends linearly on the number of qubits. It demonstrates effectiveness on up to 50 qubits for several matrices frequently encountered in differential equations, optimization problems, and quantum chemistry. We view this work as an initial stride towards the creation of genuinely practical quantum algorithms., Comment: 17 pages, 13 figures

We introduce TQCompressor, a novel method for neural network model compression with improved tensor decompositions. We explore the challenges posed by the computational and storage demands of pre-trained language models in NLP tasks and propose a permutation-based enhancement to Kronecker decomposition. This enhancement makes it possible to reduce loss in model expressivity which is usually associated with factorization. We demonstrate this method applied to the GPT-2$_{small}$. The result of the compression is TQCompressedGPT-2 model, featuring 81 mln. parameters compared to 124 mln. in the GPT-2$_{small}$. We make TQCompressedGPT-2 publicly available. We further enhance the performance of the TQCompressedGPT-2 through a training strategy involving multi-step knowledge distillation, using only a 3.1% of the OpenWebText. TQCompressedGPT-2 surpasses DistilGPT-2 and KnGPT-2 in comparative evaluations, marking an advancement in the efficient and effective deployment of models in resource-constrained environments.

We consider a problem of inferring contact network from nodal states observed during an epidemiological process. In a black--box Bayesian optimisation framework this problem reduces to a discrete likelihood optimisation over the set of possible networks. The cardinality of this set grows combinatorially with the number of network nodes, which makes this optimisation computationally challenging. For each network, its likelihood is the probability for the observed data to appear during the evolution of the epidemiological process on this network. This probability can be very small, particularly if the network is significantly different from the ground truth network, from which the observed data actually appear. A commonly used stochastic simulation algorithm struggles to recover rare events and hence to estimate small probabilities and likelihoods. In this paper we replace the stochastic simulation with solving the chemical master equation for the probabilities of all network states. Since this equation also suffers from the curse of dimensionality, we apply tensor train approximations to overcome it and enable fast and accurate computations. Numerical simulations demonstrate efficient black--box Bayesian inference of the network.

The CVAE is one of the most widely-used models in trajectory prediction for AD. It captures the interplay between a driving context and its ground-truth future into a probabilistic latent space and uses it to produce predictions. In this paper, we challenge key components of the CVAE. We leverage recent advances in the space of the VAE, the foundation of the CVAE, which show that a simple change in the sampling procedure can greatly benefit performance. We find that unscented sampling, which draws samples from any learned distribution in a deterministic manner, can naturally be better suited to trajectory prediction than potentially dangerous random sampling. We go further and offer additional improvements including a more structured Gaussian mixture latent space, as well as a novel, potentially more expressive way to do inference with CVAEs. We show wide applicability of our models by evaluating them on the INTERACTION prediction dataset, outperforming the state of the art, as well as at the task of image modeling on the CelebA dataset, outperforming the baseline vanilla CVAE. Code is available at https://github.com/boschresearch/cuae-prediction.

The suppression of relic gravitational waves due to their conversion into electromagnetic radiation in a cosmological magnetic field is studied. The coupled system of equations describing gravitational and electromagnetic wave propagation in an arbitrary curved space-time and in external magnetic field is derived. The subsequent elimination of photons from the beam due to their interaction with the primary plasma is taken into account. The resulting system of equations is solved numerically in Friedman-LeMaitre-Robertson-Walker metric for the upper limit of the intergalactic magnetic field strength of 1 nGs. We conclude that the gravitational wave conversion into photons in the intergalactic magnetic field cannot significantly change the amplitude of the relic gravitational wave and their frequency spectrum., Comment: 38 pages, 2 figures

Recent data released by James Webb Space Telescope (JWST) and, somewhat earlier, the data presented by Hubble Space Telescope (HST) are commonly understood as a strong indication for breaking of the canonical $\Lambda$CDM cosmology. It is argued in the presented work that massive primordial black holes (PBH) could seed galaxy and quasar formation in the very young universe as it has been conjectured in our paper of 1993 and resolve the tension induced by the JWST and the HST data with the standard cosmology. This point of view is presently supported by several recent works. The proposed mechanism of PBH formation leads to the log-normal mass spectrum of PBHs and predicts abundant antimatter population of our Galaxy, Milky Way. Both these predictions are in excellent agreement with astronomical observations., Comment: 9 figures, 17 pages, on the basis of plenary talk at Frascati workshop 2023 on Multifrequency Behaviour of High Energy Cosmic Sources - XIV, to be published in the proceedings

The conversion of protons to positrons at the horizon of a black hole (BH) is considered. It is shown that the process may efficiently proceed for BHs with masses in the range $\sim 10^{18}$ -- $10^{21}$ g. It is argued that the electric charge of BH acquired by the proton accretion to BH could create electric field near BH horizon close to the critical Schwinger one. It leads to efficient electron-positron pair production, when electrons are back captured by the BH while positrons are emitted into outer space. Annihilation of these positrons with electrons in the interstellar medium may at least partially explain the origin of the observed 511 keV line., Comment: 15 pages, 3 figures

The Variational Autoencoder (VAE) is a seminal approach in deep generative modeling with latent variables. Interpreting its reconstruction process as a nonlinear transformation of samples from the latent posterior distribution, we apply the Unscented Transform (UT) -- a well-known distribution approximation used in the Unscented Kalman Filter (UKF) from the field of filtering. A finite set of statistics called sigma points, sampled deterministically, provides a more informative and lower-variance posterior representation than the ubiquitous noise-scaling of the reparameterization trick, while ensuring higher-quality reconstruction. We further boost the performance by replacing the Kullback-Leibler (KL) divergence with the Wasserstein distribution metric that allows for a sharper posterior. Inspired by the two components, we derive a novel, deterministic-sampling flavor of the VAE, the Unscented Autoencoder (UAE), trained purely with regularization-like terms on the per-sample posterior. We empirically show competitive performance in Fr\'echet Inception Distance (FID) scores over closely-related models, in addition to a lower training variance than the VAE.

Accurate vehicle trajectory prediction is an unsolved problem in autonomous driving with various open research questions. State-of-the-art approaches regress trajectories either in a one-shot or step-wise manner. Although one-shot approaches are usually preferred for their simplicity, they relinquish powerful self-supervision schemes that can be constructed by chaining multiple time-steps. We address this issue by proposing a middle-ground where multiple trajectory segments are chained together. Our proposed Multi-Branch Self-Supervised Predictor receives additional training on new predictions starting at intermediate future segments. In addition, the model 'imagines' the latent context and 'predicts the past' while combining multi-modal trajectories in a tree-like manner. We deliberately keep aspects such as interaction and environment modeling simplistic and nevertheless achieve competitive results on the INTERACTION dataset. Furthermore, we investigate the sparsely explored uncertainty estimation of deterministic predictors. We find positive correlations between the prediction error and two proposed metrics, which might pave way for determining prediction confidence., Comment: 8 pages, 6 figures, to be published in 34th IEEE Intelligent Vehicles Symposium (IV)

Metric perturbations in General Relativity are usually separated into three distinct classes: scalar, vector, and tensor. In many cases these modes are separable, i.e. they satisfy independent equations of motion for each mode. However, in the present paper we argue that in many cases tensor and scalar modes are not separable, no matter what gauge conditions are chosen. The propagation of any of these mode depends on the other. A realistic example providing such mixing is presented., Comment: 10 pages

We challenge the perceived consensus that the application of deep learning to solve the automated driving planning task necessarily requires huge amounts of real-world data or highly realistic simulation. Focusing on a roundabout scenario, we show that this requirement can be relaxed in favour of targeted, simplistic simulated data. A benefit is that such data can be easily generated for critical scenarios that are typically underrepresented in realistic datasets. By applying vanilla behavioural cloning almost exclusively to lightweight simulated data, we achieve reliable and comfortable driving in a real-world test vehicle. We leverage an incremental development approach that includes regular in-vehicle testing to identify sim-to-real gaps, targeted data augmentation, and training scenario variations. In addition to a detailed description of the methodology, we share our lessons learned, touching upon scenario generation, simulation features, and evaluation metrics.

The solution of computational fluid dynamics problems is one of the most computationally hard tasks, especially in the case of complex geometries and turbulent flow regimes. We propose to use Tensor Train (TT) methods, which possess logarithmic complexity in problem size and have great similarities with quantum algorithms in the structure of data representation. We develop the Tensor train Finite Element Method -- TetraFEM -- and the explicit numerical scheme for the solution of the incompressible Navier-Stokes equation via Tensor Trains. We test this approach on the simulation of liquids mixing in a T-shape mixer, which, to our knowledge, was done for the first time using tensor methods in such non-trivial geometries. As expected, we achieve exponential compression in memory of all FEM matrices and demonstrate an exponential speed-up compared to the conventional FEM implementation on dense meshes. In addition, we discuss the possibility of extending this method to a quantum computer to solve more complex problems. This paper is based on work we conducted for Evonik Industries AG.

Multidimensional modification of gravity with a smaller mass scale of the gravitational interaction is considered. Stable by assumption dark matter particles could decay via interactions with virtual black holes. The decay rates of such processes are estimated. It is shown that with the proper fixation of the parameters the decays of these ultra-massive particles can give noticeable contribution to the flux of high energy cosmic rays in particular, near the Greisen-Zatsepin-Kuzmin limit. Such particles can also create neutrinos of very high energies observed in the existing huge underwater or ice-cube detectors., Comment: 8 pages, 3 figures. References are updated

A minor population of antistars in galaxies has been predicted by some of non-standard models of baryogenesis and nucleosynthesis in the early Universe, and their presence is not yet excluded by the currently available observations. Detection of an unusually high abundance of antinuclei in cosmic rays can probe the baryogenesis scenarios in the early Universe. Recent report of the \textit{AMS-02} collaboration on the tentative detection of a few antihelium nuclei in GeV cosmic rays provided a great hope on the progress in this issue. We discuss possible sources of antinuclei in cosmic rays from antistars which are predicted in a modified Affleck-Dine baryogenesis scenario by Dolgov and Silk (1993). The model allows us to estimate the expected fluxes and isotopic content of antinuclei in the GeV cosmic rays produced in scenarios involving antistars. We show that the flux of antihelium CRs reported by the \textit{AMS-02} experiment can be explained by Galactic anti-nova outbursts, thermonuclear anti-SN Ia explosions, a collection of flaring antistars or an extragalactic source with abundances not violating existing gamma-ray and microlensing constraints on the antistar population., Comment: 15 pages, revised version accepted for publication in JCAP

Transport map methods offer a powerful statistical learning tool that can couple a target high-dimensional random variable with some reference random variable using invertible transformations. This paper presents new computational techniques for building the Knothe--Rosenblatt (KR) rearrangement based on general separable functions. We first introduce a new construction of the KR rearrangement -- with guaranteed invertibility in its numerical implementation -- based on approximating the density of the target random variable using tensor-product spectral polynomials and downward closed sparse index sets. Compared to other constructions of KR arrangements based on either multi-linear approximations or nonlinear optimizations, our new construction only relies on a weighted least square approximation procedure. Then, inspired by the recently developed deep tensor trains (Cui and Dolgov, Found. Comput. Math. 22:1863--1922, 2022), we enhance the approximation power of sparse polynomials by preconditioning the density approximation problem using compositions of maps. This is particularly suitable for high-dimensional and concentrated probability densities commonly seen in many applications. We approximate the complicated target density by a composition of self-reinforced KR rearrangements, in which previously constructed KR rearrangements -- based on the same approximation ansatz -- are used to precondition the density approximation problem for building each new KR rearrangement. We demonstrate the efficiency of our proposed methods and the importance of using the composite map on several inverse problems governed by ordinary differential equations (ODEs) and partial differential equations (PDEs).

High-dimensional data in the form of tensors are challenging for kernel classification methods. To both reduce the computational complexity and extract informative features, kernels based on low-rank tensor decompositions have been proposed. However, what decisive features of the tensors are exploited by these kernels is often unclear. In this paper we propose a novel kernel that is based on the Tucker decomposition. For this kernel the Tucker factors are computed based on re-weighting of the Tucker matrices with tuneable powers of singular values from the HOSVD decomposition. This provides a mechanism to balance the contribution of the Tucker core and factors of the data. We benchmark support tensor machines with this new kernel on several datasets. First we generate synthetic data where two classes differ in either Tucker factors or core, and compare our novel and previously existing kernels. We show robustness of the new kernel with respect to both classification scenarios. We further test the new method on real-world datasets. The proposed kernel has demonstrated a higher test accuracy than the state-of-the-art tensor train multi-way multi-level kernel, and a significantly lower computational time.

We propose an algorithm to solve optimization problems constrained by partial (ordinary) differential equations under uncertainty, with almost sure constraints on the state variable. To alleviate the computational burden of high-dimensional random variables, we approximate all random fields by the tensor-train decomposition. To enable efficient tensor-train approximation of the state constraints, the latter are handled using the Moreau-Yosida penalty, with an additional smoothing of the positive part (plus/ReLU) function by a softplus function. In a special case of a quadratic cost minimization constrained by linear elliptic partial differential equations, and some additional constraint qualification, we prove strong convergence of the regularized solution to the optimal control. This result also proposes a practical recipe for selecting the smoothing parameter as a function of the penalty parameter. We develop a second order Newton type method with a fast matrix-free action of the approximate Hessian to solve the smoothed Moreau-Yosida problem. This algorithm is tested on benchmark elliptic problems with random coefficients, optimization problems constrained by random elliptic variational inequalities, and a real-world epidemiological model with 20 random variables. These examples demonstrate mild (at most polynomial) scaling with respect to the dimension and regularization parameters., Comment: 28 pages

The coupling of baryonic current to the derivative of the curvature scalar, $R$, inherent to gravitational baryogenesis (GBG), leads to a fourth order differential equation of motion for $R$ instead of the algebraic one of General Relativity (GR). The fourth-order differential equation is generically unstable. We consider a possible mechanism of stabilization of GBG by modification of gravity, introducing an $R^2$-term into the canonical action of GR. It is shown that this mechanism allows for stabilization of GBG with bosonic and fermionic baryon currents. We have established the region of the model parameters leading to stabilization of $R$. Still, the standard cosmology would be noticeably modified., Comment: 11 pages, no figures, presented at $6^{th}$ International Conference on Particle Physics and Astrophysics (ICCPA-2022)}. arXiv admin note: text overlap with arXiv:1808.08577

It is argued that the data presented by Hubble Space Telescope and James Webb Space Telescope, that seem to be at odds with the canonical big bang cosmology, find simple explanation if galaxy formation is seeded by massive primordial black holes (PBH), as anticipated in 1993 (A. Dolgov and J. Silk, later DS). The statement that the galaxy formation might be seeded by PBH is now rediscovered in several works. The predicted by DS log-normal mass spectrum of PBHs very well agrees with astronomical data. Abundant BH population of the Galaxy with masses of the order of tens solar masses is predicted. Extended mass spectrum of PBH together with their possible clustering allows them to make 100\% contribution into the cosmological dark matter. Another prediction of DS mechanism on noticeable amount of antimatter in the Milky Way also seems to be confirmed by the data., Comment: 23 pages, 10 figures, plenary talk at Plenary talk presented at 36th Rencontres de Physique de la Vall{\'e}e d'Aoste on Results and Perspectives in Particle Physics, extended and updated

We propose a deep importance sampling method that is suitable for estimating rare event probabilities in high-dimensional problems. We approximate the optimal importance distribution in a general importance sampling problem as the pushforward of a reference distribution under a composition of order-preserving transformations, in which each transformation is formed by a squared tensor-train decomposition. The squared tensor-train decomposition provides a scalable ansatz for building order-preserving high-dimensional transformations via density approximations. The use of composition of maps moving along a sequence of bridging densities alleviates the difficulty of directly approximating concentrated density functions. To compute expectations over unnormalized probability distributions, we design a ratio estimator that estimates the normalizing constant using a separate importance distribution, again constructed via a composition of transformations in tensor-train format. This offers better theoretical variance reduction compared with self-normalized importance sampling, and thus opens the door to efficient computation of rare event probabilities in Bayesian inference problems. Numerical experiments on problems constrained by differential equations show little to no increase in the computational complexity with the event probability going to zero, and allow to compute hitherto unattainable estimates of rare event probabilities for complex, high-dimensional posterior densities.

To improve mathematical models of epidemics it is essential to move beyond the traditional assumption of homogeneous well--mixed population and involve more precise information on the network of contacts and transport links by which a stochastic process of the epidemics spreads. In general, the number of states of the network grows exponentially with its size, and a master equation description suffers from the curse of dimensionality. Almost all methods widely used in practice are versions of the stochastic simulation algorithm (SSA), which is notoriously known for its slow convergence. In this paper we numerically solve the chemical master equation for an SIR model on a general network using recently proposed tensor product algorithms. In numerical experiments we show that tensor product algorithms converge much faster than SSA and deliver more accurate results, which becomes particularly important for uncovering the probabilities of rare events, e.g. for number of infected people to exceed a (high) threshold.

Combinations of spectroscopic analysis and microscopic techniques are used across many disciplines of scientific research, including material science, chemistry and biology. X-ray spectromicroscopy, in particular, is a powerful tool used for studying chemical state distributions at the micro and nano scales. With the beam fixed, a specimen is typically rastered through the probe with continuous motion and a range of multimodal data is collected at fixed time intervals. The application of this technique is limited in some areas due to: long scanning times to collect the data, either because of the area/volume under study or the compositional properties of the specimen; and material degradation due to the dose absorbed during the measurement. In this work, we propose a novel approach for reducing the dose and scanning times by undersampling the raster data. This is achieved by skipping rows within scans and reconstructing the x-ray spectromicroscopic measurements using low-rank matrix completion. The new method is robust and allows for x 5-6 reduction in sampling. Experimental results obtained on real data are illustrated., Comment: 22 pages,11 figures

Universe heating in $R^2$-modified gravity is considered. The rates of particle production by the scalaron are calculated for different decay channels. Freezing of massive stable relics with the interaction strength typical for supersymmetry is studied. It is shown that the bounds on masses of supersymmetry-kind particles allowing them to form the cosmological dark matter (DM)depend upon the dominant decay mode of the scalaron. In any case the results presented open much wider mass window for DM with the interaction strength typical for supersymmetry., Comment: 21 pages, 4 figures, 1 table. Extended version of the invited talk presented at International Workshop on Elementary Particles and Nuclear Physics, 24 -- 30 April 2022, Institute of Nuclear Physics of the Ministry of Energy of the Republic of Kazakhstan, Almaty

Astronomical data of the several recent years, which present an evidence in favour of abundant antimatter population in our Galaxy, Milky Way, are analysed. The data include: registration of gamma-rays with energy 0.511 MeV, which surely originate from electron-positron annihilation at rest, very large flux of anti-helium nuclei, discovered at AMS, and 14 stars which produce excessive gamma-rays with energies of several hundred MeV which may be interpreted as indication that these stars consist of antimatter. Theoretical predictions of these phenomena, made much earlier ago are described, Comment: 10 pages, 3 figures, invited talk at XXXV edition of Les Rencontres de Physique de La Vell{'e} d'Aoste,March 6-12, 2022