Search

In this work, we propose a novel sampling method for Design of Experiments. This method allows to sample such input values of the parameters of a computational model for which the constructed surrogate model will have the least possible approximation error. High efficiency of the proposed method is demonstrated by its comparison with other sampling techniques (LHS, Sobol' sequence sampling, and Maxvol sampling) on the problem of least-squares polynomial approximation. Also, numerical experiments for the Lebesgue constant growth for the points sampled by the proposed method are carried out., Comment: 17 pages

Iterative Closest Point (ICP) is a widely used method for performing scan-matching and registration. Being simple and robust method, it is still computationally expensive and may be challenging to use in real-time applications with limited resources on mobile platforms. In this paper we propose novel effective method for acceleration of ICP which does not require substantial modifications to the existing code. This method is based on an idea of Anderson acceleration which is an iterative procedure for finding a fixed point of contractive mapping. The latter is often faster than a standard Picard iteration, usually used in ICP implementations. We show that ICP, being a fixed point problem, can be significantly accelerated by this method enhanced by heuristics to improve overall robustness. We implement proposed approach into Point Cloud Library (PCL) and make it available online. Benchmarking on real-world data fully supports our claims.

Part 2 of this monograph builds on the introduction to tensor networks and their operations presented in Part 1. It focuses on tensor network models for super-compressed higher-order representation of data/parameters and related cost functions, while providing an outline of their applications in machine learning and data analytics. A particular emphasis is on the tensor train (TT) and Hierarchical Tucker (HT) decompositions, and their physically meaningful interpretations which reflect the scalability of the tensor network approach. Through a graphical approach, we also elucidate how, by virtue of the underlying low-rank tensor approximations and sophisticated contractions of core tensors, tensor networks have the ability to perform distributed computations on otherwise prohibitively large volumes of data/parameters, thereby alleviating or even eliminating the curse of dimensionality. The usefulness of this concept is illustrated over a number of applied areas, including generalized regression and classification (support tensor machines, canonical correlation analysis, higher order partial least squares), generalized eigenvalue decomposition, Riemannian optimization, and in the optimization of deep neural networks. Part 1 and Part 2 of this work can be used either as stand-alone separate texts, or indeed as a conjoint comprehensive review of the exciting field of low-rank tensor networks and tensor decompositions., Comment: 232 pages

Topological optimization finds a material density distribution minimizing a functional of the solution of a partial differential equation (PDE), subject to a set of constraints (typically, a bound on the volume or mass of the material). Using a finite elements discretization (FEM) of the PDE and functional we obtain an integer programming problem. Due to approximation error of FEM discretization, optimization problem becomes mesh-depended and possess false, physically inadequate optimums, while functional value heavily depends on fineness of discretization scheme used to compute it. To alleviate this problem, we propose regularization of given functional by error estimate of FEM discretization. This regularization provides robustness of solutions and improves obtained functional values as well. While the idea is broadly applicable, in this paper we apply our method to the heat conduction optimization. This type of problems are of practical importance in design of heat conduction channels, heat sinks and other types of heat guides.

In this work we propose an efficient black-box solver for two-dimensional stationary diffusion equations, which is based on a new robust discretization scheme. The idea is to formulate an equation in a certain form without derivatives with a non-local stencil, which leads us to a linear system of equations with dense matrix. This matrix and a right-hand side are represented in a low-rank parametric representation -- the quantized tensor train (QTT-) format, and then all operations are performed with logarithmic complexity and memory consumption. Hence very fine grids can be used, and very accurate solutions with extremely high spatial resolution can be obtained. Numerical experiments show that this formulation gives accurate results and can be used up to $2^{60}$ grid points with no problems with conditioning, while total computational time is around several seconds.

Machine learning and data mining algorithms are becoming increasingly important in analyzing large volume, multi-relational and multi--modal datasets, which are often conveniently represented as multiway arrays or tensors. It is therefore timely and valuable for the multidisciplinary research community to review tensor decompositions and tensor networks as emerging tools for large-scale data analysis and data mining. We provide the mathematical and graphical representations and interpretation of tensor networks, with the main focus on the Tucker and Tensor Train (TT) decompositions and their extensions or generalizations. Keywords: Tensor networks, Function-related tensors, CP decomposition, Tucker models, tensor train (TT) decompositions, matrix product states (MPS), matrix product operators (MPO), basic tensor operations, multiway component analysis, multilinear blind source separation, tensor completion, linear/multilinear dimensionality reduction, large-scale optimization problems, symmetric eigenvalue decomposition (EVD), PCA/SVD, huge systems of linear equations, pseudo-inverse of very large matrices, Lasso and Canonical Correlation Analysis (CCA) (This is Part 1), Comment: 176 pages

We present a method for solving the reaction-diffusion equation with general potential in free space. It is based on the approximation of the Feynman-Kac formula by a sequence of convolutions on sequentially diminishing grids. For computation of the convolutions we propose a fast algorithm based on the low-rank approximation of the Hankel matrices. The algorithm has complexity of $\mathcal{O}(nr M \log M + nr^2 M)$ flops and requires $\mathcal{O}(M r)$ floating-point numbers in memory, where $n$ is the dimension of the integral, $r \ll n$, and $M$ is the mesh size in one dimension. The presented technique can be generalized to the higher-order diffusion processes.

An efficient technique based on low-rank separated approximations is proposed for computation of three-dimensional integrals arising in the energy deposition model that describes ion-atomic collisions. Direct tensor-product quadrature requires grids of size $4000^3$ which is unacceptable. Moreover, several of such integrals have to be computed simultaneously for different values of parameters. To reduce the complexity, we use the structure of the integrand and apply numerical linear algebra techniques for the construction of low-rank approximation. The resulting algorithm is $10^3$ faster than spectral quadratures in spherical coordinates used in the original DEPOSIT code. The approach can be generalized to other multidimensional problems in physics., Comment: arXiv admin note: text overlap with arXiv:1403.4068

We introduce a definition of the volume for a general rectangular matrix, which for square matrices is equivalent to the absolute value of the determinant. We generalize results for square maximum-volume submatrices to the case of rectangular maximal-volume submatrices, show connection of the rectangular volume with optimal experimental design and provide estimates for the growth of the coefficients and approximation error in spectral and Chebyshev norms. Three promising applications of such submatrices are presented: recommender systems, finding maximal elements in low-rank matrices and preconditioning of overdetermined linear systems. The code is available online., Comment: 29 pages, 1 figure, 3 tables, submitted to Linear Algebra and its Applications

This paper presents a deterministic linear time complexity IDE-SimRank method to approximately compute SimRank with proved error bound. SimRank is a well-known similarity measure between graph vertices which relies on graph topology only and is built on intuition that "two objects are similar if they are related to similar objects". The fixed point equation for direct SimRank computation is the discrete Lyapunov equation with specific diagonal matrix in the right hand side. The proposed method is based on estimation of this diagonal matrix with GMRES and use this estimation to compute singe-source and single pairs queries. These computations are executed with the part of series converging to the discrete Lyapunov equation solution.

Due to popularity surge social networks became lucrative targets for spammers and guerilla marketers, who are trying to game ranking systems and broadcast their messages at little to none cost. Ranking systems, for example Twitter's Trends, can be gamed by scripted users also called bots, who are automatically or semi-automatically twitting essentially the same message. Judging by the prices and abundance of supply from PR firms this is an easy to implement and widely used tactic, at least in Russian blogosphere. Aggregative analysis of social networks should at best mark those messages as spam or at least correctly downplay their importance as they represent opinions only of a few, if dedicated, users. Hence bot detection plays a crucial role in social network mining and analysis. In this paper we propose technique called RepRank which could be viewed as Markov chain based model for reputation propagation on graphs utilizing simultaneous trust and anti-trust propagation and provide effective numerical approach for its computation. Comparison with another models such as TrustRank and some of its modifications on sample of 320000 Russian speaking Twitter users is presented. The dataset is presented as well.

We propose a new method for the approximate solution of the Lyapunov equation with rank-$1$ right-hand side, which is based on extended rational Krylov subspace approximation with adaptively computed shifts. The shift selection is obtained from the connection between the Lyapunov equation, solution of systems of linear ODEs and alternating least squares method for low-rank approximation. The numerical experiments confirm the effectiveness of our approach., Comment: 17 pages, 1 figures

SimRank is a well-known similarity measure between graph vertices. In this paper novel low-rank approximation of SimRank is proposed.

We propose a new cross-conv algorithm for approximate computation of convolution in different low-rank tensor formats (tensor train, Tucker, Hierarchical Tucker). It has better complexity with respect to the tensor rank than previous approaches. The new algorithm has a high potential impact in different applications. The key idea is based on applying cross approximation in the "frequency domain", where convolution becomes a simple elementwise product. We illustrate efficiency of our algorithm by computing the three-dimensional Newton potential and by presenting preliminary results for solution of the Hartree-Fock equation on tensor-product grids., Comment: 14 pages, 2 figures

A new fast algebraic method for obtaining an $\mathcal{H}^2$-approximation of a matrix from its entries is presented. The main idea behind the method is based on the nested representation and the maximum-volume principle to select submatrices in low-rank matrices. A special iterative approach for the computation of so-called representing sets is established. The main advantage of the method is that it uses only the hierarchical partitioning of the matrix and does not require special "proxy surfaces" to be selected in advance. The numerical experiments for the electrostatic problem and for the boundary integral operator confirm the effectiveness and robustness of the approach. The complexity is linear in the matrix size and polynomial in the ranks. The algorithm is implemented as an open-source Python package that is available online., Comment: Numer. Linear Algebra Appl. 2015

We consider the problem of computing approximate solution of Poisson equation in the low-parametric tensor formats. We propose a new algorithm to compute the solution based on the cross approximation algorithm in the frequency space, and it has better complexity with respect to ranks in comparison with standard algorithms, which are based on the exponential sums approximation. To illustrate the effectiveness of our solver, we incorporate into a Uzawa solver for the Stokes problem on semi-staggered grid as a subsolver. The resulting solver outperforms the standard method for $n \geq 256$.

New algorithms are proposed for the Tucker approximation of a 3-tensor, that access it using only the tensor-by-vector-by-vector multiplication subroutine. In the matrix case, Krylov methods are methods of choice to approximate the dominant column and row subspaces of a sparse or structured matrix given through the matrix-by-vector multiplication subroutine. Using the Wedderburn rank reduction formula, we propose an algorithm of matrix approximation that computes Krylov subspaces and allows generalization to the tensor case. Several variants of proposed tensor algorithms differ by pivoting strategies, overall cost and quality of approximation. By convincing numerical experiments we show that the proposed methods are faster and more accurate than the minimal Krylov recursion, proposed recently by Elden and Savas., Comment: 34 pages, 3 tables, 5 figures. Submitted to SIAM J. Scientific Computing