Search

The parallel simulation of Spiking Neural P systems is mainly based on a matrix representation, where the graph inherent to the neural model is encoded in an adjacency matrix. The simulation algorithm is based on a matrix-vector multiplication, which is an operation efficiently implemented on parallel devices. However, when the graph of a Spiking Neural P system is not fully connected, the adjacency matrix is sparse and hence, lots of computing resources are wasted in both time and memory domains. For this reason, two compression methods for the matrix representation were proposed in a previous work, but they were not implemented nor parallelized on a simulator. In this paper, they are implemented and parallelized on GPUs as part of a new Spiking Neural P system with delays simulator. Extensive experiments are conducted on high-end GPUs (RTX2080 and A100 80GB), and it is concluded that they outperform other solutions based on state-of-the-art GPU libraries when simulating Spiking Neural P systems.

Sparse general matrix–matrix multiplication (SpGEMM) is a crucial and complex computational task in many practical applications. Improving the performance of SpGEMM on SIMT processors like modern GPUs is challenging due to the unpredictable sparsity of sparse matrices. Although existing GPU solutions have made progress in improving performance through advanced algorithm design, they ignore some optimizations related to specific processor architectures. This can result in a partially inefficient implementation of their algorithms. This paper focuses on optimizing four inefficient parts of the NSparse algorithm on DCU (a GPU-like accelerator). The optimizations include: 1) setting parameters to improve the load balance of the second matrix by extracting maximum row information at runtime; 2) reducing overhead of binning operations by making full use of registers and shared memory effectively; 3) improving numerical SpGEMM performance by adjusting its calculation mode; and 4) enhancing global load balance through finer-grained grouping and kernel configurations. Experiment results demonstrate that when compared to five state-of-the-art SpGEMM algorithms (bhSparse, KokkosKernels, NSparse, rocSparse, and spECK), our optimized method achieves an average of 7.99x (up to 18.2x), 8.01x (up to 20.83x), 2.37x (up to 6.16x), 1.82x (up to 4.20x), and 1.63x (up to 5.01x) speedups on 29 sparse matrices with different sparse structures, respectively. [ABSTRACT FROM AUTHOR]

In this paper, for solving a broad class of large-scale nonconvex and nonsmooth optimization problems, we propose a stochastic two-step inertial Bregman proximal alternating linearized minimization (STiBPALM) algorithm with variance-reduced stochastic gradient estimators. And we show that SAGA and SARAH are variance-reduced gradient estimators. Under expectation conditions with the Kurdyka–Łojasiewicz property and some suitable conditions on the parameters, we obtain that the sequence generated by the proposed algorithm converges to a critical point. And the general convergence rate is also provided. Numerical experiments on sparse nonnegative matrix factorization and blind image-deblurring are presented to demonstrate the performance of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In our paper, we consider the following general problems: check feasibility, count the number of feasible solutions, find an optimal solution, and count the number of optimal solutions in P ∩ Z n , assuming that P is a polyhedron, defined by systems A x ≤ b or A x = b , x ≥ 0 with a sparse matrix A. We develop algorithms for these problems that outperform state-of-the-art ILP and counting algorithms on sparse instances with bounded elements in terms of the computational complexity. Assuming that the matrix A has bounded elements, our complexity bounds have the form s O (n) , where s is the minimum between numbers of non-zeroes in columns and rows of A, respectively. For s = o (log n) , this bound outperforms the state-of-the-art ILP feasibility complexity bound (log n) O (n) , due to Reis & Rothvoss (in: 2023 IEEE 64th Annual symposium on foundations of computer science (FOCS), IEEE, pp. 974–988). For s = ϕ o (log n) , where ϕ denotes the input bit-encoding length, it outperforms the state-of-the-art ILP counting complexity bound ϕ O (n log n) , due to Barvinok et al. (in: Proceedings of 1993 IEEE 34th annual foundations of computer science, pp. 566–572, https://doi.org/10.1109/SFCS.1993.366830, 1993), Dyer, Kannan (Math Oper Res 22(3):545–549, https://doi.org/10.1287/moor.22.3.545, 1997), Barvinok, Pommersheim (Algebr Combin 38:91–147, 1999), Barvinok (in: European Mathematical Society, ETH-Zentrum, Zurich, 2008). We use known and new methods to develop new exponential algorithms for Edge/Vertex Multi-Packing/Multi-Cover Problems on graphs and hypergraphs. This framework consists of many different problems, such as the Stable Multi-set, Vertex Multi-cover, Dominating Multi-set, Set Multi-cover, Multi-set Multi-cover, and Hypergraph Multi-matching problems, which are natural generalizations of the standard Stable Set, Vertex Cover, Dominating Set, Set Cover, and Maximum Matching problems. [ABSTRACT FROM AUTHOR]

Aiming to provide a faster and convenient truncated SVD algorithm for large sparse matrices from real applications (i.e., for computing a few of the largest singular values and the corresponding singular vectors), a dynamically shifted power iteration technique is applied to improve the accuracy of the randomized SVD method. This results in a dynamic shifts-based randomized SVD (dashSVD) algorithm, which also collaborates with the skills for handling sparse matrices. An accuracy-control mechanism is included in the dashSVD algorithm to approximately monitor the per vector error bound of computed singular vectors with negligible overhead. Experiments on real-world data validate that the dashSVD algorithm largely improves the accuracy of a randomized SVD algorithm or attains the same accuracy with fewer passes over the matrix, and provides an efficient accuracy-control mechanism to the randomized SVD computation, while demonstrating the advantages on runtime and parallel efficiency. A bound of the approximation error of the randomized SVD with the shifted power iteration is also proved. [ABSTRACT FROM AUTHOR]

This paper presents a massively parallel approach of the multilevel fast multipole algorithm (PMLFMA) on homegrown many-core SW26010 cluster of China, noted as (SW-PMLFMA), for 3-D electromagnetic scattering problems. In this approach, the multilevel fast multipole algorithm (MLFMA) octree is first partitioned among management processing elements (MPEs) of SW26010 processors following the ternary partitioning scheme using the message passing interface (MPI). Then, the computationally intensive parts of the PMLFMA on each MPI process, matrix filling, aggregation and disaggregation are accelerated by using all the 64 computing processing elements (CPEs) in the same core group of the MPE via the Athread parallel programming model. Different parallelization strategies are designed for many-core accelerators to ensure a high computational throughput. In coincidence with the special characteristic of local Lagrange interpolation, the compressed sparse row (CSR) and the compressed sparse column (CSC) sparse matrix storage format is used for storing interpolation and anterpolation matrices, respectively, together with a specially designed cache mechanism of hybrid dynamic and static buffers using the scratchpad memory (SPM) to improve data access efficiency. Numerical results are included to demonstrate the efficiency and versatility of the proposed method. The proposed parallel scheme is shown to have excellent speedup. [ABSTRACT FROM AUTHOR]

Summary: Computer supported design of materials with tailored properties is an important part of computational science so that developments of new effective algorithms is a key issue. Molecular magnetism deals with complex objects described by the quantum physics and their simulations come up against the computational constraints. In this work, we consider a numerical version of a recently developed hybrid approach that combines the ab initio DFT method with the exact diagonalization of the microscopic Hubbard model (HM). The latter avoids the prior perturbative framework but increases computing resources needed for solving the eigenvalue problem of the matrix representation of HM. We demonstrate that in the case of the ring‐shape molecular nanomagnets the computational demands can be curbed due to efficient numerical matrix construction algorithms provided. These algorithms pertain both to the calculation of the single nonzero matrix elements and to their localization in the final sparse matrix that is subsequently diagonalized. Their implementation executed on a simple six‐core‐computer system and the Mathematica environment leads to a significant gain in the computing time for the exemplary chromium‐based molecule denoted Cr8$$ {}_8 $$, running the code sequentially. We claim, referring to the numerical diagonalization of the spin Hamiltonian matrices, that the parallelization flaws related to the high‐level programming approach can be all removed, porting the code to the appropriate HPC environment. A prospective implementation of the code, based on some dedicated and optimized mathematical libraries, will ultimately open a window for further applications of the Hubbard model in molecular magnetism. [ABSTRACT FROM AUTHOR]

We introduce a mixed precision algorithm for computing sparse matrix-vector products and use it to accelerate the solution of sparse linear systems by iterative methods. Our approach is based on the idea of adapting the precision of each matrix element to their magnitude: we split the elements into buckets and use progressively lower precisions for the buckets of progressively smaller elements. We carry out a rounding error analysis of this algorithm that provides us with an explicit rule to decide which element goes into which bucket and allows us to rigorously control the accuracy of the algorithm. We implement the algorithm on a multicore computer and obtain significant speedups (up to a factor 7\times) with respect to uniform precision algorithms, without loss of accuracy, on a range of sparse matrices from real-life applications. We showcase the effectiveness of our algorithm by plugging it into various Krylov solvers for sparse linear systems and observe that the convergence of the solution is essentially unaffected by the use of adaptive precision. [ABSTRACT FROM AUTHOR]

Backpack computers require powerful, intelligent computing capabilities for field wearables while taking energy consumption into careful consideration. A recommended solution for this demand is the CPU + NPU-based SoC. In many wearable intelligence applications, the Fourier Transform is an essential, computationally intensive preprocessing task. However, due to the unique structure of the NPU, the conventional Fourier Transform algorithms cannot be applied directly to it. This paper proposes two NPU-accelerated Fourier Transform algorithms that leverage the unique hardware structure of the NPU and provides three implementations of those algorithms, namely MM-2DFT, MV-2FFTm, and MV-2FFTv. Then, we benchmarked the speed and energy efficiency of our algorithms for the gray image edge filtering task on the Huawei Atlas200I-DK-A2 development kits against the Cooley-Tukey algorithm running on CPU and GPU platforms. The experiment results reveal MM-2DFT outperforms OpenCL-based FFT on NVIDIA Tegra X2 GPU for small input sizes, with a 4- to 8-time speedup. As the input image resolution exceeds 2048, MV-2FFTv approaches GPU computation speed. Additionally, two scenarios were tested and analyzed for energy efficiency, revealing that cube units of the NPU are more energy efficient. The vector and CPU units are better suited for sparse matrix multiplication and small-scale inputs, respectively. [ABSTRACT FROM AUTHOR]

We study the issue of numerically solving the nonnegative inverse eigenvalue problem (NIEP). At first, we reformulate the NIEP as a convex composite optimization problem on Riemannian manifolds. Then we develop a scheme of the Riemannian linearized proximal algorithm (R-LPA) to solve the NIEP. Under some mild conditions, the local and global convergence results of the R-LPA for the NIEP are established, respectively. Moreover, numerical experiments are presented. Compared with the Riemannian Newton-CG method in Z. Zhao et al. (Numer. Math. 140:827–855, 2018), this R-LPA owns better numerical performances for large scale problems and sparse matrix cases, which is due to the smaller dimension of the Riemannian manifold derived from the problem formulation of the NIEP as a convex composite optimization problem. [ABSTRACT FROM AUTHOR]

Simultaneous online identification of the structure and parameters for multiple systems can use the multi‐task learning to share parameter sparse patterns across different identification tasks to improve the estimation accuracy. To reduce the communication and computational burden, improve the convergence rate and increase the reliability of the identification task, a distributed asynchronous multi‐task sparse identification algorithm is proposed. The alternating minimization algorithm is applied to design a distributed multi‐task loss function. Then by constructing the auxiliary matrix and transforming the sparse constraint on the parameter matrix to the constructed auxiliary matrix, the asynchronous parameter update and sparse model update can be realized. The algorithm uses a recursive form to complete online identification which is feasible to analyze. After proving the convergence of the identification algorithm, we present an example to support the theoretical analysis and show the advantages of the proposed algorithm. [ABSTRACT FROM AUTHOR]

We study the problem of detecting and locating change points in high-dimensional Vector Autoregressive (VAR) models, whose transition matrices exhibit low rank plus sparse structure. We first address the problem of detecting a single change point using an exhaustive search algorithm and establish a finite sample error bound for its accuracy. Next, we extend the results to the case of multiple change points that can grow as a function of the sample size. Their detection is based on a two-step algorithm, wherein the first step, an exhaustive search for a candidate change point is employed for overlapping windows, and subsequently a backward elimination procedure is used to screen out redundant candidates. The two-step strategy yields consistent estimates of the number and the locations of the change points. To reduce computation cost, we also investigate conditions under which a surrogate VAR model with a weakly sparse transition matrix can accurately estimate the change points and their locations for data generated by the original model. This work also addresses and resolves a number of novel technical challenges posed by the nature of the VAR models under consideration. The effectiveness of the proposed algorithms and methodology is illustrated on both synthetic and two real datasets. for this article are available online. [ABSTRACT FROM AUTHOR]

Here, a novel image‐based quantized compressive sensing (QCS) framework based on complex‐valued Vandermonde (Vander) matrix is proposed. In the proposed QCS framework, a discrete wavelet transform (DWT) serves as a sparse basis and a partial complex‐valued Vander matrix serves as a measurement matrix. The theoretical analysis based on mutual coherence metric of compressive sensing (CS) theory shows that the proposed Vander measurement matrix has the best reconstruction performance among other conventional measurement matrices. The simulation results also show that the recovery quality using the proposed measurement matrix can be greatly improved compared with the other existing real‐valued measurement matrices. In particular, the experiment results also show that under the same measurement matrix, the reconstruction performance of Smoothed l0 norm (SL0) algorithm is better than that of Orthogonal Matching Pursuit (OMP) algorithm, sparsity adaptive matching pursuit (SAMP) algorithm and approximate message passing (AMP) algorithm. In addition, a sparse measurement matrix scheme is further proposed to achieve a trade‐off between recovery performance and computational complexity. The theoretical analysis and simulation results both show the proposed image‐based QCS is efficient. [ABSTRACT FROM AUTHOR]

SuiteSparse:GraphBLAS is a full parallel implementation of the GraphBLAS standard, which defines a set of sparse matrix operations on an extended algebra of semirings using an almost unlimited variety of operators and types. When applied to sparse adjacency matrices, these algebraic operations are equivalent to computations on graphs. A description of the parallel implementation of SuiteSparse:GraphBLAS is given, including its novel parallel algorithms for sparse matrix multiply, addition, element-wise multiply, submatrix extraction and assignment, and the GraphBLAS mask/accumulator operation. Its performance is illustrated by solving the graph problems in the GAP Benchmark and by comparing it with other sparse matrix libraries. [ABSTRACT FROM AUTHOR]

In this article, we study the problem of deterministic factorization of sparse polynomials. We show that if f ∈ F[x1, x2, . . ., xn] is a polynomial with s monomials, with individual degrees of its variables bounded by d, then f can be deterministically factored in time spoly(d) log n. Prior to our work, the only efficient factoring algorithms known for this class of polynomials were randomized, and other than for the cases of d = 1 and d = 2, only exponential time-deterministic factoring algorithms were known. A crucial ingredient in our proof is a quasi-polynomial sparsity bound for factors of sparse polynomials of bounded individual degree. In particular, we show that if f is an s-sparse polynomial in n variables, with individual degrees of its variables bounded by d, then the sparsity of each factor of f is bounded by s (9d² log n). This is the first non-trivial bound on factor sparsity ford > 2. Our sparsity bound uses techniques fromconvex geometry, such as the theory of Newton polytopes and an approximate version of the classical Carathéodory's Theorem. Our work addresses and partially answers a question of von zur Gathen and Kaltofen [1985] who asked whether a quasi-polynomial bound holds for the sparsity of factors of sparse polynomials. [ABSTRACT FROM AUTHOR]

Application with a set of dependent distributed tasks is generally regarded as a direct acyclic graph or an out-tree. Tree-shaped task graphs are widely applied in a variety of computational domains, including electronic structure computations and sparse matrix factorization. Efficient algorithms for tree-shaped task partition and allocation can dominate the performance of heterogeneous computing systems, as most relevant publications have pointed out. This paper presents efficient algorithms for partitioning and allocating tree-shaped tasks on heterogeneous multiprocessor systems with limited memory to improve task-parallel computing. The proposed main algorithm consists of two stages: partition and allocation. During partition, an algorithm is provided for partitioning a task tree into multiple subtrees. It iteratively partitions the subtrees on the critical path of the quotient tree. During allocation, two algorithms are proposed for task allocation to minimize the task tree's execution time. One is to preferentially allocate the largest subtree of the whole tree, and the other is to preferentially allocate the subtree located on the quotient tree's critical path. Experimental results show that the proposed algorithms significantly improve the latest works in terms of average makespan, both on randomly generated trees and on a real-world dataset. On a real-world dataset, the average makespan of existing work is approximately 6.28 × 10 8 . However, it is approximately 2.13 × 10 8 for our proposed algorithm. This results in a reduction of 64.33%. On randomly generated trees, the average makespan of existing work is approximately 8973. However, it is approximately 4040 for our proposed algorithm. This results in a reduction of 54.96%. [ABSTRACT FROM AUTHOR]

The LU factorization of very large sparse matrices requires a significant amount of computing resources, including memory and broadband communication. A hybrid MPI + OpenMP + CUDA algorithm named SuperLU3D can efficiently compute the LU factorization with GPU acceleration. However, this algorithm faces difficulties when dealing with very large sparse matrices with limited GPU resources. Factorizing very large matrices involves a vast amount of nonblocking communication between processes, often leading to a break in SuperLU3D calculation due to the overflow of cluster communication buffers. In this paper, we present an improved GPU-accelerated algorithm named SuperLU3D_Alternate for the LU factorization of very large sparse matrices with fewer GPU resources. The basic idea is "divide and conquer", which means dividing a very large matrix into multiple submatrices, performing LU factorization on each submatrix, and then assembling the factorized results of all submatrices into two complete matrices L and U. In detail, according to the number of available GPUs, a very large matrix is first divided into multiple submatrices using the elimination tree. Then, the LU factorization of each submatrix is alternately computed with limited GPU resources, and its intermediate LU factors from GPUs are saved to the host memory or hard disk. Finally, after finishing the LU factorization of all submatrices, these factorized submatrices are assembled into a complete lower triangular matrix L and a complete upper triangular matrix U, respectively. The SuperLU3D_Alternate algorithm is suitable for hybrid CPU/GPU cluster systems, especially for a subset of nodes without GPUs. To accommodate different hardware resources in various clusters, we designed the algorithm to run in the following three cases: sufficient memory for GPU nodes, insufficient memory for GPU nodes, and insufficient memory for the entire cluster. The results from LU factorization test on different matrices in various cases show that the larger the matrix is, the more efficient this algorithm is under the same GPU memory consumption. In our numerical experiments, SuperLU3D_Alternate achieves speeds of up to 8× that of SuperLU3D (CPU only) and 2.5× that of SuperLU3D (CPU + GPU) on the hybrid cluster with six Tesla V100S GPUs. Furthermore, when the matrix is too big to be handled by SuperLU3D, SuperLU3D_Alternate can still utilize the cluster's host memory or hard disk to solve it. By reducing the amount of data exchange to prevent exceeding the buffer's limit of the cluster MPI nonblocking communication, our algorithm enhances the stability of the program. [ABSTRACT FROM AUTHOR]

Sparse arrays are able to generate more lags to extend the array aperture, which is a distinct advantage in mixed-field localization. To exploit these lags, existing algorithms in the known literature can be mainly divided into two types: the subspace-based algorithm and the sparsity-based algorithm. However, the former algorithm cannot fully utilize the time delay information provided by sparse array, and the second algorithm has basis mismatch problem. In this paper, an interpolation processing method based on atomic norm is proposed to solve the sparse array localization problem. The high-order cumulant matrix is reconstructed by the interpolation method to generate an augmented cumulant matrix without holes, which can make full use of all the lags. Then, the atomic norm minimization method is used to recover the sparse matrix after interpolation in a gridless way. The matrix after recovery enables gridless direction-of-arrival (DOA) estimation. After the interpolation reconstruction, more lags can be exploited, the degrees of freedom are further increased. The proposed algorithm can not only make full use of the array receiving information but also avoid the base mismatch problem, and the accuracy of DOA estimation is improved. Numerical simulations verify the superiority of the proposed algorithm compared with the existing algorithms. [ABSTRACT FROM AUTHOR]

An image encryption scheme based on compressive sensing (CS) and chaos is proposed. Firstly, the plain image is transformed into the sparse coefficient matrix by discrete wavelet transform (DWT). Secondly, to achieve compression and encryption, the sparse coefficient matrix is measured by the measurement matrix, which is constructed by the Logistic-Tent system (LTS). This process can effectively reduce storage space or transmission bandwidth of the image. Finally, the resulting measurement value matrix is re-encrypted by executing dual random index permutation and bit-level diffusion to improve security of the cryptosystem, and then the cipher image is obtained. In addition, the SHA 256 hash value of the original image is utilized to calculate the initial value and parameter of LTS, making the proposed algorithm robust to known-plaintext and chosen-plaintext attacks. The simulation results show that our algorithm has good image compression-encryption capability and security performance. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Railway Science & Engineering is the property of Journal of Railway Science & Engineering Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Systems Engineering & Electronics is the property of Journal of Systems Engineering & Electronics Editorial Department and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

This paper presents an efficient fast Walsh-Hadamard-Hartley transform (FWHT) algorithm that incorporates the computation of the Walsh-Hadamard transform (WHT) with the discrete Hartley transform (DHT) into an orthogonal, unitary single fast transform possesses the block diagonal structure. The proposed algorithm is implemented in an integrated butterfly structure utilizing the sparse matrices factorization approach and the Kronecker (tensor) product technique, which proved a valuable and fast tool for developing and analyzing the proposed algorithm. The proposed approach was distinguished by ease of implementation and reduced computational complexity compared to previous algorithms, which were based on the concatenation of WHT and FHT by saving up to 3N-4 of real multiplication and 7.5N-10 of real addition. [ABSTRACT FROM AUTHOR]

We propose recursive algorithms to update an orthogonal numerical basis matrix of the null space of the block row, (banded) block Toeplitz, and block Macaulay matrix, which is the multivariate generalization of the (banded) block Toeplitz matrix. These structured matrices are often constructed in an iterative way, and, for some applications, a basis matrix of the null space is required in every iteration. Consequently, recursively updating a numerical basis matrix of the null space, while exploiting the inherent structure of the matrices involved, induces large savings in the computation time. Moreover, we also develop a sparse adaptation of one of the recursive algorithms that avoids the explicit construction of the block Macaulay matrix and results in a considerable reduction of the required memory. We provide several numerical experiments to illustrate the proposed algorithms: for example, we solve four multiparameter eigenvalue problems via the null space of the block Macaulay matrix and notice that the recursive and sparse approach are, on average, 450 and 1300 times faster than the standard approach, respectively. [ABSTRACT FROM AUTHOR]

The sparse precision matrix plays an essential role in the Gaussian graphical model since a zero off-diagonal element indicates conditional independence of the corresponding two variables given others. In the Gaussian graphical model, many methods have been proposed, and their theoretical properties are given as well. Among these, the sparse precision matrix estimation via scaled lasso (SPMESL) has an attractive feature in which the penalty level is automatically set to achieve the optimal convergence rate under the sparsity and invertibility conditions. Conversely, other methods need to be used in searching for the optimal tuning parameter. Despite such an advantage, the SPMESL has not been widely used due to its expensive computational cost. In this paper, we develop a GPU-parallel coordinate descent (CD) algorithm for the SPMESL and numerically show that the proposed algorithm is much faster than the least angle regression (LARS) tailored to the SPMESL. Several comprehensive numerical studies are conducted to investigate the scalability of the proposed algorithm and the estimation performance of the SPMESL. The results show that the SPMESL has the lowest false discovery rate for all cases and the best performance in the case where the level of the sparsity of the columns is high. [ABSTRACT FROM AUTHOR]

In this paper, the problem of direction-of-arrival (DOA) estimation for strictly noncircular sources under the condition of unknown mutual coupling is concerned, and then a robust real-valued weighted subspace fitting (WSF) algorithm is proposed via block sparse recovery. Inspired by noncircularity, the real-valued coupled extended array output with double array aperture is first structured via exploiting the real-valued conversion. Then, an efficient real-valued block extended sparse recovery model is constructed by performing the parameterized decoupling operation to avoid the unknown mutual coupling and noncircular phase effects. Thereafter, the WSF framework is investigated to recover the real-valued block sparse matrix, where the spectrum of real-valued NC MUSIC-like is utilized to design a weighted matrix for strengthening the solutions sparsity. Eventually, DOA estimation is achieved based on the support set of the reconstructed block sparse matrix. Owing to the combination of noncircularity, parametrized decoupling thought, and reweighted strategy, the proposed method not only effectively achieves high-precision estimation, but also efficiently reduces the computational complexity. Plenty of simulation results demonstrate the effectiveness and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Sinkhorn algorithm has been used pervasively to approximate the solution to optimal transport (OT) and unbalanced optimal transport (UOT) problems. However, its practical application is limited due to the high computational complexity. To alleviate the computational burden, we propose a novel importance sparsification method, called Spar-Sink, to efficiently approximate entropy-regularized OT and UOT solutions. Specifically, our method employs natural upper bounds for unknown optimal transport plans to establish effective sampling probabilities, and constructs a sparse kernel matrix to accelerate Sinkhorn iterations, reducing the computational cost of each iteration from O(n2) to eO(n) for a sample of size n. Theoretically, we show the proposed estimators for the regularized OT and UOT problems are consistent under mild regularity conditions. Experiments on various synthetic data demonstrate Spar-Sink outperforms mainstream competitors in terms of both estimation error and speed. A real-world echocardiogram data analysis shows Spar-Sink can effectively estimate and visualize cardiac cycles, from which one can identify heart failure and arrhythmia. To evaluate the numerical accuracy of cardiac cycle prediction, we consider the task of predicting the end-systole time point using the end-diastole one. Results show Spar-Sink performs as well as the classical Sinkhorn algorithm, requiring significantly less computational time. [ABSTRACT FROM AUTHOR]

Estimation of the precision matrix (or inverse covariance matrix) is of great importance in statistical data analysis and machine learning. However, as the number of parameters scales quadratically with the dimension p, the computation becomes very challenging when p is large. In this paper, we propose an adaptive sieving reduction algorithm to generate a solution path for the estimation of precision matrices under the l1 penalized D-trace loss, with each subproblem being solved by a second-order algorithm. In each iteration of our algorithm, we are able to greatly reduce the number of variables in the problem based on the Karush-Kuhn-Tucker (KKT) conditions and the sparse structure of the estimated precision matrix in the previous iteration. As a result, our algorithm is capable of handling data sets with very high dimensions that may go beyond the capacity of the existing methods. Moreover, for the sub-problem in each iteration, other than solving the primal problem directly, we develop a semismooth Newton augmented Lagrangian algorithm with global linear convergence rate on the dual problem to improve the efficiency. Theoretical properties of our proposed algorithm have been established. In particular, we show that the convergence rate of our algorithm is asymptotically superlinear. The high efficiency and promising performance of our algorithm are illustrated via extensive simulation studies and real data applications, with comparison to several state-of-the-art solvers. [ABSTRACT FROM AUTHOR]

We have developed a new simple iterative algorithm to determine entries of a normalized matrix given its marginal probabilities. Our method has been successfully used to obtain two different solutions by maximizing the entropy of a desired matrix and by minimizing its Kullback--Leibler divergence from the initial probability distribution. The latter is fully equivalent to the well-known RAS balancing algorithm. The presented method has been evaluated using a traffic matrix of the GÉANT pan-European network and randomly generated matrices of various sparsities. It turns out to be computationally faster than RAS. We have shown that our approach is suitable for efficient balancing both dense and sparse matrices. [ABSTRACT FROM AUTHOR]

An interesting question for linear programming (LP) algorithms is how to deal with solutions in which the number of nonzero variables is less than the number of rows of the matrix in standard form. An approach is that of basis deficiency-allowing (BDA) simplex variations, which work with a subset of independent columns of the coefficient matrix in standard form, wherein the basis is not necessarily represented by a square matrix. We describe one such algorithm with several variants. The research question deals with studying the computational behaviour by using small, extreme cases. For these instances, we must wonder which parameter setting or variants are more appropriate. We compare the setting of two nonsimplex active-set methods with Holmström's TomLab LpSimplex v3.0 commercial sparse primal simplex commercial implementation. All of them update a sparse QR factorization in Matlab. The first two implementations require fewer iterations and provide better solution quality and running time. [ABSTRACT FROM AUTHOR]

For any fixed field K ∈ { Q 2 , Q 3 , Q 5 , ... } , we prove that all univariate polynomials f with exactly 3 (resp. 2) monomial terms, degree d , and all coefficients in { ± 1 , ... , ± H } , can be solved over K within deterministic time log 4 + o (1) ⁡ (d H) log 3 ⁡ d (resp. log 2 + o (1) ⁡ (d H)) in the classical Turing model: Our underlying algorithm correctly counts the number of roots of f in K , and for each such root generates an approximation in Q with logarithmic height O (log 2 ⁡ (d H) log ⁡ d) that converges at a rate of O ((1 / p) 2 i ) after i steps of Newton iteration. We also prove significant speed-ups in certain settings, a minimal spacing bound of p − O (p log p 2 ⁡ (d H) log ⁡ d) for distinct roots in C p , and even stronger root repulsion when there are nonzero degenerate roots in C p : p -adic distance p − O (log p ⁡ (d H)). On the other hand, we prove that there is an explicit family of tetranomials with distinct nonzero roots in Z p indistinguishable in their first Ω (d log p ⁡ H) most significant base- p digits. So speed-ups for t -nomials with t ≥ 4 will require evasion or amortization of such worst-case instances. For a video summary of this paper, please visit https://youtu.be/npfdxLk04MY. [ABSTRACT FROM AUTHOR]

We introduce the python software package Kernel Mixed Model (KMM), which allows users to incorporate the network structure into transcriptome-wide association studies (TWASs). Our software is based on the association algorithm KMM, which is a method that enables the incorporation of the network structure as the kernels of the linear mixed model for TWAS. The implementation of the algorithm aims to offer users simple access to the algorithm through a one-line command. Furthermore, to improve the computing efficiency in case when the interaction network is sparse, we also provide the flexibility of computing with the sparse counterpart of the matrices offered in Python, which reduces both the computation operations and the memory required. [ABSTRACT FROM AUTHOR]

Recent hardware-aware matrix-free algorithms for higher-order finite-element (FE) discretized matrix-vector multiplications reduce floating point operations and data access costs compared to traditional sparse matrix approaches. In this work, we address a critical gap in existing matrix-free implementations which are not well suited for the action of FE discretized matrices on very large number of vectors. In particular, we propose efficient matrix-free algorithms for evaluating FE discretized matrix-multivector products on both multi-node CPU and GPU architectures. To this end, we employ batched evaluation strategies, with the batchsize tailored to underlying hardware architectures, leading to better data locality and enabling further parallelization. On CPUs, we utilize even-odd decomposition, SIMD vectorization, and overlapping computation and communication strategies. On GPUs, we develop strategies to overlap compute with data movement for achieving efficient pipelining and reduced data accesses through the use of GPU-shared memory, constant memory and kernel fusion. Our implementation outperforms the baselines for Helmholtz operator action on 1024 vectors, achieving up to 1.4x improvement on one CPU node and up to 2.8x on one GPU node, while reaching up to 4.4x and 1.5x improvement on multiple nodes for CPUs (3072 cores) and GPUs (24 GPUs), respectively. We further benchmark the performance of the proposed implementation for solving a model eigenvalue problem for 1024 smallest eigenvalue-eigenvector pairs by employing the Chebyshev Filtered Subspace Iteration method, achieving up to 1.5x improvement on one CPU node and up to 2.2x on one GPU node while reaching up to 3.0x and 1.4x improvement on multi-node CPUs (3072 cores) and GPUs (24 GPUs), respectively. • Matrix-free algorithms for FE matrix-multivector products with thousands of vectors on multi-node CPU and GPU architectures. • Hardware-tuned batched evaluation strategies for improved data locality. • Architecture-specific implementation strategies to evaluate the tensor contractions. • Even-Odd decomposition, SIMD vectorization, overlapping compute and communication on CPUs. • Shared memory, constant memory, kernel fusion, overlap compute with data movement, and efficient pipelining on GPUs. [ABSTRACT FROM AUTHOR]

Nonnegative Matrix Factorization (NMF) has attracted a great deal of attention as an effective technique for dimensionality reduction of large-scale nonnegative data. Given a nonnegative matrix, NMF aims to obtain two low-rank nonnegative factor matrices by solving a constrained optimization problem. The Hierarchical Alternating Least Squares (HALS) algorithm is a well-known and widely-used iterative method for solving such optimization problems. However, the original update rule used in the HALS algorithm is not well defined. In this paper, we propose a novel well-defined update rule of the HALS algorithm, and prove its global convergence in the sense of Zangwill. Unlike conventional globally-convergent update rules, the proposed one allows variables to take the value of zero and hence can obtain sparse factor matrices. We also present two stopping conditions that guarantee the finite termination of the HALS algorithm. The practical usefulness of the proposed update rule is shown through experiments using real-world datasets. [ABSTRACT FROM AUTHOR]

Cutting planes from the Boolean Quadric Polytope can be used to reduce the optimality gap of the NP -hard nonconvex quadratic program with box constraints (BoxQP). It is known that all cuts of the Chvátal–Gomory closure of the Boolean Quadric Polytope are A-odd cycle inequalities. We obtain a compact extended relaxation of allA-odd cycle inequalities, which allows to optimize over the Chvátal–Gomory closure without repeated calls to separation algorithms and has less inequalities than the formulation provided by Boros et al. (SIAM J Discrete Math 5(2):163–177, 1992) for sparse matrices. In a computational study, we confirm the strength of this relaxation and show that we can provide very strong bounds for the BoxQP, even with a plain linear program. The resulting bounds are significantly stronger than these from Bonami et al. (Math Program Comput 10(3):333–382, 2018), which arise from separating A-odd cycle inequalities heuristically. [ABSTRACT FROM AUTHOR]

In real-world applications, it is important for machine learning algorithms to be robust against data outliers or corruptions. In this paper, we focus on improving the robustness of a large class of learning algorithms that are formulated as low-rank semi-definite programming (SDP) problems. Traditional formulations use the square loss, which is notorious for being sensitive to outliers. We propose to replace this with more robust noise models, including the $\ell _1$ ℓ 1 -loss and other nonconvex losses. However, the resultant optimization problem becomes difficult as the objective is no longer convex or smooth. To alleviate this problem, we design an efficient algorithm based on majorization-minimization. The crux is on constructing a good optimization surrogate, and we show that this surrogate can be efficiently obtained by the alternating direction method of multipliers (ADMM). By properly monitoring ADMM's convergence, the proposed algorithm is empirically efficient and also theoretically guaranteed to converge to a critical point. Extensive experiments are performed on four machine learning applications using both synthetic and real-world data sets. Results show that the proposed algorithm is not only fast but also has better performance than the state-of-the-arts. [ABSTRACT FROM AUTHOR]

We introduce a novel eigenvalue algorithm for near-diagonal matrices inspired by Rayleigh-Schrödinger perturbation theory and termed iterative perturbative theory (IPT). Contrary to standard eigenvalue algorithms, which are either "direct" (to compute all eigenpairs) or "iterative" (to compute just a few), IPT computes any number of eigenpairs with the same basic iterative procedure. Thanks to this perfect parallelism, IPT proves more efficient than classical methods (LAPACK or CUSOLVER for the full-spectrum problem, preconditioned Davidson solvers for extremal eigenvalues). We give sufficient conditions for linear convergence and demonstrate performance on dense and sparse test matrices, including one from quantum chemistry. The code is available at http://github.com/msmerlak/IterativePerturbationTheory.jl. [ABSTRACT FROM AUTHOR]

Density peaks clustering (DPC) algorithm provides an efficient method to quickly find cluster centers with decision graphs. In recent years, due to its unique parameters, no iteration, and good robustness, it has been widely studied and applied. However, it also has some shortcomings, such as no adaptability, inadaptability to high-dimensional data and accuracy is easily affected. For reducing the higher time complexity of DPC, we introduce the polar coordinates to DPC (PC-DPC). Firstly, obtain the distance from every point to third-party point and the cosine value of the angle formed with the third-party vector, and reorder the points by distances and cosine values. Then, select other points in the adjacent sequence number of each point to calculate distances, and build a sparse distance matrix. Finally, the sparse distance matrix is used as the input of DPC to obtain clustering results. Theoretical analysis and experiments show that, compared with DPC and other algorithms, PC-DPC greatly reduces running time of DPC while maintaining clustering precision. [ABSTRACT FROM AUTHOR]

We derive a conjugate-gradient type algorithm to produce approximate least-squares (LS) solutions for an inconsistent generalized Sylvester-transpose matrix equation. The algorithm is always applicable for any given initial matrix and will arrive at an LS solution within finite steps. When the matrix equation has many LS solutions, the algorithm can search for the one with minimal Frobenius-norm. Moreover, given a matrix Y, the algorithm can find a unique LS solution closest to Y. Numerical experiments show the relevance of the algorithm for square/non-square dense/sparse matrices of medium/large sizes. The algorithm works well in both the number of iterations and the computation time, compared to the direct Kronecker linearization and well-known iterative methods. [ABSTRACT FROM AUTHOR]

Sparse tensor best rank-1 approximation (BR1Approx), which is a sparsity generalization of the dense tensor BR1Approx, and is a higher-order extension of the sparse matrix BR1Approx, is one of the most important problems in sparse tensor decomposition and related problems arising from statistics and machine learning. By exploiting the multilinearity as well as the sparsity structure of the problem, four polynomial-time approximation algorithms are proposed, which are easily implemented, of low computational complexity, and can serve as initial procedures for iterative algorithms. In addition, theoretically guaranteed approximation lower bounds are derived for all the algorithms. We provide numerical experiments on synthetic and real data to illustrate the efficiency and effectiveness of the proposed algorithms; in particular, serving as initialization procedures, the approximation algorithms can help in improving the solution quality of iterative algorithms while reducing the computational time. [ABSTRACT FROM AUTHOR]

Spectral clustering is one of the most popular clustering methods. However, how to balance the efficiency and effectiveness of the large-scale spectral clustering with limited computing resources has not been properly solved for a long time. In this paper, we propose a divide-and-conquer based large-scale spectral clustering method to strike a good balance between efficiency and effectiveness. In the proposed method, a divide-and-conquer based landmark selection algorithm and a novel approximate similarity matrix approach are designed to construct a sparse similarity matrix within low computational complexities. Then clustering results can be computed quickly through a bipartite graph partition process. The proposed method achieves the lower computational complexity than most existing large-scale spectral clustering methods. Experimental results on ten large-scale datasets have demonstrated the efficiency and effectiveness of the proposed method. The MATLAB code of the proposed method and experimental datasets are available at https://github.com/Li-Hongmin/MyPaperWithCode. [ABSTRACT FROM AUTHOR]

The bundle adjustment( BA) is a key technology for the back-end optimization of simultaneous positioning and map construction( SLAM) . Using BA online can meet real-time requirements, which is a key factor in applying it to real-time systems such as autonomous driving vehicles. This paper firstly analyzed the characteristics of SLAM data in real scene and proposed a method using sliding window to reduce the calculation scale. Then it featured the parallelism of the algorithm by analyzing the properties of sparse matrices in local BA. Finally, it implemented and accelerated the algorithm based on the embedded GPU. Applied and tested into a vehicle-mounted SLAM system in a real scene, the experimental results show that it has 4.8 times higher processing performance than the CPU on the same platform on average on the AGX Xavier with embedded GPU for720P images from road scenes. It can also handle position, pose and map data at 15 fps, which meets the real-time requirements of the vehicle-mounted system. [ABSTRACT FROM AUTHOR]

This article presents a sparse, low-memory footprint optimization algorithm for the implementation of model predictive control (MPC) for tracking formulation in embedded systems. This MPC formulation has several advantages over standard MPC formulations, such as an increased domain of attraction and guaranteed recursive feasibility even in the event of a sudden reference change. However, this comes at the expense of the addition of a small amount of decision variables to the MPC’s optimization problem that complicates the structure of its matrices. We propose a sparse optimization algorithm, based on an extension of the alternating direction method of multipliers, that exploits the structure of this particular MPC formulation. We describe the controller formulation and detail how its structure is exploited by means of the aforementioned optimization algorithm. We show closed-loop simulations comparing the proposed solver against other solvers and approaches from the literature. [ABSTRACT FROM AUTHOR]

This paper considers stochastic convex optimization problems with two sets of constraints: (a) deterministic constraints on the domain of the optimization variable, which are difficult to project onto; and (b) deterministic or stochastic constraints that admit efficient projection. Problems of this form arise frequently in the context of semidefinite programming as well as when various NP-hard problems are solved approximately via semidefinite relaxation. Since projection onto the first set of constraints is difficult, it becomes necessary to explore projection-free algorithms, such as the stochastic Frank-Wolfe (FW) algorithm. On the other hand, the second set of constraints cannot be handled in the same way, and must be incorporated as an indicator function within the objective function, thereby complicating the application of FW methods. Similar problems have been studied before; however, they suffer from slow convergence rates. This work, equipped with momentum based gradient tracking technique, guarantees fast convergence rates on par with the best-known rates for problems without the second set of constraints. Zeroth-order variants of the proposed algorithms are also developed and again improve upon the state-of-the-art rate results. We further propose the novel trimmed FW variants that enjoy the same convergence rates as their classical counterparts, but are empirically shown to require significantly fewer calls to the linear minimization oracle speeding up the overall algorithm. The efficacy of the proposed algorithms is tested on relevant applications of sparse matrix estimation, clustering via semidefinite relaxation, and uniform sparsest cut problem. [ABSTRACT FROM AUTHOR]

Due to the sparsity of GNSS signal in the correlation domain, compressed sensing theory is considered to be a promising technology for GNSS signal acquisition. However, the detection probability of the traditional compression acquisition algorithm is low under low signal-to-noise ratio (SNR) conditions. This paper proposes a GNSS compression acquisition algorithm based on sensing matrix optimization. The Frobenius norm of the difference between Gram matrix and an approximate equiangular tight frame (ETF) matrix is taken as the objective function, and the modified conjugate gradient method is adopted to reduce the mutual coherence between the measurement matrix and the sparse basis. Theoretical analysis and simulation results show that the proposed algorithm can significantly improve the detection probability compared with the existing compression acquisition algorithms under the same SNR. [ABSTRACT FROM AUTHOR]

In this paper, the robustness of the orthogonal matching pursuit (OMP) algorithm for multiple measurement vector (MMV) problem under the restricted isometry property (RIP) is presented. Moreover, it is shown that the OMP algorithm can exactly recover K‐row sparse matrices in K iterations under the RIP condition we have presented. Furthermore, the condition is sharp. [ABSTRACT FROM AUTHOR]

Controlled-source electromagnetic (CSEM) method is crucial for detecting and locating underground anomalies and structures. However, it is challenging to interpret the field data with multifrequency via 3-D CSEM inversion. To fully excavate and utilize the valuable information of CSEM data at different frequencies, we propose an efficient algorithm for 3-D multifrequency CSEM (MFCSEM) inversion based on rational Krylov (RK) subspace. Within the framework of our algorithm, we first use the three-term Lanczos recursion to construct the RK basis matrix quickly; thus, the fast MFCSEM forward modeling can be realized via the RK approximation. Subsequently, we present a novel cyclic projection and correction (CPC) algorithm to solve the MFCSEM adjoint forward problems. Finally, the nonlinear conjugate gradient (NLCG) method is adopted to seek a solution to this nonlinear inverse problem. We demonstrate the excellent performance of our algorithm by synthetic and field datasets. The inversion results show that our algorithm is computationally efficient, resulting in considerable speedup compared with the conventional method. Our algorithm provides a new idea that would significantly improve the efficiency of MFCSEM inversion. [ABSTRACT FROM AUTHOR]