Search

Combining multiple data sources, Digital Earth is an integrated observation platform based on air–space–ground–sea monitoring systems. Among these data sources, the Inverse Synthetic Aperture Radar (ISAR) is a crucial observation method. ISAR is typically utilized to monitor both military and civilian ships due to its all-day and all-weather superiority. However, in complex scenarios, multiple targets may exist within the same radar antenna beam, resulting in severe defocusing due to different motion conditions. Therefore, this paper proposes a multiple-target ISAR imaging method with the removal of micro-motion connections based on the integration of joint constraints. The fully motion-compensated targets exhibit low rank and local similarity in the high-resolution range profile (HRRP) domain, while the micro-motion components possess sparsity. Additionally, targets display sparsity in the image domain. Inspired by this, we formulate a novel optimization by promoting the low-rank, the Laplacian, and the sparsity constraints of targets and the sparsity constraints of the micro-motion components. This optimization problem is solved by the linearized alternative direction method with adaptive penalty (LADMAP). Furthermore, the different motions of various targets degrade their inherent characteristics. Therefore, we integrate motion compensation transformation into the optimization, accordingly achieving the separation of rigid bodies and the micro-motion components of different targets. Experiments based on simulated data demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Sonar automatic target recognition (ATR) systems suffer from complex acoustic scattering, background clutter, and waveguide effects that are ever-present in the ocean. Traditional signal processing techniques often struggle to distinguish targets when noise and complicated target geometries are introduced. Recent advancements in machine learning and wavelet theory offer promising directions for extracting informative features from sonar return data. This work introduces a feature extraction and dimensionality reduction technique using the invariant scattering transform and Sparse Multi-view Discriminant Analysis for identifying highly informative features in the PONDEX09/PONDEX10 datasets. The extracted features are used to train a support vector machine classifier that achieves an average classification accuracy of 97.3% using six unique targets. [ABSTRACT FROM AUTHOR]

As data-driven deep learning (DL) has been applied in various scenarios, the privacy threats have become a widely recognized problem. To boost privacy protection in federated learning (FL), some methods adopt a one-shot differential privacy (DP) approach to obfuscate model updates, yet they do not take into account the dynamic balance between efficiency and privacy protection. To this end, we propose ASPFL—an efficient FL approach with adaptive sparsity-based pruning and differential privacy protection. We further propose the adaptive pruning mechanism by utilizing the Jensen-Shannon divergence as the metric to generate sparse matrices, which are then employed in the model updates. In addition, we introduce adaptive Gaussian noise by assessing the variation of sensitivity through post-pruning uploading. Extensive experiments validate that our proposed ASPFL boosts convergence speed by more than two times under non-IID data. Compared with existing DP-FL methods, ASPFL can maximally achieve over 82% accuracy on CIFAR-10, while the communication cost is greatly reduced by 40% under the same level of privacy protection. [ABSTRACT FROM AUTHOR]

A technique is presented that reduces the required memory of neural networks through improving weight storage. In contrast to traditional methods, which have an exponential memory overhead with the increase in network size, the proposed method stores only the number of connections between neurons. The proposed method is evaluated on feedforward networks and demonstrates memory saving capabilities of up to almost 80% while also being more efficient, especially with larger architectures. [ABSTRACT FROM AUTHOR]

As higher spatiotemporal resolution tactile sensing systems are being developed for prosthetics, wearables, and other biomedical applications, they demand faster sampling rates and generate larger data streams. Sparsifying transformations can alleviate these requirements by enabling compressive sampling and efficient data storage through compression. However, research on the best sparsifying transforms for tactile interactions is lagging. In this work we construct a library of orthogonal and biorthogonal wavelet transforms as sparsifying transforms for tactile interactions and compare their tradeoffs in compression and sparsity. We tested the sparsifying transforms on a publicly available high-density tactile object grasping dataset (548 sensor tactile glove, grasping 26 objects). In addition, we investigated which dimension wavelet transform—1D, 2D, or 3D—would best compress these tactile interactions. Our results show that wavelet transforms are highly efficient at compressing tactile data and can lead to very sparse and compact tactile representations. Additionally, our results show that 1D transforms achieve the sparsest representations, followed by 3D, and lastly 2D. Overall, the best wavelet for coarse approximation is Symlets 4 evaluated temporally which can sparsify to 0.5% sparsity and compress 10-bit tactile data to an average of 0.04 bits per pixel. Future studies can leverage the results of this paper to assist in the compressive sampling of large tactile arrays and free up computational resources for real-time processing on computationally constrained mobile platforms like neuroprosthetics. [ABSTRACT FROM AUTHOR]

The key to high-dimensional clustering lies in discovering the intrinsic structures and patterns in data to provide valuable information. However, high-dimensional clustering faces enormous challenges such as dimensionality disaster, increased data sparsity, and reduced reliability of the clustering results. In order to address these issues, we propose a sparse clustering algorithm based on a multi-domain dimensionality reduction model. This method achieves high-dimensional clustering by integrating the sparse reconstruction process and sparse L1 regularization into a deep autoencoder model. A sparse reconstruction module is designed based on the L1 sparse reconstruction of features under different domains to reconstruct the data. The proposed method mainly contributes in two aspects. Firstly, the spatial and frequency domains are combined by taking into account the spatial distribution and frequency characteristics of the data to provide multiple perspectives and choices for data analysis and processing. Then, a neural network-based clustering model with sparsity is conducted by projecting data points onto multi-domains and implementing adaptive regularization penalty terms to the weight matrix. The experimental results demonstrate superior performance of the proposed method in handling clustering problems on high-dimensional datasets. [ABSTRACT FROM AUTHOR]

Faced with the problem of incompatibility between traditional information acquisition mode and spaceborne earth observation tasks, starting from the general mathematical model of compressed sensing, a theoretical model of block compressed sensing was established, and a full-process adaptive coding and decoding compressed sensing framework for remote sensing images was proposed, which includes five parts: mode selection, feature factor extraction, adaptive shape segmentation, adaptive sampling rate allocation and image reconstruction. Unlike previous semi-adaptive or local adaptive methods, the advantages of the adaptive encoding and decoding method proposed in this paper are mainly reflected in four aspects: (1) Ability to select encoding modes based on image content, and maximizing the use of the richness of the image to select appropriate sampling methods; (2) Capable of utilizing image texture details for adaptive segmentation, effectively separating complex and smooth regions; (3) Being able to detect the sparsity of encoding blocks and adaptively allocate sampling rates to fully explore the compressibility of images; (4) The reconstruction matrix can be adaptively selected based on the size of the encoding block to alleviate block artifacts caused by non-stationary characteristics of the image. Experimental results show that the method proposed in this article has good stability for remote sensing images with complex edge textures, with the peak signal-to-noise ratio and structural similarity remaining above 35 dB and 0.8. Moreover, especially for ocean images with relatively simple image content, when the sampling rate is 0.26, the peak signal-to-noise ratio reaches 50.8 dB, and the structural similarity is 0.99. In addition, the recovered images have the smallest BRISQUE value, with better clarity and less distortion. In the subjective aspect, the reconstructed image has clear edge details and good reconstruction effect, while the block effect is effectively suppressed. The framework designed in this paper is superior to similar algorithms in both subjective visual and objective evaluation indexes, which is of great significance for alleviating the incompatibility between traditional information acquisition methods and satellite-borne earth observation missions. [ABSTRACT FROM AUTHOR]

Due to the advantage that the non-convex penalty accurately characterizes the sparsity of structural damage, various models based on non-convex penalties have been effectively utilized to the field of structural damage identification. However, these models generally ignore the influence of the uncertainty on the damage identification, which inevitably reduces the accuracy of damage identification. To improve the damage identification accuracy, a probabilistic structural damage identification method with a generic non-convex penalty is proposed, where the uncertainty corresponding to each mode is quantified using the separate Gaussian distribution. The proposed model is estimated via the iteratively reweighted least squares optimization algorithm according to the maximum likelihood principle. The numerical and experimental results illustrate that the proposed method improves the damage identification accuracy by 3.98% and 7.25% compared to the original model, respectively. [ABSTRACT FROM AUTHOR]

The Recurrent Neural Networks (RNNs) are an essential class of supervised learning algorithms. Complex tasks like speech recognition, machine translation, sentiment classification, weather prediction, etc., are now performed by well-trained RNNs. Local or cloud-based GPU machines are used to train them. However, inference is now shifting to miniature, mobile, IoT devices and even micro-controllers. Due to their colossal memory and computing requirements, mapping RNNs directly onto resource-constrained platforms is arcane and challenging. The efficacy of edge-intelligent RNNs (EI-RNNs) must satisfy both performance and memory-fitting requirements at the same time without compromising one for the other. This study's aim was to provide an empirical evaluation and optimization of historic as well as recent RNN architectures for high-performance and low-memory footprint goals. We focused on Human Activity Recognition (HAR) tasks based on wearable sensor data for embedded healthcare applications. We evaluated and optimized six different recurrent units, namely Vanilla RNNs, Long Short-Term Memory (LSTM) units, Gated Recurrent Units (GRUs), Fast Gated Recurrent Neural Networks (FGRNNs), Fast Recurrent Neural Networks (FRNNs), and Unitary Gated Recurrent Neural Networks (UGRNNs) on eight publicly available time-series HAR datasets. We used the hold-out and cross-validation protocols for training the RNNs. We used low-rank parameterization, iterative hard thresholding, and spare retraining compression for RNNs. We found that efficient training (i.e., dataset handling and preprocessing procedures, hyperparameter tuning, and so on, and suitable compression methods (like low-rank parameterization and iterative pruning) are critical in optimizing RNNs for performance and memory efficiency. We implemented the inference of the optimized models on Raspberry Pi. [ABSTRACT FROM AUTHOR]

This paper expands traditional stochastic volatility models by allowing for time-varying skewness without imposing it. While dynamic asymmetry may capture the likely direction of future asset returns, it comes at the risk of leading to overparameterization. Our proposed approach mitigates this concern by leveraging sparsity-inducing priors to automatically select the skewness parameter as dynamic, static or zero in a data-driven framework. We consider two empirical applications. First, in a bond yield application, dynamic skewness captures interest rate cycles of monetary easing and tightening and is partially explained by central banks' mandates. In a currency modeling framework, our model indicates no skewness in the carry factor after accounting for stochastic volatility. This supports the idea of carry crashes resulting from volatility surges instead of dynamic skewness. [ABSTRACT FROM AUTHOR]

The robust and sparse portfolio selection problem is one of the most-popular and -frequently studied problems in the optimization and financial literature. By considering the uncertainty of the parameters, the goal is to construct a sparse portfolio with low volatility and decent returns, subject to other investment constraints. In this paper, we propose a new portfolio selection model, which considers the perturbation in the asset return matrix and the parameter uncertainty in the expected asset return. We define three types of stationary points of the penalty problem: the Karush–Kuhn–Tucker point, the strong Karush–Kuhn–Tucker point, and the partial minimizer. We analyze the relationship between these stationary points and the local/global minimizer of the penalty model under mild conditions. We design a penalty alternating-direction method to obtain the solutions. Compared with several existing portfolio models on seven real-world datasets, extensive numerical experiments demonstrate the robustness and effectiveness of our model in generating lower volatility. [ABSTRACT FROM AUTHOR]

Despite the popularity of factor models with simple loading matrices, little attention has been given to formally address the identifiability of these models beyond standard rotation-based identification such as the positive lower triangular (PLT) constraint. To fill this gap, we review the advantages of variance identification in simple factor analysis and introduce the generalized lower triangular (GLT) structures. We show that the GLT assumption is an improvement over PLT without compromise: GLT is also unique but, unlike PLT, a non-restrictive assumption. Furthermore, we provide a simple counting rule for variance identification under GLT structures, and we demonstrate that within this model class, the unknown number of common factors can be recovered in an exploratory factor analysis. Our methodology is illustrated for simulated data in the context of post-processing posterior draws in sparse Bayesian factor analysis. [ABSTRACT FROM AUTHOR]

Digital beamforming techniques find wide applications in the field of underwater acoustic array signal processing. However, their azimuthal resolution has long been constrained by the Rayleigh limit, consequently limiting their detection performance. In this paper, we propose a novel two-dimensional Hessian–sparse deconvolution algorithm based on image processing techniques. This method assumes a priori that the underwater acoustic bearing time record (BTR) images exhibit sparsity, and then it first constructs partial differential equations in the beamforming domain with sparsity-norm constraints for optimal noise reduction. Subsequently, a two-dimensional deconvolution operation is applied to narrow the main lobe, aiming to achieve additional temporal gains in two-dimensional processing. The simulation and real sea trial data processing results show that the main lobe width of the proposed method is about 1.3 degrees at 0 dB. It effectively reduces the main lobe width and enhances the detection resolution of BTRs in the post-processing part, especially in low-signal-to-noise-ratio (SNR) environments. Therefore, the proposed method provides nice opportunities to further improve the target-detecting ability of hydrophone arrays. [ABSTRACT FROM AUTHOR]

As one of the supervised tensor learning methods, the support tensor machine (STM) for tensorial data classification is receiving increasing attention in machine learning and related applications, including remote sensing imaging, video processing, fault diagnosis, etc. Existing STM approaches lack consideration for support tensors in terms of data reduction. To address this deficiency, we built a novel sparse STM model to control the number of support tensors in the binary classification of tensorial data. The sparsity is imposed on the dual variables in the context of the feature space, which facilitates the nonlinear classification with kernel tricks, such as the widely used Gaussian RBF kernel. To alleviate the local risk associated with the constant width in the tensor Gaussian RBF kernel, we propose a two-stage classification approach; in the second stage, we advocate for a scaling strategy on the kernel function in a data-dependent way, using the information of the support tensors obtained from the first stage. The essential optimization models in both stages share the same type, which is non-convex and discontinuous, due to the sparsity constraint. To resolve the computational challenge, a subspace Newton method is tailored for the sparsity-constrained optimization for effective computation with local convergence. Numerical experiments were conducted on real datasets, and the numerical results demonstrate the effectiveness of our proposed two-stage sparse STM approach in terms of classification accuracy, compared with the state-of-the-art binary classification approaches. [ABSTRACT FROM AUTHOR]

In this work, we proposed a sparse version of the Support Vector Regression (SVR) algorithm that uses regularization to achieve sparsity in function estimation. To achieve this, we used an adaptive L0 penalty that has a ridge structure and, therefore, does not introduce additional computational complexity to the algorithm. In addition to this, we used an alternative approach based on a similar proposal in the Support Vector Machine (SVM) literature. Through numerical studies, we demonstrated the effectiveness of our proposals. We believe that this is the first time someone discussed a sparse version of Support Vector Regression (in terms of variable selection and not in terms of support vector selection). [ABSTRACT FROM AUTHOR]

Single-cell RNA sequencing (scRNA-seq) has become a powerful technique to investigate cellular heterogeneity and complexity in various fields by revealing the gene expression status of individual cells. Despite the undeniable benefits of scRNA-seq, it is not immune to its inherent limitations, such as sparsity and noise, which would hinder downstream analysis. In this paper, we introduce scCGImpute, a model-based approach for addressing the challenges of sparsity in scRNA-seq data through imputation. After identifying possible dropouts using mixed models, scCGImpute takes advantage of the cellular similarity in the same subpopulation to impute and then uses random forest regression to obtain the final imputation. scCGImpute only imputes the likely dropouts without changing the non-dropout data and can use information from the similarity of cells and genetic correlation simultaneously. Experiments on simulation data and real data were made, respectively, to evaluate the performance of scCGImpute in terms of gene expression recovery and clustering analysis. The results demonstrated that scCGImpute can effectively restore gene expression and improve the identification of cell types. [ABSTRACT FROM AUTHOR]

Transformer models have emerged as the state-of-the-art in many natural language processing and computer vision applications due to their capability of attending to longer sequences of tokens and supporting parallel processing more efficiently. Nevertheless, the training and inference of transformer models are computationally expensive and memory intensive. Meanwhile, utilizing the sparsity in deep learning models has proven to be an effective approach to alleviate the computation challenge as well as help to fit large models in edge devices. As high-performance CPUs and GPUs are generally not flexible enough to explore low-level sparsity, a number of specialized hardware accelerators have been proposed for transformer models. This paper provides a comprehensive review of hardware transformer accelerators that have been proposed to explore sparsity for computation and memory optimizations. We classify existing works based on the strategies of utilizing sparsity and identify their pros and cons in those strategies. Based on our analysis, we point out promising directions and recommendations for future works on improving the effective sparse execution of transformer hardware accelerators. [ABSTRACT FROM AUTHOR]

Collaborative filtering (CF) usually suffers from data sparsity and cold starts. Knowledge graphs (KGs) are widely used to improve recommendation performance. To verify that knowledge graphs can further alleviate the above problems, this paper proposes an end-to-end framework that uses attentive knowledge graph perceptual propagation for recommendations (AKGP). This framework uses a knowledge graph as a source of auxiliary information to extract user–item interaction information and build a sub-knowledge base. The fusion of structural and contextual information is used to construct fine-grained knowledge graphs via knowledge graph embedding methods and to generate initial embedding representations. Through multi-layer propagation, the structured information and historical preference information are embedded into a unified vector space, and the potential user–item vector representation is expanded. This article used a knowledge perception attention module to achieve feature representation, and finally, the model was optimized using the stratified sampling joint learning method. Compared with the baseline model using MovieLens-1M, Last-FM, Book-Crossing and other data sets, the experimental results demonstrate that the model outperforms state-of-the-art KG-based recommendation methods, and the shortcomings of the existing model are improved. The model was applied to product design data and historical maintenance records provided by an automotive parts manufacturing company. The predictions of the recommended system are matched to the product requirements and possible failure records. This helped reduce costs and increase productivity, helping the company to quickly determine the cause of failures and reduce unplanned downtime. [ABSTRACT FROM AUTHOR]

Recommender systems aim to forecast users' rank, interests, and preferences in specific products and recommend them to a user for purchase. Collaborative filtering is the most popular approach, where the user's past purchase behavior consists of the user's feedback. One of the most challenging problems in collaborative filtering is handling users whose previous item purchase behavior is unknown, (e.g., new users) or products for which user interactions are not available, (e.g., new products). In this work, we address the cold-start problem in recommender systems based on frequent patterns which are highly frequent in one set of users, but less frequent or infrequent in other sets of users. Such discriminant frequent patterns can distinguish one target set of users from all other sets. The proposed methodology, first forms different clusters of old users and then discovers discriminant frequent patterns for each different such cluster of users and finally exploits the latter to hallucinate the purchase behavior of new users. We also present empirical results to demonstrate the efficiency and accuracy of the proposed methodology. [ABSTRACT FROM AUTHOR]

Urban railway track infrastructures often suffer from damage that affects their service performance due to a variety of factors. In this study, an unsupervised feature selection and damage identification method based on globally sparse probabilistic principal component analysis (PCA) is proposed for urban railway tracks using the monitoring data of train-induced dynamic responses. A Bayesian framework is proposed for generating principal components on a basis of vectors (original variables) with a global sparseness pattern instead of separate patterns in a traditional sparse PCA. In this framework, a variational expectation-maximization algorithm is employed to obtain the tractable calculation of the marginal likelihood function for learning all uncertain parameters of the Bayesian model. The obtained principal components are linear combinations of the very same set of important variables, making our method better interpretable than the traditional sparse PCA. We can clearly understand which original variables are most relevant for describing the data. The track damage is identified simply by discriminating the corresponding measured dynamic responses using the binary elements of the latent vector inferred from the Bayesian globally sparse PCA algorithm. The usefulness is demonstrated by successfully identifying the track bed plate crack damage through the actual train-induced dynamic responses collected from the structural health monitoring system of an urban railway track infrastructure, where the method is able to achieve F 1 scores of 90% or higher for various scenarios. [ABSTRACT FROM AUTHOR]

In order to detect weak underwater tonals, adaptive line enhancers (ALEs) have been widely applied in passive sonars. Unfortunately, conventional ALEs cannot perform well amid impulse noise generated by ice cracking, snapping shrimp or other factors. This kind of noise has a different noise model compared to Gaussian noise and leads to noise model mismatch problems in conventional ALEs. To mitigate the performance degradation of conventional ALEs in under-ice impulse noise, in this study, a modified ALE is proposed for passive sonars. The proposed ALE is based on the least mean p-power (LMP) error criterion and the prior information of the frequency domain sparsity to improve the enhancement performance under impulse noise. The signal-to-noise ratio (SNR) gain is chosen as the metric for evaluating the proposed ALE. The simulation results show that the output SNR gain of the proposed ALE was, respectively, 9.3 and 2.6 dB higher than that of the sparsity-based ALE (SALE) and the least mean p-power ALE (PALE) when the input GSNR was −12 dB. The results of processing the under-ice noise data also demonstrate that the proposed ALE is distinguished among the four ALEs. [ABSTRACT FROM AUTHOR]

It is known that feature selection/screening for high-dimensional nonparametric models is an important but very difficult issue. In this paper, we first point out the limitations of existing screening methods. In particular, model-free sure independence screening methods, which are defined on random predictors, may completely miss some important features in the underlying nonparametric function when the predictors follow certain distributions. To overcome these limitations, we propose an ensemble screening procedure for nonparametric models. It elaborately combines several existing screening methods and outputs a result close to the best one of these methods. Numerical examples indicate that the proposed method is very competitive and has satisfactory performance even when existing methods fail. [ABSTRACT FROM AUTHOR]

Bearing fault diagnosis for equipment-safe operation has a crucial role. In recent years, more achievements have been made in bearing fault diagnosis. However, for the fault diagnosis model, the representation and sensitivity of bearing fault features have a great influence on the diagnosis output results; thus, the attention mechanism is particularly important for the selection of features. However, global attention focuses on all sequences, which is computationally expensive and not ideal for fault diagnosis tasks. The local attention mechanism ignores the relationship between non-adjacent sequences. To address the respective shortcomings of global attention and local attention, an adaptive sparse attention network is proposed in this paper to filter fault-sensitive information by soft threshold filtering. In addition, the effects of different signal representation domains on fault diagnosis results are investigated to filter out signal representation forms with better performance. Finally, the proposed adaptive sparse attention network is applied to cross-working conditions diagnosis of bearings. The adaptive sparse attention mechanism focuses on the signal characteristics of different frequency bands for different fault types. The proposed network model achieves better overall performance when comparing the cross-conditions diagnosis accuracy and model convergence speed. [ABSTRACT FROM AUTHOR]

LiDAR is a commonly used sensor for autonomous driving to make accurate, robust, and fast decision-making when driving. The sensor is used in the perception system, especially object detection, to understand the driving environment. Although 2D object detection has succeeded during the deep-learning era, the lack of depth information limits understanding of the driving environment and object location. Three-dimensional sensors, such as LiDAR, give 3D information about the surrounding environment, which is essential for a 3D perception system. Despite the attention of the computer vision community to 3D object detection due to multiple applications in robotics and autonomous driving, there are challenges, such as scale change, sparsity, uneven distribution of LiDAR data, and occlusions. Different representations of LiDAR data and methods to minimize the effect of the sparsity of LiDAR data have been proposed. This survey presents the LiDAR-based 3D object detection and feature-extraction techniques for LiDAR data. The 3D coordinate systems differ in camera and LiDAR-based datasets and methods. Therefore, the commonly used 3D coordinate systems are summarized. Then, state-of-the-art LiDAR-based 3D object-detection methods are reviewed with a selected comparison among methods. [ABSTRACT FROM AUTHOR]

In recent years, especially with the (COVID-19) pandemic, shopping has been a challenging task. Increased online shopping has increased information available via the World Wide Web. Finding new products or identifying the most suitable products according to customers' personalization trends is the main benefit of E-commerce recommendation systems, which use different techniques such as rating, ranking, or reviewing. These recommendations can be formed using different techniques and approaches, particularly using the technology of intelligent agents, and specific interfaces or personal agents can be used to model this type of system. These agents usually use the techniques and algorithms of Artificial Intelligence internally. A recommendation system is a prediction system that has been created to help the user to select the proper product for them, and to reduce the effort spent in the search process using advanced technology such as deep learning techniques. We investigate all studies using a standard review process for collecting and retrieving data from previous studies and illustrate their relevant accuracy and interpretability along with pros and cons helpful to business firms to adopt the most legitimate approach. The study's findings revealed that recommendation problems are solved better by using deep learning algorithms such as CNN, RNN, and sentiment analysis, especially for popular problems such as cold start and sparsity. [ABSTRACT FROM AUTHOR]

In multiple response regression, the reduced rank regression model is an effective method to reduce the number of model parameters and it takes advantage of interrelation among the response variables. To improve the prediction performance of the multiple response regression, a method for the sparse robust reduced rank regression with covariance estimation(Cov-SR4) is proposed, which can carry out variable selection, outlier detection, and covariance estimation simultaneously. The random error term of this model follows a multivariate normal distribution which is a symmetric distribution and the covariance matrix or precision matrix must be a symmetric matrix that reduces the number of parameters. Both the element-wise penalty function and row-wise penalty function can be used to handle different types of outliers. A numerical algorithm with a covariance estimation method is proposed to solve the robust sparse reduced rank regression. We compare our method with three recent reduced rank regression methods in a simulation study and real data analysis. Our method exhibits competitive performance both in prediction error and variable selection accuracy. [ABSTRACT FROM AUTHOR]

With the development of science and technology, more and more data have been produced. For many of these datasets, only some of the data have labels. In order to make full use of the information in these data, it is necessary to classify them. In this paper, we propose a strong sparse quadratic kernel-free least squares semi-supervised support vector machine ( S S Q L S S 3 V M ), in which we add a ℓ 0 norm regularization term to make it sparse. An NP-hard problem arises since the proposed model contains the ℓ 0 norm and another nonconvex term. One important method for solving the nonconvex problem is the DC (difference of convex function) programming. Therefore, we first approximate the ℓ 0 norm by a polyhedral DC function. Moreover, due to the existence of the nonsmooth terms, we use the sGS-ADMM to solve the subproblem. Finally, empirical numerical experiments show the efficiency of the proposed algorithm. [ABSTRACT FROM AUTHOR]

As applied sciences grow by leaps and bounds, semiparametric regression analyses have broad applications in various fields, such as engineering, finance, medicine, and public health. Single-index varying-coefficient model is a common class of semiparametric models due to its flexibility and ease of interpretation. The standard single-index varying-coefficient regression models consist mainly of parametric regression and semiparametric regression, which assume that all covariates can be observed. The assumptions are relaxed by taking the models with missing covariates into consideration. To eliminate the possibility of bias due to missing data, we propose a probability weighted objective function. In this paper, we investigate the robust variable selection for a single-index varying-coefficient model with missing covariates. Using parametric and nonparametric estimates of the likelihood of observations with fully observed covariates, we examine the estimators for estimating the likelihood of observations. For variable selection, we use a weighted objective function penalized by a non-convex SCAD. Theoretical challenges include the treatment of missing data and a single-index varying-coefficient model that uses both the non-smooth loss function and the non-convex penalty function. We provide Monte Carlo simulations to evaluate the performance of our approach. [ABSTRACT FROM AUTHOR]

Solving polynomial equations inevitably faces many severe challenges, such as easily occupying storage space and demanding prohibitively expensive computation resources. There has been considerable interest in exploiting the sparsity to improve computation efficiency, since asymmetry phenomena are prevalent in scientific and engineering fields, especially as most of the systems in real applications have sparse representations. In this paper, we propose an efficient parallel hybrid algorithm for constructing a Dixon matrix. This approach takes advantage of the asymmetry (i.e., sparsity) in variables of the system and introduces a heuristics strategy. Our method supports parallel computation and has been implemented on a multi-core system. Through time-complexity analysis and extensive benchmarks, we show our new algorithm has significantly reduced computation and memory overhead. In addition, performance evaluation via the Fermat–Torricelli point problem demonstrates its effectiveness in combinatorial geometry optimizations. [ABSTRACT FROM AUTHOR]

There is an increasing interest in machine learning (ML) algorithms for predicting patient outcomes, as these methods are designed to automatically discover complex data patterns. For example, the random forest (RF) algorithm is designed to identify relevant predictor variables out of a large set of candidates. In addition, researchers may also use external information for variable selection to improve model interpretability and variable selection accuracy, thereby prediction quality. However, it is unclear to which extent, if at all, RF and ML methods may benefit from external information. In this paper, we examine the usefulness of external information from prior variable selection studies that used traditional statistical modeling approaches such as the Lasso, or suboptimal methods such as univariate selection. We conducted a plasmode simulation study based on subsampling a data set from a pharmacoepidemiologic study with nearly 200,000 individuals, two binary outcomes and 1152 candidate predictor (mainly sparse binary) variables. When the scope of candidate predictors was reduced based on external knowledge RF models achieved better calibration, that is, better agreement of predictions and observed outcome rates. However, prediction quality measured by cross-entropy, AUROC or the Brier score did not improve. We recommend appraising the methodological quality of studies that serve as an external information source for future prediction model development. [ABSTRACT FROM AUTHOR]

In this paper, we study the asymptotic normality in high-dimensional linear regression. We focus on the case where the covariance matrix of the regression variables has a KMS structure, in asymptotic settings where the number of predictors, p, is proportional to the number of observations, n. The main result of the paper is the derivation of the exact asymptotic distribution for the suitably centered and normalized squared norm of the product between predictor matrix, X , and outcome variable, Y, i.e., the statistic ∥ X ′ Y ∥ 2 2 , under rather unrestrictive assumptions for the model parameters β j . We employ variance-gamma distribution in order to derive the results, which, along with the asymptotic results, allows us to easily define the exact distribution of the statistic. Additionally, we consider a specific case of approximate sparsity of the model parameter vector β and perform a Monte Carlo simulation study. The simulation results suggest that the statistic approaches the limiting distribution fairly quickly even under high variable multi-correlation and relatively small number of observations, suggesting possible applications to the construction of statistical testing procedures for the real-world data and related problems. [ABSTRACT FROM AUTHOR]

Inverse problems arise in many areas of science and engineering, such as geophysics, biology, and medical imaging. One of the main imaging modalities that have seen a huge increase in recent years is the noninvasive, nonionizing, and radiation-free imaging technique of electrical impedance tomography (EIT). Other advantages of such a technique are the low cost and ubiquitousness. An imaging technique is used to recover the internal conductivity of a body using measurements from electrodes from the body's surface. The standard procedure is to obtain measurements by placing electrodes in the body and measuring conductivity inside the object. A current with low frequency is applied on the electrodes below a threshold, rendering the technique harmless for the body, especially when applied to living organisms. As with many inverse problems, EIT suffers from ill-posedness, i.e., the reconstruction of internal conductivity is a severely ill-posed inverse problem and typically yields a poor-quality solution. Moreover, the desired solution has step changes in the electrical properties that are typically challenging to be reconstructed by traditional smoothing regularization methods. To counter this difficulty, one solves a regularized problem that is better conditioned than the original problem by imposing constraints on the regularization term. The main contribution of this work is to develop a general ℓ p regularized method with total variation to solve the nonlinear EIT problem through a iteratively reweighted majorization–minimization strategy combined with the Gauss–Newton approach. The main idea is to majorize the linearized EIT problem at each iteration and minimize through a quadratic tangent majorant. Simulated numerical examples from complete electrode model illustrate the effectiveness of our approach. [ABSTRACT FROM AUTHOR]

Due to a point cloud's sparse nature, a sparse convolution block design is necessary to deal with its particularities. Mechanisms adopted in computer vision have recently explored the advantages of data processing in more energy-efficient hardware, such as the FPGA, as a response to the need to run these algorithms on resource-constrained edge devices. However, implementing it in hardware has not been properly explored, resulting in a small number of studies aimed at analyzing the potential of sparse convolutions and their efficiency on resource-constrained hardware platforms. This article presents the design of a customizable hardware block for the voting convolution. We carried out an in-depth analysis to determine under which conditions the use of the voting scheme is justified instead of dense convolutions. The proposed hardware design achieves an energy consumption about 8.7 times lower than similar works in the literature by ignoring unnecessary arithmetic operations with null weights and leveraging data dependency. Access to data memory was also reduced to the minimum necessary, leading to improvements of around 55% in processing time. To evaluate both the performance and applicability of the proposed solution, the voting convolution was integrated into the well-known PointPillars model, where it achieves improvements between 23.05% and 80.44% without a significant effect on detection performance. [ABSTRACT FROM AUTHOR]

Blind image deblurring (BID) is a long-standing challenging problem in low-level image processing. To achieve visually pleasing results, it is of utmost importance to select good image priors. In this work, we develop the ratio of the dark channel prior (DCP) to the bright channel prior (BCP) as an image prior for solving the BID problem. Specifically, the above two channel priors obtained from RGB images are used to construct an innovative sparse channel prior at first, and then the learned prior is incorporated into the BID tasks. The proposed sparse channel prior enhances the sparsity of the DCP. At the same time, it also shows the inverse relationship between the DCP and BCP. We employ the auxiliary variable technique to integrate the proposed sparse prior information into the iterative restoration procedure. Extensive experiments on real and synthetic blurry sets show that the proposed algorithm is efficient and competitive compared with the state-of-the-art methods and that the proposed sparse channel prior for blind deblurring is effective. [ABSTRACT FROM AUTHOR]

Vector autoregressions (VARs) and their multiple variants are standard models in economic and financial research due to their power for forecasting, data analysis and inference. These properties are a consequence of their capabilities to include multiple variables and lags which, however, turns into an exponential growth of the parameters to be estimated. This means that high-dimensional models with multiple variables and lags are difficult to estimate, leading to omitted variables, information biases and a loss of potential forecasting power. Traditionally, the existing literature has resorted to factor analysis, and specially, to Bayesian methods to overcome this situation. This paper explores the so-called machine learning regularization methods as an alternative to traditional methods of forecasting and impulse response analysis. We find that regularization structures, which allow for high dimensional models, perform better than standard Bayesian methods in nowcasting and forecasting. Moreover, impulse response analysis is robust and consistent with economic theory and evidence, and with the different regularization structures. Specifically, regarding the best regularization structure, an elementwise machine learning structure performs better in nowcasting and in computational efficiency, whilst a componentwise structure performs better in forecasting and cross-validation methods. [ABSTRACT FROM AUTHOR]

Many studies have been conducted on recommender systems in both the academic and industrial fields, as they are currently broadly used in various digital platforms to make personalized suggestions. Despite the improvement in the accuracy of recommenders, the diversity of interest areas recommended to a user tends to be reduced, and the sparsity of explicit feedback from users has been an important issue for making progress in recommender systems. In this paper, we introduce a novel approach, namely re-enrichment learning, which effectively leverages the implicit logged feedback from users to enhance user retention in a platform by enriching their interest areas. The approach consists of (i) graph-based domain transfer and (ii) metadata saliency, which (i) find an adaptive and collaborative domain representing the relations among many users' metadata and (ii) extract attentional features from a user's implicit logged feedback, respectively. The experimental results show that our proposed approach has a better capacity to enrich the diversity of interests of a user by means of implicit feedback and to help recommender systems achieve more balanced personalization. Our approach, finally, helps recommenders improve user retention, i.e., encouraging users to click more items or dwell longer on the platform. [ABSTRACT FROM AUTHOR]

Fluid simulation can be automatically interpolated by using data-driven fluid simulations based on a space-time deformation. In this paper, we propose a novel data-driven fluid simulation scheme with the L 0 based optical flow deformation method by matching two fluid surfaces rather than the L 2 regularization. The L 0 gradient smooth regularization can result in prominent structure of the fluid in a sparsity-control manner, thus the misalignment of the deformation can be suppressed. We adopt the objective function using an alternating minimization with a half-quadratic splitting for solving the L 0 based optical flow deformation model. Experiment results demonstrate that our proposed method can generate more realistic fluid surface with the optimal space-time deformation under the L 0 gradient smooth constraint than the L 2 one, and outperform the state-of-the-art methods in terms of both objective and subjective quality. [ABSTRACT FROM AUTHOR]

Channel estimation is a formidable challenge in mmWave Multiple Input Multiple Output (MIMO) systems due to the large number of antennas. Therefore, compressed sensing (CS) techniques are used to exploit channel sparsity at mmWave frequencies to calculate fewer dominant paths in mmWave channels. However, conventional CS techniques require a higher training overhead for efficient recovery. In this paper, an efficient extended alternation direction method of multipliers (Ex-ADMM) is proposed for mmWave channel estimation. In the proposed scheme, a joint optimization problem is formulated to exploit low rank and channel sparsity individually in the antenna domain. Moreover, a relaxation factor is introduced which improves the proposed algorithm's convergence. Simulation experiments illustrate that the proposed algorithm converges at lower Normalized Mean Squared Error (NMSE) with improved spectral efficiency. The proposed algorithm also ameliorates NMSE performance at low, mid and high Signal to Noise (SNR) ranges. [ABSTRACT FROM AUTHOR]

In this paper, an image recovery problem under the case of salt-and-pepper noise and data missing that degrade image quality is addressed if they are not effectively handled, where the salt-and-pepper noise as the impulsive noise is remodeled as a sparse signal due to its impulsiveness and the data missing pattern, denoted by a sparse vector, contains only zeros and ones to formulate the data missing. In particular, the salt-and-pepper noise and data missing are reformatted by their sparsity, respectively. The wavelet and framelet domains are explored to sparsely represent the image in order to accurately reconstruct the clean image. From the reformulations conducted and to recover the image, under one optimization framework, a joint estimation is developed to perform the image recovery, the salt-and-pepper noise suppression, and the missing patter estimation. To solve the optimization problem, two efficient solvers are developed to obtain the joint estimation solution, and they are based on the alternating direction method of multipliers (ADMM) and accelerated proximal gradient (APG). Finally, numerical studies verify that the joint estimation algorithm outperforms the state-of-the-art approaches in terms of both objective and subjective evaluation standards. [ABSTRACT FROM AUTHOR]

Featured Application: This algorithm can be applied to various kinds of sparse channel estimations, e.g., room impulse response, early reflection, and underwater channel response. This paper presents a new l1-RLS method to estimate a sparse impulse response estimation. A new regularization factor calculation method is proposed for l1-RLS that requires no information of the true channel response in advance. In addition, we also derive a new model to compensate for uncertainty in the regularization factor. The results of the estimation for many different kinds of sparse impulse responses show that the proposed method without a priori channel information is comparable to the conventional method with a priori channel information. [ABSTRACT FROM AUTHOR]

One of the major concerns in photoacoustic computed tomography (PACT) is obtaining a high-quality image using the minimum number of ultrasound transducers/view angles. This issue is of importance when a cost-effective PACT system is needed. On the other hand, analytical reconstruction algorithms such as back projection (BP) and time reversal, when a limited number of view angles is used, cause artifacts in the reconstructed image. Iterative algorithms provide a higher image quality, compared to BP, due to a model used for image reconstruction. The performance of the model can be further improved using the sparsity concept. In this paper, we propose using a novel sparse dictionary to capture important features of the photoacoustic signal and eliminate the artifacts while few transducers is used. Our dictionary is an optimum combination of Wavelet Transform (WT), Discrete Cosine Transform (DCT), and Total Variation (TV). We utilize two quality assessment metrics including peak signal-to-noise ratio and edge preservation index to quantitatively evaluate the reconstructed images. The results show that the proposed method can generate high-quality images having fewer artifacts and preserved edges, when fewer view angles are used for reconstruction in PACT. [ABSTRACT FROM AUTHOR]

Hyperspectral remote sensing is based on measuring the scattered and reflected electromagnetic signals from the Earth's surface emitted by the Sun. The received radiance at the sensor is usually degraded by atmospheric effects and instrumental (sensor) noises which include thermal (Johnson) noise, quantization noise, and shot (photon) noise. Noise reduction is often considered as a preprocessing step for hyperspectral imagery. In the past decade, hyperspectral noise reduction techniques have evolved substantially from two dimensional bandwise techniques to three dimensional ones, and varieties of low-rank methods have been forwarded to improve the signal to noise ratio of the observed data. Despite all the developments and advances, there is a lack of a comprehensive overview of these techniques and their impact on hyperspectral imagery applications. In this paper, we address the following two main issues; (1) Providing an overview of the techniques developed in the past decade for hyperspectral image noise reduction; (2) Discussing the performance of these techniques by applying them as a preprocessing step to improve a hyperspectral image analysis task, i.e., classification. Additionally, this paper discusses about the hyperspectral image modeling and denoising challenges. Furthermore, different noise types that exist in hyperspectral images have been described. The denoising experiments have confirmed the advantages of the use of low-rank denoising techniques compared to the other denoising techniques in terms of signal to noise ratio and spectral angle distance. In the classification experiments, classification accuracies have improved when denoising techniques have been applied as a preprocessing step. [ABSTRACT FROM AUTHOR]

Uplink and Downlink channel estimation in massive Multiple Input Multiple Output (MIMO) systems is an intricate issue because of the increasing channel matrix dimensions. The channel feedback overhead using traditional codebook schemes is very large, which consumes more bandwidth and decreases the overall system efficiency. The purpose of this paper is to decrease the channel estimation overhead by taking the advantage of sparse attributes and also to optimize the Energy Efficiency (EE) of the system. To cope with this issue, we propose a novel approach by using Compressed-Sensing (CS), Block Iterative-Support-Detection (Block-ISD), Angle-of-Departure (AoD) and Structured Compressive Sampling Matching Pursuit (S-CoSaMP) algorithms to reduce the channel estimation overhead and compare them with the traditional algorithms. The CS uses temporal-correlation of time-varying channels to produce Differential-Channel Impulse Response (DCIR) among two CIRs that are adjacent in time-slots. DCIR has greater sparsity than the conventional CIRs as it can be easily compressed. The Block-ISD uses spatial-correlation of the channels to obtain the block-sparsity which results in lower pilot-overhead. AoD quantizes the channels whose path-AoDs variation is slower than path-gains and such information is utilized for reducing the overhead. S-CoSaMP deploys structured-sparsity to obtain reliable Channel-State-Information (CSI). MATLAB simulation results show that the proposed CS based algorithms reduce the feedback and pilot-overhead by a significant percentage and also improve the system capacity as compared with the traditional algorithms. Moreover, the EE level increases with increasing Base Station (BS) density, UE density and lowering hardware impairments level. [ABSTRACT FROM AUTHOR]

In this work, a multiple speech source separation method using inter-channel correlation and relaxed sparsity is proposed. A B-format microphone with four spatially located channels is adopted due to the size of the microphone array to preserve the spatial parameter integrity of the original signal. Specifically, we firstly measure the proportion of overlapped components among multiple sources and find that there exist many overlapped time-frequency (TF) components with increasing source number. Then, considering the relaxed sparsity of speech sources, we propose a dynamic threshold-based separation approach of sparse components where the threshold is determined by the inter-channel correlation among the recording signals. After conducting a statistical analysis of the number of active sources at each TF instant, a form of relaxed sparsity called the half-K assumption is proposed so that the active source number in a certain TF bin does not exceed half the total number of simultaneously occurring sources. By applying the half-K assumption, the non-sparse components are recovered by regarding the extracted sparse components as a guide, combined with vector decomposition and matrix factorization. Eventually, the final TF coefficients of each source are recovered by the synthesis of sparse and non-sparse components. The proposed method has been evaluated using up to six simultaneous speech sources under both anechoic and reverberant conditions. Both objective and subjective evaluations validated that the perceptual quality of the separated speech by the proposed approach outperforms existing blind source separation (BSS) approaches. Besides, it is robust to different speeches whilst confirming all the separated speeches with similar perceptual quality. [ABSTRACT FROM AUTHOR]

Rendering spatial sound scenes via audio objects has become popular in recent years, since it can provide more flexibility for different auditory scenarios, such as 3D movies, spatial audio communication and virtual classrooms. To facilitate high-quality bitrate-efficient distribution for spatial audio objects, an encoding scheme based on intra-object sparsity (approximate k-sparsity of the audio object itself) is proposed in this paper. The statistical analysis is presented to validate the notion that the audio object has a stronger sparseness in the Modified Discrete Cosine Transform (MDCT) domain than in the Short Time Fourier Transform (STFT) domain. By exploiting intra-object sparsity in the MDCT domain, multiple simultaneously occurring audio objects are compressed into a mono downmix signal with side information. To ensure a balanced perception quality of audio objects, a Psychoacoustic-based time-frequency instants sorting algorithm and an energy equalized Number of Preserved Time-Frequency Bins (NPTF) allocation strategy are proposed, which are employed in the underlying compression framework. The downmix signal can be further encoded via Scalar Quantized Vector Huffman Coding (SQVH) technique at a desirable bitrate, and the side information is transmitted in a lossless manner. Both objective and subjective evaluations show that the proposed encoding scheme outperforms the Sparsity Analysis (SPA) approach and Spatial Audio Object Coding (SAOC) in cases where eight objects were jointly encoded. [ABSTRACT FROM AUTHOR]

In image and signal processing, the beta-divergence is well known as a similarity measure between two positive objects. However, it is unclear whether or not the distance-like structure of beta-divergence is preserved, if we extend the domain of the beta-divergence to the negative region. In this article, we study the domain of the beta-divergence and its connection to the Bregman-divergence associated with the convex function of Legendre type. In fact, we show that the domain of beta-divergence (and the corresponding Bregman-divergence) include negative region under the mild condition on the beta value. Additionally, through the relation between the beta-divergence and the Bregman-divergence, we can reformulate various variational models appearing in image processing problems into a unified framework, namely the Bregman variational model. This model has a strong advantage compared to the beta-divergence-based model due to the dual structure of the Bregman-divergence. As an example, we demonstrate how we can build up a convex reformulated variational model with a negative domain for the classic nonconvex problem, which usually appears in synthetic aperture radar image processing problems. [ABSTRACT FROM AUTHOR]

Modeling of a time-varying dynamical system provides insights into the functions of biological neural networks and contributes to the development of next-generation neural prostheses. In this paper, we have formulated a novel sparse multiwavelet-based generalized Laguerre-Volterra (sMGLV) modeling framework to identify the time-varying neural dynamics from multiple spike train data. First, the significant inputs are selected by using a group least absolute shrinkage and selection operator (LASSO) method, which can capture the sparsity within the neural system. Second, the multiwavelet-based basis function expansion scheme with an efficient forward orthogonal regression (FOR) algorithm aided by mutual information is utilized to rapidly capture the time-varying characteristics from the sparse model. Quantitative simulation results demonstrate that the proposed sMGLV model in this paper outperforms the initial full model and the state-of-the-art modeling methods in tracking performance for various time-varying kernels. Analyses of experimental data show that the proposed sMGLV model can capture the timing of transient changes accurately. The proposed framework will be useful to the study of how, when, and where information transmission processes across brain regions evolve in behavior. [ABSTRACT FROM AUTHOR]

In order to alleviate the pressure of information overload and enhance consumer satisfaction, personalization recommendation has become increasingly popular in recent years. As a result, various approaches for recommendation have been proposed in the past few years. However, traditional recommendation methods are still troubled with typical issues such as cold start, sparsity, and low accuracy. To address these problems, this paper proposed an improved recommendation method based on trust relationships in social networks to improve the performance of recommendations. In particular, we define trust relationship afresh and consider several representative factors in the formalization of trust relationships. To verify the proposed approach comprehensively, this paper conducted experiments in three ways. The experimental results show that our proposed approach leads to a substantial increase in prediction accuracy and is very helpful in dealing with cold start and sparsity. [ABSTRACT FROM AUTHOR]

The segmentation of point clouds is an important aspect of automated processing tasks such as semantic extraction. However, the sparsity and non-uniformity of the point clouds gathered by the popular 3D mobile LiDAR devices pose many challenges for existing segmentation methods. To improve the segmentation results of point clouds from mobile LiDAR devices, we propose an optimized segmentation method based on Scanline Continuity Constraint (SLCC) in this work. Unlike conventional scanline-based segmentation methods, SLCC clusters scanlines using the continuity constraints in terms of the distance as well as the direction of two consecutive points. In addition, scanline clusters are agglomerated not only into primitive geometrical shapes but also irregular shapes. Another downside to existing segmentation methods is that they are not capable of incremental processing. This causes unnecessary memory and time consumption for applications that require frame-wise segmentation or when new point clouds are added. In order to address this, we propose an incremental scheme--the Incremental Recursive Segmentation (IRIS), that can be easily applied to any segmentation method. IRIS is achieved by combining the segments of newly added point clouds and the previously segmented results. Furthermore, as an example application, we construct a processing pipeline consisting of plane fitting and surface reconstruction using the segmentation results. Finally, we evaluate the proposed methods on three datasets acquired from a handheld Velodyne HDL-32E LiDAR device. The experimental results verify the efficiency of IRIS for any segmentation method and the advantages of SLCC for processing mobile LiDAR data. [ABSTRACT FROM AUTHOR]