Search

In the semiconductor industry, virtual metrology (VM) is a cost-effective and efficient technique for monitoring the processes from one wafer to another. This technique is implemented by generating a predictive model that uses real-time data from equipment sensors in conjunction with measured wafer quality characteristics. Before establishing a prediction model for the VM system, appropriate selection of relevant input variables should be performed to maintain the efficiency of subsequent analyses considering the large dimensionality of the sensor data inputs. However, wafer production processes usually employ multiple sensors, which leads to cost escalations. Herein, we propose a variant of the sparse principal component analysis (PCA) called true sparse PCA (TSPCA). The proposed method uses a small number of input variables in the first few principal components. The main contribution of the proposed TSPCA is reducing the number of essential sensors. Our experimental results demonstrate that compared to the existing sparse PCA methods, the proposed approach can reduce the number of sensors required while explaining an approximately equivalent amount of variance. [ABSTRACT FROM AUTHOR]

The evaluation of sound quality is a pivotal area of research within audio and acoustics. The sound quality evaluation methods commonly used include both objective and subjective, the latter being time-consuming and costly as they rely on listening tests. This research work aims to investigate the use of predictive sound quality models as a way to objectively assess the Desire-to-buy of side-byside vehicles, in a more efficient, faster, and less costly way than conventional methods. Multiple linear regression algorithms were used to validate the objective models derived from objective physical metrics and perceptual psycho-physical metrics. The sensory profile objective models reported in this paper were constructed using parsimonious linear Lasso and Elastic-net algorithms. Our results show that linear objective models effectively account for each of the perceptual attributes of the sensory profiles and the Desire-to-buy, while only requiring a few physical and psychophysical metrics. [ABSTRACT FROM AUTHOR]

This article proposes penalized Mallow's model averaging (pMMA) in the linear regression framework given non nested candidate models. Compared to the MMA, additional constraints are imposed on model weights. We introduce a general framework and allow for non convex constraints such as SCAD, MCP, and TLP. We establish the asymptotic optimality of our proposed penalized MMA (pMMA) estimator and show that the pMMA can achieve a higher sparsity level than the classic MMA. A coordinate-wise descent algorithm has been developed to compute the pMMA estimator efficiently. We conduct simulation and empirical studies to show that our pMMA estimator produces a more sparse weight vector than the MMA, but with better out-of-sample performance. [ABSTRACT FROM AUTHOR]

Determining Jacobian sparsity structure is an important step in the efficient computation of sparse Jacobians. We introduce a new method for determining Jacobian sparsity patterns by combining bit vector probing with Bloom filters. We further refine Bloom filter probing by combining it with hierarchical probing to yield a highly effective strategy for Jacobian sparsity pattern determination. [ABSTRACT FROM AUTHOR]

Combining multiple data sources, Digital Earth is an integrated observation platform based on air–space–ground–sea monitoring systems. Among these data sources, the Inverse Synthetic Aperture Radar (ISAR) is a crucial observation method. ISAR is typically utilized to monitor both military and civilian ships due to its all-day and all-weather superiority. However, in complex scenarios, multiple targets may exist within the same radar antenna beam, resulting in severe defocusing due to different motion conditions. Therefore, this paper proposes a multiple-target ISAR imaging method with the removal of micro-motion connections based on the integration of joint constraints. The fully motion-compensated targets exhibit low rank and local similarity in the high-resolution range profile (HRRP) domain, while the micro-motion components possess sparsity. Additionally, targets display sparsity in the image domain. Inspired by this, we formulate a novel optimization by promoting the low-rank, the Laplacian, and the sparsity constraints of targets and the sparsity constraints of the micro-motion components. This optimization problem is solved by the linearized alternative direction method with adaptive penalty (LADMAP). Furthermore, the different motions of various targets degrade their inherent characteristics. Therefore, we integrate motion compensation transformation into the optimization, accordingly achieving the separation of rigid bodies and the micro-motion components of different targets. Experiments based on simulated data demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Analytical differentiation for a smooth and accurate sensitivity field is typically used for efficient structural and multidisciplinary optimization. However, it can be challenging for multiphysics and non-linear problems. An alternative approach is automatic differentiation (AD). For large problems with many design variables AD can be computationally expensive and memory demanding and thus its use is still limited. To address some of these challenges, we propose to exploit the sparsity of the level set topology optimization (LSTO) in combination with a hybrid mode when using AD. The modularized LSTO used here enables the use of AD in combination with the classical level set method. In our method, we utilize the operator overloading (OO) approach, and we start our development by comparing different OO libraries. Next, a sparse AD implementation is proposed to take advantage of the sparsity of the level set method, in which sensitivities are only required within a narrow band close to the level set boundary. The obtained results indicate that this sparsity can improve the efficiency of the implementation. However, the reduction in memory requirements is not as significant. To improve the memory consumption as well, we introduce a hybrid mode, where instead of computing the total sensitivity directly with OO, the expression is first derived analytically and then OO is used to obtain the partial derivatives. Our studies show that combining the hybrid mode with the sparsity of the LSTO results in improvements in efficiency, with almost an order of magnitude less computational time for the biggest mesh size studied. Finally, the combination of the forward and reverse modes depending on the partial derivatives at hand is exploited to further improve memory requirements and computational cost. [ABSTRACT FROM AUTHOR]

In this paper we study the optimal control of a parabolic initial-boundary value problem of viscous Cahn–Hilliard type with zero Neumann boundary conditions. Phase field systems of this type govern the evolution of diffusive phase transition processes with conserved order parameter. It is assumed that the nonlinear functions driving the physical processes within the spatial domain are double-well potentials of logarithmic type whose derivatives become singular at the boundary of their respective domains of definition. For such systems, optimal control problems have been studied in the past. We focus here on the situation when the cost functional of the optimal control problem contains a nondifferentiable term like the L 1 -norm, which leads to sparsity of optimal controls. For such cases, we establish first-order necessary and second-order sufficient optimality conditions for locally optimal controls. In the approach to second-order sufficient conditions, the main novelty of this paper, we adapt a technique introduced by Casas et al. in the paper (SIAM J Control Optim 53:2168–2202, 2015). In this paper, we show that this method can also be successfully applied to systems of viscous Cahn–Hilliard type with logarithmic nonlinearity. Since the Cahn–Hilliard system corresponds to a fourth-order partial differential equation in contrast to the second-order systems investigated before, additional technical difficulties have to be overcome. [ABSTRACT FROM AUTHOR]

Sonar automatic target recognition (ATR) systems suffer from complex acoustic scattering, background clutter, and waveguide effects that are ever-present in the ocean. Traditional signal processing techniques often struggle to distinguish targets when noise and complicated target geometries are introduced. Recent advancements in machine learning and wavelet theory offer promising directions for extracting informative features from sonar return data. This work introduces a feature extraction and dimensionality reduction technique using the invariant scattering transform and Sparse Multi-view Discriminant Analysis for identifying highly informative features in the PONDEX09/PONDEX10 datasets. The extracted features are used to train a support vector machine classifier that achieves an average classification accuracy of 97.3% using six unique targets. [ABSTRACT FROM AUTHOR]

Recent advancements in neuroimaging have led to greater data sharing among the scientific community. However, institutions frequently maintain control over their data, citing concerns related to research culture, privacy, and accountability. This creates a demand for innovative tools capable of analyzing amalgamated datasets without the need to transfer actual data between entities. To address this challenge, we propose a decentralized sparse federated learning (FL) strategy. This approach emphasizes local training of sparse models to facilitate efficient communication within such frameworks. By capitalizing on model sparsity and selectively sharing parameters between client sites during the training phase, our method significantly lowers communication overheads. This advantage becomes increasingly pronounced when dealing with larger models and accommodating the diverse resource capabilities of various sites. We demonstrate the effectiveness of our approach through the application to the Adolescent Brain Cognitive Development (ABCD) dataset. [ABSTRACT FROM AUTHOR]

Multivariate regression is a widely used technique for large-scale statistical applications. In this paper, we propose a novel method for the multivariate regression model, called multivariate weighted lasso (MWL). We consider the heteroscedasticity in the noise matrix and introduce an adjustable weight to calibrate the fitting residuals of different responses. The proposed method shows an advantage in complex multivariate regressions with heteroscedasticity. To implement the procedure, an efficient algorithm is provided based on the multivariate coordinate descent. We provide theoretical guarantees for the proposed method. Numerical simulations and financial applications are conducted using the proposed and other existing methods. The results indicate that the proposed method performs well in both variable selection and coefficient estimation. [ABSTRACT FROM AUTHOR]

The high dimensionality of genetic data poses many challenges for subgroup identification, both computationally and theoretically. This paper proposes a double-penalized regression model for subgroup analysis and variable selection for heterogeneous high-dimensional data. The proposed approach can automatically identify the underlying subgroups, recover the sparsity, and simultaneously estimate all regression coefficients without prior knowledge of grouping structure or sparsity construction within variables. We optimize the objective function using the alternating direction method of multipliers with a proximal gradient algorithm and demonstrate the convergence of the proposed procedure. We show that the proposed estimator enjoys the oracle property. Simulation studies demonstrate the effectiveness of the novel method with finite samples, and a real data example is provided for illustration. [ABSTRACT FROM AUTHOR]

Apart from verbal communication among humans, non-verbal interactions also play a significant role in conveying meaningful information. Non-verbal cues mainly comprise gestures, body postures, and facial expressions. Hand gestures constitute the preferred mechanism for non-verbal communication, and today, they also find utility in human–computer interaction (HCI), gaming, virtual reality, robotics, sign language, etc. While extensive research has been conducted on utilizing deep learning for hand gesture recognition, there has been a notable scarcity of efforts focused on leveraging the sparse characteristics of deeply acquired features to distinguish hand postures, even in the presence of challenges such as varying hand sizes, diverse spatial positions within images, and background clutter. We demonstrate the effect of data augmentation, transfer learning, and sparsity on the performance of the proposed algorithm using publicly available hand gesture datasets. We also provide a quantitative comparative analysis of the proposed approach with state-of-the-art algorithms for static hand gesture recognition. We illustrate a noteworthy finding wherein dictionary learning through LC-KSVD, when applied to fine-tuned features extracted from a deep architecture, outperforms the results achieved by state-of-the-art architectures in the context of hand gesture classification. We have realized substantial enhancements with our proposed methodology when compared to a baseline convolutional model. For instance, in the case of the EgoGesture dataset, we attained an accuracy of 94.9 % , as opposed to the baseline accuracy of 63.3 % , through the utilization of sparsity in deep features. [ABSTRACT FROM AUTHOR]

Purpose: We present SCAMPI (Sparsity Constrained Application of deep Magnetic resonance Priors for Image reconstruction), an untrained deep Neural Network for MRI reconstruction without previous training on datasets. It expands the Deep Image Prior approach with a multidomain, sparsity‐enforcing loss function to achieve higher image quality at a faster convergence speed than previously reported methods. Methods: Two‐dimensional MRI data from the FastMRI dataset with Cartesian undersampling in phase‐encoding direction were reconstructed for different acceleration rates for single coil and multicoil data. Results: The performance of our architecture was compared to state‐of‐the‐art Compressed Sensing methods and ConvDecoder, another untrained Neural Network for two‐dimensional MRI reconstruction. SCAMPI outperforms these by better reducing undersampling artifacts and yielding lower error metrics in multicoil imaging. In comparison to ConvDecoder, the U‐Net architecture combined with an elaborated loss‐function allows for much faster convergence at higher image quality. SCAMPI can reconstruct multicoil data without explicit knowledge of coil sensitivity profiles. Moreover, it is a novel tool for reconstructing undersampled single coil k‐space data. Conclusion: Our approach avoids overfitting to dataset features, that can occur in Neural Networks trained on databases, because the network parameters are tuned only on the reconstruction data. It allows better results and faster reconstruction than the baseline untrained Neural Network approach. [ABSTRACT FROM AUTHOR]

We study the Compressed Sensing (CS) problem, which is the problem of finding the most sparse vector that satisfies a set of linear measurements up to some numerical tolerance. CS is a central problem in Statistics, Operations Research and Machine Learning which arises in applications such as signal processing, data compression, image reconstruction, and multi-label learning. We introduce an ℓ 2 regularized formulation of CS which we reformulate as a mixed integer second order cone program. We derive a second order cone relaxation of this problem and show that under mild conditions on the regularization parameter, the resulting relaxation is equivalent to the well studied basis pursuit denoising problem. We present a semidefinite relaxation that strengthens the second order cone relaxation and develop a custom branch-and-bound algorithm that leverages our second order cone relaxation to solve small-scale instances of CS to certifiable optimality. When compared against solutions produced by three state of the art benchmark methods on synthetic data, our numerical results show that our approach produces solutions that are on average 6.22 % more sparse. When compared only against the experiment-wise best performing benchmark method on synthetic data, our approach produces solutions that are on average 3.10 % more sparse. On real world ECG data, for a given ℓ 2 reconstruction error our approach produces solutions that are on average 9.95 % more sparse than benchmark methods ( 3.88 % more sparse if only compared against the best performing benchmark), while for a given sparsity level our approach produces solutions that have on average 10.77 % lower reconstruction error than benchmark methods ( 1.42 % lower error if only compared against the best performing benchmark). When used as a component of a multi-label classification algorithm, our approach achieves greater classification accuracy than benchmark compressed sensing methods. This improved accuracy comes at the cost of an increase in computation time by several orders of magnitude. Thus, for applications where runtime is not of critical importance, leveraging integer optimization can yield sparser and lower error solutions to CS than existing benchmarks. [ABSTRACT FROM AUTHOR]

As data-driven deep learning (DL) has been applied in various scenarios, the privacy threats have become a widely recognized problem. To boost privacy protection in federated learning (FL), some methods adopt a one-shot differential privacy (DP) approach to obfuscate model updates, yet they do not take into account the dynamic balance between efficiency and privacy protection. To this end, we propose ASPFL—an efficient FL approach with adaptive sparsity-based pruning and differential privacy protection. We further propose the adaptive pruning mechanism by utilizing the Jensen-Shannon divergence as the metric to generate sparse matrices, which are then employed in the model updates. In addition, we introduce adaptive Gaussian noise by assessing the variation of sensitivity through post-pruning uploading. Extensive experiments validate that our proposed ASPFL boosts convergence speed by more than two times under non-IID data. Compared with existing DP-FL methods, ASPFL can maximally achieve over 82% accuracy on CIFAR-10, while the communication cost is greatly reduced by 40% under the same level of privacy protection. [ABSTRACT FROM AUTHOR]

A technique is presented that reduces the required memory of neural networks through improving weight storage. In contrast to traditional methods, which have an exponential memory overhead with the increase in network size, the proposed method stores only the number of connections between neurons. The proposed method is evaluated on feedforward networks and demonstrates memory saving capabilities of up to almost 80% while also being more efficient, especially with larger architectures. [ABSTRACT FROM AUTHOR]

One of the main challenges in using adaptive filtering algorithms is efficiently emulating a system subject to noisy disturbances. This can be facilitated in applications where the system response to impulse is sparse, which allows for acceleration of the convergence rate if appropriate strategies are used. As a result, methods that impose norm constraints on the estimates are widely used. However, in the case of non-stationary plants to be identified, there is a gap in terms of theoretical performance guarantees of these algorithms. This paper proposes a novel stochastic model capable of predicting the performance of the ℓ 0 -LMS algorithm in identifying a plant subjected to a first-order Markovian disturbance. Therefore, a tracking analysis is carried out, including both the average performance of the adaptive coefficients and second-order statistics of these coefficients. The theoretical model offers an analytical equation that predicts the asymptotic mean squared deviation in terms of the variance of the Markovian disturbance. Further, for most simulated scenarios, the theoretical model's error in mean squared deviation remains below 1 dB, even when the learning step varies across several orders of magnitude. It was possible to observe that the theoretical model can accurately predict the steady-state regime for a wide range of learning step values and calculate an optimal value for this parameter. The findings are confirmed through extensive simulations. [ABSTRACT FROM AUTHOR]

Recommendation systems are ubiquitous in various domains, facilitating users in finding relevant items according to their preferences. Identifying pertinent items that meet their preferences enables users to target the right items. To predict ratings for more accurate forecasts, recommender systems often use collaborative filtering (CF) approaches to sparse user-rated item matrices. Due to a lack of knowledge regarding newly formed entities, the data sparsity of the user-rated item matrix has an enormous effect on collaborative filtering algorithms, which frequently face lazy learning issues. Real-world datasets with exponentially increasing users and reviews make this situation worse. Matrix factorization (MF) stands out as a key strategy in recommender systems, especially for CF tasks. This paper presents a neural network matrix factorization (NNMF) model through machine learning to overcome data sparsity challenges. This approach aims to enhance recommendation quality while mitigating the impact of data sparsity, a common issue in CF algorithms. A thorough comparative analysis was conducted on the well-known MovieLens dataset, spanning from 1.6 to 9.6 M records. The outcomes consistently favored the NNMF algorithm, showcasing superior performance compared to the state-of-the-art method in this domain in terms of precision, recall, F 1 score , MAE, and RMSE. [ABSTRACT FROM AUTHOR]

Abstract The estimation sound fields over space is of interest in sound field control and analysis, spatial audio, room acoustics and virtual reality. Sound fields can be estimated from a number of measurements distributed over space yet this remains a challenging problem due to the large experimental effort required. In this work we investigate sensor distributions that are optimal to estimate sound fields. Such optimization is valuable as it can greatly reduce the number of measurements required. The sensor positions are optimized with respect to the parameters describing a sound field, or the pressure reconstructed at the area of interest, by finding the positions that minimize the Bayesian Cramér-Rao bound (BCRB). The optimized distributions are investigated in a numerical study as well as with measured room impulse responses. We observe a reduction in the number of measurements of approximately 50% when the sensor positions are optimized for reconstructing the sound field when compared with random distributions. The results indicate that optimizing the sensors positions is also valuable when the vector of parameters is sparse, specially compared with random sensor distributions, which are often adopted in sparse array processing in acoustics.

Anomaly detection is a binary classification task, which is to determine whether each pixel is abnormal or not. The difficulties are that it is hard to obtain abnormal samples and predict the shape of abnormal regions. Due to these difficulties, traditional supervised segmentation methods fail. The usual weakly supervised segmentation methods will use artificially generate defects to construct training samples. However, the model will be overfitted to artificially generated defects during training, resulting in insufficient generalization ability of the model. In this paper, we presented a novel reconstruction-based weakly supervised method for sparse anomaly detection. We proposed to use generative adversarial networks (GAN) to learn the distribution of positive samples, and reconstructed negative samples which contained the sparse defect into positive ones. Due to the nature of GAN, the training dataset only needs to contain normal samples. Subsequently, the segmentation network performs progressive feature fusion on reconstructed and original samples to complete the anomaly detection. Specifically, we designed the loss function based on kullback-leibler divergence for sparse anomalous defects. The final weakly-supervised segmentation network only assumes a sparsity prior of the defect region; thus, it can circumvent the detailed semantic labels and alleviate the potential overfitting problem. We compared our method with the state of the art generation-based generative anomaly detection methods and observed the average area under the receiver operating characteristic curve increase of 3% on MVTec anomaly detection.

Dictionary-based methods are recognized for their ability to estimate clear speech from speech contaminated with noise. However, these techniques face a limitation in distinguishing between speech and noise components. The presented algorithm addresses this challenge by introducing an innovative solution through an iterative thresholding method. In this method, the thresholding is based on noise characteristics and remains independent of variations in noise power. To determine the stopping criteria for thresholding, the algorithm leverages the structure of the speech signal in time–frequency domains using the Gini Index. Remarkably, this technique adeptly extracts reliable signal components from a noisy time–frequency magnitude spectrum, performing well under non-varying and varying Signal-to-Noise Ratio (SNR) conditions. Importantly, it achieves this without the need for mask functions, voice activity detection techniques, noise or mixture dictionaries, or SNR information, which are essential components in other dictionary-based methods. Following the thresholding process, the clean speech is assessed using a dictionary-based approach to restore perceptual loss. In evaluating its performance, the algorithm is compared with traditional enhancement techniques concerning perceptual evaluation of speech quality and short-time objective intelligibility. The assessment is conducted under various background noise conditions, including babble, white, factory, and Volvo noises. The proposed algorithm exhibits superior performance, particularly in low and varying SNR scenarios. [ABSTRACT FROM AUTHOR]

Sparse linear prediction methods suffer from decreased prediction accuracy when the predictor variables have cluster structure (e.g., highly correlated groups of variables). To improve prediction accuracy, various methods have been proposed to identify variable clusters from the data and integrate cluster information into a sparse modeling process. But none of these methods achieve satisfactory performance for prediction, variable selection and variable clustering performed simultaneously. This paper presents Variable Cluster Principal Component Regression (VC‐PCR), a prediction method that uses variable selection and variable clustering in order to solve this problem. Experiments with real and simulated data demonstrate that, compared to competitor methods, VC‐PCR is the only method that achieves simultaneously good prediction, variable selection, and clustering performance when cluster structure is present. [ABSTRACT FROM AUTHOR]

AbstractWe are interested in the problem of two-sample network hypothesis testing: given two networks with the same set of nodes, we wish to test whether the underlying Bernoulli probability matrices of the two networks are the same or not. We propose Interlacing Balance Measure (IBM) as a new two-sample testing approach. We consider the Degree-Corrected Mixed-Membership (DCMM) model for undirected networks, where we allow severe degree heterogeneity, mixed-memberships, flexible sparsity levels, and weak signals. In such a broad setting, how to find a test that has a tractable limiting null and optimal testing performances is a challenging problem. We show that IBM is such a test: in a broad DCMM setting with only mild regularity conditions, IBM has N(0,1) as the limiting null and achieves the optimal phase transition.While the above is for undirected networks, IBM is a unified approach and is directly implementable for directed networks. For a broad directed-DCMM (extension of DCMM for directed networks) setting, we show that IBM has N(0,1/2) as the limiting null and continues to achieve the optimal phase transition. We have also applied IBM to the Enron email network and a gene co-expression network, with interesting results. [ABSTRACT FROM AUTHOR]

The estimation sound fields over space is of interest in sound field control and analysis, spatial audio, room acoustics and virtual reality. Sound fields can be estimated from a number of measurements distributed over space yet this remains a challenging problem due to the large experimental effort required. In this work we investigate sensor distributions that are optimal to estimate sound fields. Such optimization is valuable as it can greatly reduce the number of measurements required. The sensor positions are optimized with respect to the parameters describing a sound field, or the pressure reconstructed at the area of interest, by finding the positions that minimize the Bayesian Cramér-Rao bound (BCRB). The optimized distributions are investigated in a numerical study as well as with measured room impulse responses. We observe a reduction in the number of measurements of approximately 50% when the sensor positions are optimized for reconstructing the sound field when compared with random distributions. The results indicate that optimizing the sensors positions is also valuable when the vector of parameters is sparse, specially compared with random sensor distributions, which are often adopted in sparse array processing in acoustics. [ABSTRACT FROM AUTHOR]

Summary: Cointegration has shown useful for modeling non‐stationary data with long‐run equilibrium relationships among variables, with applications in many fields such as econometrics, climate research and biology. However, the analyses of vector autoregressive models are becoming more difficult as data sets of higher dimensions are becoming available, in particular because the number of parameters is quadratic in the number of variables. This leads to lack of statistical robustness, and regularisation methods are paramount for obtaining valid estimates. In the last decade, many papers have appeared suggesting different penalisation approaches to the inference problem. Here, we make a comprehensive review of different penalisation methods adapted to the specific structure of vector cointegrated models suggested in the literature, with relevant references to software packages. The methods are evaluated and compared according to a range of error measures in a simulation study, considering combinations of low and high dimension of the system and small and large sample sizes. [ABSTRACT FROM AUTHOR]

This paper considers the deep-learning-aided compressed video sensing problem. To this end, a deep generative adversarial network has been proposed to provide an approximation of the non-reference frame using its corresponding reference frame. The tests confirm the superiority of this scheme over the conventional methods used earlier. Furthermore, two scenarios have been suggested for deep compressed video sensing and recovery. In the first scenario, the difference between the non-reference frame and its approximation obtained from the pre-trained network is compressively sampled and transmitted to the receiver where the proposed residual reconstruction network is adopted to reconstruct the signal. The second scenario utilizes a pre-trained network followed by an augmented layer to approximate the non-reference frames. In the transmitter, the parameters of the augmented layer are trained for the current non-reference block. Instead of transmitting the samples of the block, the parameters of its trained augmented layer are sent to the receiver where the reconstruction is done using the same pre-trained network. The performances of the proposed scenarios demonstrate their objective and subjective superiority over the state-of-the-art algorithms in both the reconstruction quality and run time. [ABSTRACT FROM AUTHOR]

In this paper, we present a novel approach for the reconstruction of sparse graph signals using a distributed adaptive thresholding recursive least squares algorithm. Our proposed scheme leverages signal measurements across multiple time steps to estimate the underlying graph signal. To achieve this, we define a cost function that combines a weighted least squares term and an L 0 norm. The cost function is then minimized using a surrogate minimization scheme and the recursive least squares technique. The resulting algorithm is an adaptive thresholding recursive least squares scheme that effectively recovers sparse graph signals. Additionally, we apply a consensus-based method to enable distributed recovery, and derive recursive updating relations to speed up the algorithm. Our approach offers a significant novelty in the field of graph signal recovery, providing an efficient and accurate solution for reconstructing sparse graph signals. The numerical simulations show the superiority of the proposed algorithm over the other state-of-the-art algorithms from the point of view of the convergence rate and the mean square deviation (MSD) criterion. [ABSTRACT FROM AUTHOR]

This article presents a novel approach to image restoration utilizing a unique non-convex l 1 / 2 -TV regularization model. This model integrates the l 1 / 2 -quasi norm as a regularization function, introducing non-convexity to promote sparsity and unevenly penalize elements, thereby enhancing restoration outcomes. To tackle this model, an efficient algorithm named the Alternating Direction Method of Multipliers, based on the Lagrangian multiplier, is introduced. This effectively prevents the penalty parameter from reaching infinity and ensures excellent convergence behavior. The proposed algorithm decomposes the optimization problem into subproblems, for which closed-form solutions are derived, particularly addressing the challenging l 1 / 2 regularization problem. To validate its effectiveness, a comprehensive set of experiments are conducted to compare its performance with existing methods. The experimental results demonstrate that the proposed model performs well in both qualitative and quantitative evaluations. Consequently, the proposed model is not only efficient and stable but also exhibits excellent convergence behavior. • A novel regularization model for image restoration, based on non-convex total variation is proposed in this work. • We propose an effective solution approach to solve the model using the Alternating Direction Method of Multipliers (ADMM). • The proposed solution provides the closed-form thresholding formula for regularization model. • Extensive numerical experiments demonstrate the superiority of the proposed method over compared methods. [ABSTRACT FROM AUTHOR]

Collaborative filtering (CF) is a method to be used in recommendation systems. CF works by analyzing rating data patterns from previous users to produce recommendations according to their interests. However, it faces a crucial problem, sparsity, a condition where a lot of data is empty, which will affect the quality of the recommendations produced. To state this problem, the purpose of this study is to input methods including mean, min, max, and knearest neighbor imputation (KNNI). The steps taken include imputation of empty data, followed by similarity calculations using the cosin similarity method, and evaluation using root mean square error (RMSE). The experimental result shows that the mean method is excellent with an average similarity value of 0.99 and an RMSE value of 0.98. [ABSTRACT FROM AUTHOR]

Multi-dimensional images can be viewed as tensors and have often embedded a low-rankness property that can be evaluated by tensor low-rank measures. In this paper, we first introduce a tensor low-rank and sparsity measure and then propose low-rank and sparsity models for tensor completion, tensor robust principal component analysis, and tensor denoising. The resulting tensor recovery models are further solved by the augmented Lagrangian method with a convergence guarantee. And its augmented Lagrangian subproblem is computed by the proximal alternative method, in which each variable has a closed-form solution. Numerical experiments on several multi-dimensional image recovery applications show the superiority of the proposed methods over the state-of-the-art methods in terms of several quantitative quality indices and visual quality. [ABSTRACT FROM AUTHOR]

Quantisation is the process of mapping an input signal from an infinite continuous set to a countable set with a finite number of elements. It is a non-linear irreversible process, which makes the traditional methods of system identification no longer applicable. In this work, we propose a method for parsimonious linear time invariant system identification when only quantised observations, discerned from noisy data, are available. More formally, given a priori information on the system, represented by a compact set containing the poles of the system, and quantised realizations, our algorithm aims at identifying the least order system that is compatible with the available information. The proposed approach takes also into account that the available data can be subject to fragmentation. Our proposed algorithm relies on an ADMM approach to solve a $ \ell _{p},(0 \lt p \lt 1), $ ℓ p , (0 < p < 1) , quasi-norm objective problem. Numerical results highlight the performance of the proposed approach when compared to the $ \ell _{1} $ ℓ 1 minimisation in terms of the sparsity of the induced solution. [ABSTRACT FROM AUTHOR]

We consider high-dimensional Bayesian inverse problems with arbitrary likelihood and product-form Laplace prior for which we provide a certified approximation of the posterior in the Hellinger distance. The approximate posterior differs from the prior only in a small number of relevant coordinates that contribute the most to the update from the prior to the posterior. We propose and analyze a gradient-based diagnostic to identify these relevant coordinates. Although this diagnostic requires computing an expectation with respect to the posterior, we propose tractable methods for the classical case of a linear forward model with Gaussian likelihood. Our methods can be employed to estimate the diagnostic before solving the Bayesian inverse problem via, e.g., Markov chain Monte Carlo (MCMC) methods. After selecting the coordinates, the approximate posterior can be efficiently inferred since most of its coordinates are only informed by the prior. Moreover, specialized MCMC methods, such as the pseudo-marginal MCMC algorithm, can be used to obtain less correlated samples when sampling the exact posterior. We show the applicability of our method using a 1D signal deblurring problem and a high-dimensional 2D super-resolution problem. [ABSTRACT FROM AUTHOR]

As higher spatiotemporal resolution tactile sensing systems are being developed for prosthetics, wearables, and other biomedical applications, they demand faster sampling rates and generate larger data streams. Sparsifying transformations can alleviate these requirements by enabling compressive sampling and efficient data storage through compression. However, research on the best sparsifying transforms for tactile interactions is lagging. In this work we construct a library of orthogonal and biorthogonal wavelet transforms as sparsifying transforms for tactile interactions and compare their tradeoffs in compression and sparsity. We tested the sparsifying transforms on a publicly available high-density tactile object grasping dataset (548 sensor tactile glove, grasping 26 objects). In addition, we investigated which dimension wavelet transform—1D, 2D, or 3D—would best compress these tactile interactions. Our results show that wavelet transforms are highly efficient at compressing tactile data and can lead to very sparse and compact tactile representations. Additionally, our results show that 1D transforms achieve the sparsest representations, followed by 3D, and lastly 2D. Overall, the best wavelet for coarse approximation is Symlets 4 evaluated temporally which can sparsify to 0.5% sparsity and compress 10-bit tactile data to an average of 0.04 bits per pixel. Future studies can leverage the results of this paper to assist in the compressive sampling of large tactile arrays and free up computational resources for real-time processing on computationally constrained mobile platforms like neuroprosthetics. [ABSTRACT FROM AUTHOR]

Blind compressive sensing generates a reduced set of measurements of a signal without the knowledge of the sparsifying basis. In this context, the unique recovery and reconstruction of the underlying signal is a challenging problem. One of the properties of the Householder operator is to transform a non-sparse vector in a finite dimensional vector space to a sparse representation of the vector, preserving its ℓ 2 -norm. In this work, we present an algorithm that invokes the sparsifying property of the Householder transforms for the recovery and reconstruction of a signal from its reduced set of blind measurements. The algorithm is an iterative procedure using two objective functions. The first objective function is to estimate the sparse set of representing coefficients. The second objective function jointly estimates the unknown sparsifying transform and the unknown signal. Both the objective functions are entangled to ensure that the estimates meet the fidelity requirement with the available reduced set of measurements. An ℓ 1 -trend filtering is applied to minimize overfitting the estimate of the signal. The signal is uniquely recovered and reconstructed upto an acceptable lower error bound. [ABSTRACT FROM AUTHOR]

We propose a new method for separating valid inequalities for the epigraph of a function of binary variables. The proposed inequalities are disjunctive cuts defined by disjunctive terms obtained by enumerating a subset I of the binary variables. We show that by restricting the support of the cut to the same set of variables I, a cut can be obtained by solving a linear program with 2 | I | constraints. While this limits the size of the set I used to define the multi-term disjunction, the procedure enables generation of multi-term disjunctive cuts using far more terms than existing approaches. We present two approaches for choosing the subset of variables. Experience on three MILP problems with block diagonal structure using |I| up to size 10 indicates the sparse cuts can often close nearly as much gap as the multi-term disjunctive cuts without this restriction and in a fraction of the time. We also find that including these cuts within a cut-and-branch solution method for these MILP problems leads to significant reductions in solution time or ending optimality gap for instances that were not solved within the time limit. Finally, we describe how the proposed approach can be adapted to optimally "tilt" a given valid inequality by modifying the coefficients of a sparse subset of the variables. [ABSTRACT FROM AUTHOR]

Distributed inference tasks could be performed by adaptive filtering techniques. Several enhancement strategies for such techniques were proposed, such as sparsity-aware algorithms, coefficients reuse and correntropy-based cost functions in the case of impulsive noise. In this paper, a general framework based on Lagrange multipliers for the derivation of sophisticated algorithms that incorporate most of these improvements is described. A new general identification algorithm is derived as an example of the proposed approach and its performance is assessed in a distributed setting. [ABSTRACT FROM AUTHOR]

AbstractThe severity of multivariate extreme events is driven by the dependence between the largest marginal observations. The Hüsler–Reiss distribution is a versatile model for this extremal dependence, and it is usually parameterized by a variogram matrix. In order to represent conditional independence relations and obtain sparse parameterizations, we introduce the novel Hüsler–Reiss precision matrix. Similarly to the Gaussian case, this matrix appears naturally in density representations of the Hüsler–Reiss Pareto distribution and encodes the extremal graphical structure through its zero pattern. For a given, arbitrary graph we prove the existence and uniqueness of the completion of a partially specified Hüsler–Reiss variogram matrix so that its precision matrix has zeros on non-edges in the graph. Using suitable estimators for the parameters on the edges, our theory provides the first consistent estimator of graph structured Hüsler–Reiss distributions. If the graph is unknown, our method can be combined with recent structure learning algorithms to jointly infer the graph and the corresponding parameter matrix. Based on our methodology, we propose new tools for statistical inference of sparse Hüsler–Reiss models and illustrate them on large flight delay data in the United States, as well as Danube river flow data. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work. [ABSTRACT FROM AUTHOR]

AbstractIndependent component analysis (ICA) is widely used to estimate spatial resting-state networks and their time courses in neuroimaging studies. It is thought that independent components correspond to sparse patterns of co-activating brain locations. Previous approaches for introducing sparsity to ICA replace the non-smooth objective function with smooth approximations, resulting in components that do not achieve exact zeros. We propose a novel Sparse ICA method that enables sparse estimation of independent source components by solving a non-smooth non-convex optimization problem via the relax-and-split framework. The proposed Sparse ICA method balances statistical independence and sparsity simultaneously and is computationally fast. In simulations, we demonstrate improved estimation accuracy of both source signals and signal time courses compared to existing approaches. We apply our Sparse ICA to cortical surface resting-state fMRI in school-aged autistic children. Our analysis reveals differences in brain activity between certain regions in autistic children compared to children without autism. Sparse ICA selects coactivating locations, which we argue is more interpretable than dense components from popular approaches. Sparse ICA is fast and easy to apply to big data. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work. [ABSTRACT FROM AUTHOR]

Plant breeders want to develop cultivars that outperform existing genotypes. Some characteristics (here 'main traits') of these cultivars are categorical and difficult to measure directly. It is important to predict the main trait of newly developed genotypes accurately. In addition to marker data, breeding programs often have information on secondary traits (or 'phenotypes') that are easy to measure. Our goal is to improve prediction of main traits with interpretable relations by combining the two data types using variable selection techniques. However, the genomic characteristics can overwhelm the set of secondary traits, so a standard technique may fail to select any phenotypic variables. We develop a new statistical technique that ensures appropriate representation from both the secondary traits and the genotypic variables for optimal prediction. When two data types (markers and secondary traits) are available, we achieve improved prediction of a binary trait by two steps that are designed to ensure that a significant intrinsic effect of a phenotype is incorporated in the relation before accounting for extra effects of genotypes. First, we sparsely regress the secondary traits on the markers and replace the secondary traits by their residuals to obtain the effects of phenotypic variables as adjusted by the genotypic variables. Then, we develop a sparse logistic classifier using the markers and residuals so that the adjusted phenotypes may be selected first to avoid being overwhelmed by the genotypic variables due to their numerical advantage. This classifier uses forward selection aided by a penalty term and can be computed effectively by a technique called the one-pass method. It compares favorably with other classifiers on simulated and real data. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new technique for handling a class of sparse supervised learning problems, specifically focusing on regression, binary, and multi-class classification problems. Sparse regression problems are associated with datasets including both arithmetic and categorical features and by encoding the dataset leads to an underdetermined sparse linear system. Furthermore, logistic regression can be used for both sparse binary and multi-class classification problems, solving an underdetermined sparse linear system. A new technique called Sparse Approximate Pseudoinverse Preconditioning (SAPP), namely the Explicit Preconditioned Conjugate Gradient for Normal Equations (EPCGNE) method based on Generic Approximate Sparse Pseudoinverse matrices is introduced for solving underdetermined sparse least square problems. Numerical experiments were carried out demonstrating a significant improvement of the performance metrics for the proposed SAPP scheme compared to other learners. [ABSTRACT FROM AUTHOR]

We analyse a solution method for minimization problems over a space of $ \mathbb {R}^d $ R d -valued functions of bounded variation on an interval I. The presented method relies on piecewise constant iterates. In each iteration, the algorithm alternates between proposing a new point at which the iterate is allowed to be discontinuous and optimizing the magnitude of its jumps as well as the offset. A sublinear $ \mathcal {O}(1/k) $ O (1 / k) convergence rate for the objective function values is obtained in general settings. Under additional structural assumptions on the dual variable, this can be improved to a locally linear rate of convergence $ \mathcal {O}(\zeta ^k) $ O (ζ k) for some $ \zeta \lt 1 $ ζ < 1. Moreover, in this case, the same rate can be expected for the iterates in $ L^1(I;\mathbb {R}^d) $ L 1 (I ; R d). [ABSTRACT FROM AUTHOR]

For sequencing-based spatial transcriptomics data, the gene-spot count matrix is highly sparse. This feature is similar to scRNA-seq. The goal of this paper is to identify whether there exist genes that are frequently under-detected in Visium compared to bulk RNA-seq, and the underlying potential mechanism of under-detection in Visium. We collected paired Visium and bulk RNA-seq data for 28 human samples and 19 mouse samples, which covered diverse tissue sources. We compared the two data types and observed that there indeed exists a collection of genes frequently under-detected in Visium compared to bulk RNA-seq. We performed a motif search to examine the last 350 bp of the frequently under-detected genes, and we observed that the poly (T) motif was significantly enriched in genes identified from both human and mouse data, which matches with our previous finding about frequently under-detected genes in scRNA-seq. We hypothesized that the poly (T) motif may be able to form a hairpin structure with the poly (A) tails of their mRNA transcripts, making it difficult for their mRNA transcripts to be captured during Visium library preparation. [ABSTRACT FROM AUTHOR]

Anomaly detection is a binary classification task, which is to determine whether each pixel is abnormal or not. The difficulties are that it is hard to obtain abnormal samples and predict the shape of abnormal regions. Due to these difficulties, traditional supervised segmentation methods fail. The usual weakly supervised segmentation methods will use artificially generate defects to construct training samples. However, the model will be overfitted to artificially generated defects during training, resulting in insufficient generalization ability of the model. In this paper, we presented a novel reconstruction-based weakly supervised method for sparse anomaly detection. We proposed to use generative adversarial networks (GAN) to learn the distribution of positive samples, and reconstructed negative samples which contained the sparse defect into positive ones. Due to the nature of GAN, the training dataset only needs to contain normal samples. Subsequently, the segmentation network performs progressive feature fusion on reconstructed and original samples to complete the anomaly detection. Specifically, we designed the loss function based on kullback-leibler divergence for sparse anomalous defects. The final weakly-supervised segmentation network only assumes a sparsity prior of the defect region; thus, it can circumvent the detailed semantic labels and alleviate the potential overfitting problem. We compared our method with the state of the art generation-based generative anomaly detection methods and observed the average area under the receiver operating characteristic curve increase of 3% on MVTec anomaly detection. [ABSTRACT FROM AUTHOR]

Differential network analysis plays a crucial role in capturing nuanced changes in conditional correlations between two samples. Under the high‐dimensional setting, the differential network, that is, the difference between the two precision matrices are usually stylized with sparse signals and some low‐rank latent factors. Recognizing the distinctions inherent in the precision matrices of such networks, we introduce a novel approach, termed 'SR‐Network' for the estimation of sparse and reduced‐rank differential networks. This method directly assesses the differential network by formulating a convex empirical loss function with ℓ1$$ {\ell}_1 $$‐norm and nuclear norm penalties. The study establishes finite‐sample error bounds for parameter estimation and highlights the superior performance of the proposed method through extensive simulations and real data studies. This research significantly contributes to the advancement of methodologies for accurate analysis of differential networks, particularly in the context of structures characterized by sparsity and low‐rank features. [ABSTRACT FROM AUTHOR]

Unsupervised defect detection methods are applied to an unlabeled dataset by producing a ranked list based on defect scores. Unfortunately, many of the top-ranked instances by unsupervised algorithms are not defects, which leads to high false-positive rates. Active Defect Discovery (ADD) is proposed to overcome this deficiency, which sequentially selects instances to get the labeling information (defects or not). However, labeling is often costly. Therefore, balancing detection accuracy and labeling cost is essential. Along this line, this article proposes a novel ADD method to achieve the goal. Our approach is based on the state-of-the-art unsupervised defect detection method, namely, Isolation Forest, as the baseline defect detector to extract features. Thereafter, the sparsity of the extracted features is utilized to adjust the defect detector so that it can focus on more important features for defect detection. To enforce the sparsity of the features and subsequent improvement of the detection accuracy, a new algorithm based on online gradient descent, namely, Sparse Approximated Linear Defect Discovery (SALDD), is proposed with its theoretical Regret analysis. Extensive experiments are conducted on real-world datasets including healthcare, manufacturing, security, etc. The performance demonstrates that the proposed algorithm significantly outperforms the state-of-the-art algorithms for defect detection. [ABSTRACT FROM AUTHOR]

We introduce and study two new inferential challenges associated with the sequential detection of change in a high-dimensional mean vector. First, we seek a confidence interval for the changepoint, and second, we estimate the set of indices of coordinates in which the mean changes. We propose an online algorithm that produces an interval with guaranteed nominal coverage, and whose length is, with high probability, of the same order as the average detection delay, up to a logarithmic factor. The corresponding support estimate enjoys control of both false negatives and false positives. Simulations confirm the effectiveness of our methodology, and we also illustrate its applicability on the U.S. excess deaths data from 2017 to 2020. The , which contains the proofs of our theoretical results, is available online. [ABSTRACT FROM AUTHOR]

Data collection and the availability of large data sets has increased over the last decades. In both statistical and machine learning frameworks, two methodological issues typically arise when performing regression analysis on large data sets. First, variable selection is crucial in regression modeling, as it helps to identify an appropriate model with respect to the considered set of conditioning variables. Second, especially in the context of survey data, handling of missing values is important for estimation, which occur even with state-of-the-art data collection and processing methods. Within this paper, we provide an Bayesian approach based on a spike-and-slab prior for the regression coefficients, which allows for simultaneous handling of variable selection and estimation in combination with handling of missing values in covariate data. The paper also discusses the implementation of the approach using Markov chain Monte Carlo techniques and provides results for simulated data sets and an empirical illustration based on data from the German National Educational Panel Study. The suggested Bayesian approach is compared to other statistical and machine learning frameworks such as Lasso, ridge regression, and Elastic net, and is shown to perform well in terms of estimation performance and variable selection accuracy. The simulation results demonstrate that ignoring the handling of missing values in data sets can lead to the generation of biased selection results. Overall, the proposed Bayesian method offers a holistic, flexible, and powerful framework for variable selection in the presence of missing covariate data. [ABSTRACT FROM AUTHOR]

Collaborative filtering-based recommendation systems have become significant in various domains due to their ability to provide personalised recommendations. In e-commerce, these systems analyse the browsing history and purchase patterns of users to recommend items. In the entertainment industry, collaborative filtering helps platforms like Netflix and Spotify recommend movies, shows and songs based on users' past preferences and ratings. This technology also finds significance in online education, where it assists in suggesting relevant courses and learning materials based on a user's interests and previous learning behaviour. Even though much research has been done in this domain, the problems of sparsity and scalability in collaborative filtering still exist. Data sparsity refers to too few preferences of users on items, and hence it would be difficult to understand users' preferences. Recommendation systems must keep users engaged with fast responses, and hence there is a challenge in handling large data as these days it is growing quickly. Sparsity affects the recommendation accuracy, while scalability influences the complexity of processing the recommendations. The motivation behind the paper is to design an efficient algorithm to address the sparsity and scalability problems, which in turn provide a better user experience and increased user satisfaction. This paper proposes two separate, novel approaches that deal with both problems. In the first approach, an improved autoencoder is used to address sparsity, and later, its outcome is processed in a parallel and distributed manner using a MapReduce-based k -means clustering algorithm with the Elbow method. Since the k -means clustering technique uses a predetermined number of clusters, it may not improve accuracy. So, the elbow method identifies the optimal number of clusters for the k -means algorithm. In the second approach, a MapReduce-based Gaussian Mixture Model (GMM) with Expectation-Maximization (EM) is proposed to handle large volumes of sparse data. Both the proposed algorithms are implemented using MovieLens 20M and Netflix movie recommendation datasets to generate movie recommendations and are compared with the other state-of-the-art approaches. For comparison, metrics like RMSE, MAE, precision, recall, and F-score are used. The outcomes demonstrate that the second proposed strategy outperformed state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Neural networks are becoming increasingly popular in applications, but our mathematical understanding of their potential and limitations is still limited. In this paper, we further this understanding by developing statistical guarantees for sparse deep learning. In contrast to previous work, we consider different types of sparsity, such as few active connections, few active nodes, and other norm-based types of sparsity. Moreover, our theories cover important aspects that previous theories have neglected, such as multiple outputs, regularization, and ℓ 2 -loss. The guarantees have a mild dependence on network widths and depths, which means that they support the application of sparse but wide and deep networks from a statistical perspective. Some of the concepts and tools that we use in our derivations are uncommon in deep learning and, hence, might be of additional interest. [ABSTRACT FROM AUTHOR]