Search

Accurate self-motion estimation is critical for various navigational tasks in mobile robotics. Optic flow provides a means to estimate self-motion using a camera sensor and is particularly valuable in GPS- and radio-denied environments. The present study investigates the influence of different activation functions—ReLU, leaky ReLU, GELU, and Mish—on the accuracy, robustness, and encoding properties of convolutional neural networks (CNNs) and multi-layer perceptrons (MLPs) trained to estimate self-motion from optic flow. Our results demonstrate that networks with ReLU and leaky ReLU activation functions not only achieved superior accuracy in self-motion estimation from novel optic flow patterns but also exhibited greater robustness under challenging conditions. The advantages offered by ReLU and leaky ReLU may stem from their ability to induce sparser representations than GELU and Mish do. Our work characterizes the encoding of optic flow in neural networks and highlights how the sparseness induced by ReLU may enhance robust and accurate self-motion estimation from optic flow. [ABSTRACT FROM AUTHOR]

Learning tasks are often based on penalized optimization problems in which a sparse solution is desired. This can lead to more interpretative results by identifying a smaller subset of important features or components and reducing the dimensionality of the data representation, as well. In this study, we propose a new method to solve a constrained Frobenius norm-based nonnegative low-rank approximation, and the tuning of the associated penalty hyperparameter, simultaneously. The penalty term added is a particular diversity measure that is more effective for sparseness purposes than other classical norm-based penalties (i.e., ℓ 1 or ℓ 2 , 1 norms). As it is well known, setting the hyperparameters of an algorithm is not an easy task. Our work drew on developing an optimization method and the corresponding algorithm that simultaneously solves the sparsity-constrained nonnegative approximation problem and optimizes its associated penalty hyperparameters. We test the proposed method by numerical experiments and show its promising results on several synthetic and real datasets. [ABSTRACT FROM AUTHOR]

In this paper, the quadratic stability of continuous-time uncertain large-scale networked systems is considered. The systems are composed of many subsystems distributed in different locations in space and interconnected according to a certain network topology. A computationally efficient sufficient condition for the quadratic stability of large-scale networked systems is established, utilising effectively the block diagonal structure of the system parameter matrix and the sparseness of the subsystem connection matrix (SCM). Furthermore, a stability condition is presented, which is entirely contingent on the parameters of each subsystem. The simulation results show that the obtained conditions have obvious advantages in analysing the quadratic stability of large-scale networked systems. [ABSTRACT FROM AUTHOR]

Temporal text data, such as news articles or Twitter feeds, often comprises a mixture of long-lasting trends and transient topics. Effective topic modeling strategies should detect both types and clearly locate them in time. We first demonstrate that nonnegative CANDECOMP/PARAFAC decomposition (NCPD) can automatically identify topics of variable persistence. We then introduce sparseness-constrained NCPD (S-NCPD) and its online variant to control the duration of the detected topics more effectively and efficiently, along with theoretical analysis of the proposed algorithms. Through an extensive study on both semi-synthetic and real-world datasets, we find that our S-NCPD and its online variant can identify both short- and long-lasting temporal topics in a quantifiable and controlled manner, which traditional topic modeling methods are unable to achieve. Additionally, the online variant of S-NCPD shows a faster reduction in reconstruction error and results in more coherent topics compared to S-NCPD, thus achieving both computational efficiency and quality of the resulting topics. Our findings indicate that S-NCPD and its online variant are effective tools for detecting and controlling the duration of topics in temporal text data, providing valuable insights into both persistent and transient trends. [ABSTRACT FROM AUTHOR]

Proportional data arise frequently in a wide variety of fields of study. Such data often exhibit extra variation such as over/under dispersion, sparseness and zero inflation. For example, the hepatitis data present both sparseness and zero inflation with 19 contributing non-zero denominators of 5 or less and with 36 having zero seropositive out of 83 annual age groups. The whitefly data consists of 640 observations with 339 zeros (53%), which demonstrates extra zero inflation. The catheter management data involve excessive zeros with over 60% zeros averagely for outcomes of 193 urinary tract infections, 194 outcomes of catheter blockages and 193 outcomes of catheter displacements. However, the existing models cannot always address such features appropriately. In this paper, a new two-parameter probability distribution called Lindley–binomial (LB) distribution is proposed to analyze the proportional data with such features. The probabilistic properties of the distribution such as moment, moment generating function are derived. The Fisher scoring algorithm and EM algorithm are presented for the computation of estimates of parameters in the proposed LB regression model. The issues on goodness of fit for the LB model are discussed. A limited simulation study is also performed to evaluate the performance of derived EM algorithms for the estimation of parameters in the model with/without covariates. The proposed model is illustrated through three aforementioned proportional datasets. [ABSTRACT FROM AUTHOR]

Least Squares Support Vector Regression (LSSVR) which is a least squares version of the Support Vector Regression (SVR) is defined with a regularized squared loss without epsilon-insensitiveness. LSSVR is formulated in the dual space as a linear equality constrained quadratic minimization which can be transformed into solution of a linear algebraic equation system. As a consequence of this system where the number of Lagrange multipliers is half that of classical SVR, LSSVR has much less time consumption compared to the classical SVR. Despite this computationally attractive feature, it lacks the sparsity characteristic of SVR due to epsilon-insensitiveness. In LSSVR, every (training) input data is treated as a support vector, yielding extremely poor generalization performance. To overcome these drawbacks, the epsilon-insensitive LSSVR with epsilon-insensitivity at quadratic loss, in which sparsity is directly controlled by the epsilon parameter, is derived in this paper. Since the quadratic loss is sensitive to outliers, its weighted version (epsilon insensitive WLSSVR) has also been developed. Finally, the performances of epsilon-insensitive LSSVR and epsilon-insensitive WLSSVR are quantitatively compared in detail with those commonly used in the literature, pruning-based LSSVR and weighted pruning-based LSSVR. Experimental results on simulated and 8 different real-life data show that epsilon-insensitive LSSVR and epsilon-insensitive WLSSVR are superior in terms of computation time, generalization ability, and sparsity. [ABSTRACT FROM AUTHOR]

Temporal text data, such as news articles or Twitter feeds, often comprises a mixture of long-lasting trends and transient topics. Effective topic modeling strategies should detect both types and clearly locate them in time. We first demonstrate that nonnegative CANDECOMP/PARAFAC decomposition (NCPD) can automatically identify topics of variable persistence. We then introduce sparseness-constrained NCPD (S-NCPD) and its online variant to control the duration of the detected topics more effectively and efficiently, along with theoretical analysis of the proposed algorithms. Through an extensive study on both semi-synthetic and real-world datasets, we find that our S-NCPD and its online variant can identify both short- and long-lasting temporal topics in a quantifiable and controlled manner, which traditional topic modeling methods are unable to achieve. Additionally, the online variant of S-NCPD shows a faster reduction in reconstruction error and results in more coherent topics compared to S-NCPD, thus achieving both computational efficiency and quality of the resulting topics. Our findings indicate that S-NCPD and its online variant are effective tools for detecting and controlling the duration of topics in temporal text data, providing valuable insights into both persistent and transient trends.

The robust stability problem of spatially interconnected systems with signal saturation among the many composed subsystems is considered. The system structure is usually sparse, and each subsystem has different dynamics. Firstly, a robust stability condition is established based on integral quadratic constraint (IQC) theory, which makes full use of the sparseness of the subsystem connection topology. Secondly, a decoupling robust condition that only depends on the subsystem parameters is proved. Finally, it is shown through numerical simulations that the obtained conditions are computationally valid in analyzing spatially interconnected systems with signal saturation constraints among the subsystems.

Social media platforms such as (Twitter, Facebook, and Weibo) are being increasingly embraced by individuals, groups, and organizations as a valuable source of information. This social media generated information comes in the form of tweets or posts, and normally characterized as short text, huge, sparse, and low density. Since many real-world applications need semantic interpretation of such short texts, research in Short Text Topic Modeling (STTM) has recently gained a lot of interest to reveal unique and cohesive latent topics. This article examines the current state of the art in STTM algorithms. It presents a comprehensive survey and taxonomy of STTM algorithms for short text topic modelling. The article also includes a qualitative and quantitative study of the STTM algorithms, as well as analyses of the various strengths and drawbacks of STTM techniques. Moreover, a comparative analysis of the topic quality and performance of representative STTM models is presented. The performance evaluation is conducted on two real-world Twitter datasets: the Real-World Pandemic Twitter (RW-Pand-Twitter) dataset and Real-world Cyberbullying Twitter (RW-CB-Twitter) dataset in terms of several metrics such as topic coherence, purity, NMI, and accuracy. Finally, the open challenges and future research directions in this promising field are discussed to highlight the trends of research in STTM. The work presented in this paper is useful for researchers interested in learning state-of-the-art short text topic modelling and researchers focusing on developing new algorithms for short text topic modelling. [ABSTRACT FROM AUTHOR]

Both biological and artificial neural networks inherently balance their performance with their operational cost, which characterizes their computational abilities. Typically, an efficient neuromorphic neural network is one that learns representations that reduce the redundancies and dimensionality of its input. For instance, in the case of sparse coding (SC), sparse representations derived from natural images yield representations that are heterogeneous, both in their sampling of input features and in the variance of those features. Here, we focused on this notion, and sought correlations between natural images’ structure, particularly oriented features, and their corresponding sparse codes. We show that representations of input features scattered across multiple levels of variance substantially improve the sparseness and resilience of sparse codes, at the cost of reconstruction performance. This echoes the structure of the model’s input, allowing to account for the heterogeneously aleatoric structures of natural images. We demonstrate that learning kernel from natural images produces heterogeneity by balancing between approximate and dense representations, which improves all reconstruction metrics. Using a parametrized control of the kernels’ heterogeneity of a convolutional SC algorithm, we show that heterogeneity emphasizes sparseness, while homogeneity improves representation granularity. In a broader context, this encoding strategy can serve as inputs to deep convolutional neural networks. We prove that such variance-encoded sparse image datasets enhance computational efficiency, emphasizing the benefits of kernel heterogeneity to leverage naturalistic and variant input structures and possible applications to improve the throughput of neuromorphic hardware.

This paper investigates the dissipativity analysis of large-scale networked systems with linear time-invariant dynamics. The networked system is composed of a large number of subsystems whose connections are arbitrary, and each subsystem can have different dynamics. A sufficient and necessary condition for the strict dissipativity analysis of the networked system is derived, which takes advantage of the block-diagonal structure of the system parameter matrix and the sparseness characteristics of the subsystem interconnections. Then, a necessary condition and a sufficient condition that depend only on a single subsystem parameter are given separately. Numerical simulations illustrate that compared with the existing results, the conditions suggested in this paper have higher computational efficiency in the dissipative analysis of large-scale networked systems. [ABSTRACT FROM AUTHOR]

This paper presents a novel random vector functional link (RVFL) formulation called the 1-norm RVFL (1N RVFL) networks, for solving the binary classification problems. The solution to the optimization problem of 1N RVFL is obtained by solving its exterior dual penalty problem using a Newton technique. The 1-norm makes the model robust and delivers sparse outputs, which is the fundamental advantage of this model. The sparse output indicates that most of the elements in the output matrix are zero; hence, the decision function can be achieved by incorporating lesser hidden nodes compared to the conventional RVFL model. 1N RVFL produces a classifier that is based on a smaller number of input features. To put it another way, this method will suppress the neurons in the hidden layer. Statistical analyses have been carried out on several real-world benchmark datasets. The proposed 1N RVFL with two activation functions viz., ReLU and sine are used in this work. The classification accuracies of 1N RVFL are compared with the extreme learning machine (ELM), kernel ridge regression (KRR), RVFL, kernel RVFL (K-RVFL) and generalized Lagrangian twin RVFL (GLTRVFL) networks. The experimental results with comparable or better accuracy indicate the effectiveness and usability of 1N RVFL for solving binary classification problems. [ABSTRACT FROM AUTHOR]

In this paper, we address the challenge of low recognition rates in existing methods for radar signals from unmanned aerial vehicles (UAV) with low signal-to-noise ratios (SNRs). To overcome this challenge, we propose the utilization of the bispectral slice approach for accurate recognition of complex UAV radar signals. Our approach involves extracting the bispectral diagonal slice and the maximum bispectral amplitude horizontal slice from the bispectrum amplitude spectrum of the received UAV radar signal. These slices serve as the basis for subsequent identification by calculating characteristic parameters such as convexity, box dimension, and sparseness. To accomplish the recognition task, we employ a GA-BP neural network. The significant variations observed in the bispectral slices of different signals, along with their robustness against Gaussian noise, contribute to the high separability and stability of the extracted bispectral convexity, bispectral box dimension, and bispectral sparseness. Through simulations involving five radar signals, our proposed method demonstrates superior performance. Remarkably, even under challenging conditions with an SNR as low as −3 dB, the recognition accuracy for the five different radar signals exceeds 90%. Our research aims to enhance the understanding and application of modulation recognition techniques for UAV radar signals, particularly in scenarios with low SNRs.

An issue that exists during the meta-analysis process is that one or maybe more studies may have sparse data, such as no events in the treatment and control groups. Estimating Odds Ratio, one of the three association measures, will be challenging for such a dataset. In certain cases, it is standard procedure to either add a constant or remove the study to estimate the unknown values. This technique, however, is relied on the asymptotic characteristics of the estimates and may underperform for sample sizes. Another strategy would be to employ Bayesian approaches to have a better understanding of the problem. The main motive of this work is to perform a comparative study in analysing the robustness of highly imbalanced datasets with zero events for various values of K studies (0 < k = 50) in the dataset between the Binomial-Normal and Normal-Normal models.A Bayesian method with an appropriate prior might be a feasible option for dealing with sparse data, according to the findings of a comparative research. Both methodologies performed better in all sparse datasets and are suggested to be used in meta-analyses of similar scope. [ABSTRACT FROM AUTHOR]

In active sonar systems, detection always suffers from reverberation interference from multiple scatterers in oceanic environments; therefore, numerous studies have been conducted on reverberation suppression. Recently, a non-negative matrix factorization (NMF)-based method was proposed and successfully applied to reverberation suppression. However, the conventional NMF-based method makes convergence challenging because the frequency basis matrix is initialized without considering reverberation characteristic information from oceanic environments. To solve these problems, We propose an improved NMF-based reverberation suppression method adopting a pre-trained reverberation basis matrix and modified sparse update rule. The proposed method is evaluated by analyzing simulation and sea experiment data and the study confirmed that the detection performance was improved compared to the conventional method under various signal-to-reverberation ratio conditions. Several topics are also discussed to analyze the proposed method in detail.

Earthquake early warning is an effective method to reduce casualties and losses. Based on the theory of compressive sensing, this paper proposes a strong earthquake signal processing architecture based on compressive sensing for difficulties of the acquisition, transmission and storage of massive strong earthquake data that the large-scale earthquake warning systems faced. Based on this, the observation vector of strong earthquake signal can be obtained with a predetermined compression ratio and used for transmission and storage. When necessary, the observation value is reconstructed to restore the original strong earthquake signal with a high probability. The paper analyzes the selection of sparse basis and observation matrix, discusses the reconstruction algorithms, base pursuit (BP) and orthogonal matching pursuit (OMP), and verifies the feasibility of the whole framework from compressed sampling to data reconstruction through experiments. The proposed framework has brought great convenience to the sampling, transmission, storage, and processing of the earthquake early warning system. It is foreseeable that replacing the traditional Nyquist sampling value with the observation values in the compressive sensing theory will cause a fundamental change in the signal characteristics, which will further affect the theory and technical system of the entire earthquake early warning system.

Depending on the scanning mode, existing short text stream clustering methods can be divided into the following two kinds of methods: one-pass-based and batch-based. The one-pass-based method handles each text only one time, but cannot deal with the sparseness problem very well. The batch-based method obtains better results by allowing multiple iterations of each batch, but the efficiency is relatively low. To overcome these problems, this paper presents Lifelong learning Augmented Short Text stream clustering method (LAST), which incorporates the episodic memory module and sparse experience replay module of lifelong learning into the clustering process. Specifically, LAST processes each text one time, but at a certain interval it randomly samples some previously seen texts of the episodic memory to update cluster features by performing sparse experience replay. Empirical studies on two public datasets demonstrate that the performance of the LAST-based method is on a par with the batch-based method, and runs close to the speed of the one-pass-based method.

Analyzing short texts infers discriminative and coherent latent topics that is a critical and fundamental task since many real-world applications require semantic understanding of short texts. Traditional long text topic modeling algorithms (e.g., PLSA and LDA) based on word co-occurrences cannot solve this problem very well since only very limited word co-occurrence information is available in short texts. Therefore, short text topic modeling has already attracted much attention from the machine learning research community in recent years, which aims at overcoming the problem of sparseness in short texts. In this survey, we conduct a comprehensive review of various short text topic modeling techniques proposed in the literature. We present three categories of methods based on Dirichlet multinomial mixture, global word co-occurrences, and self-aggregation, with example of representative approaches in each category and analysis of their performance on various tasks. We develop the first comprehensive open-source library, called STTM, for use in Java that integrates all surveyed algorithms within a unified interface, benchmark datasets, to facilitate the expansion of new methods in this research field. Finally, we evaluate these state-of-the-art methods on many real-world datasets and compare their performance against one another and versus long text topic modeling algorithm. [ABSTRACT FROM AUTHOR]

Feature analysis is a fundamental research area in computer graphics; meanwhile, meaningful and part-aware feature bases are always demanding. This paper proposes a framework for conducting feature analysis on a three-dimensional (3D) model by introducing modified Non-negative Matrix Factorization (NMF) model into the graphical feature space and push forward further applications. By analyzing and utilizing the intrinsic ideas behind NMF, we propose conducting the factorization on feature matrices constructed based on descriptors or graphs, which provides a simple but effective way to raise compressed and scale-aware descriptors. In order to enable part-aware model analysis, we modify the NMF model to be sparse and constrained regarding to both bases and encodings, which gives rise to Sparse and Constrained Non-negative Matrix Factorization (SAC-NMF). Subsequently, by adapting the analytical components (including hidden variables, bases, and encodings) to design descriptors, several applications have been easily but effectively realized. The extensive experimental results demonstrate that the proposed framework has many attractive advantages, such as being efficient, extendable, and so forth.

This paper investigates the dissipativity analysis of large-scale networked systems with linear time-invariant dynamics. The networked system is composed of a large number of subsystems whose connections are arbitrary, and each subsystem can have different dynamics. A sufficient and necessary condition for the strict dissipativity analysis of the networked system is derived, which takes advantage of the block-diagonal structure of the system parameter matrix and the sparseness characteristics of the subsystem interconnections. Then, a necessary condition and a sufficient condition that depend only on a single subsystem parameter are given separately. Numerical simulations illustrate that compared with the existing results, the conditions suggested in this paper have higher computational efficiency in the dissipative analysis of large-scale networked systems.

Reinforcement learning as autonomous learning is greatly driving artificial intelligence (AI) development to practical applications. Having demonstrated the potential to significantly improve synchronously parallel learning, the parallel computing based asynchronous advantage actor-critic (A3C) opens a new door for reinforcement learning. Unfortunately, the acceleration's inuence on A3C robustness has been largely overlooked. In this paper, we perform the first robustness assessment of A3C based on parallel computing. By perceiving the policy's action, we construct a global matrix of action probability deviation and define two novel measures of skewness and sparseness to form an integral robustness measure. Based on such static assessment, we then develop a dynamic robustness assessing algorithm through situational whole-space state sampling of changing episodes. Extensive experiments with different combinations of agent number and learning rate are implemented on an A3C-based pathfinding application, demonstrating that our proposed robustness assessment can effectively measure the robustness of A3C, which can achieve an accuracy of 83.3%. [ABSTRACT FROM AUTHOR]

Efficiently solving a system identification problem represents an important step in numerous important applications. In this framework, some of the most popular solutions rely on the Wiener filter, which is widely used in practice. Moreover, it also represents a benchmark for other related optimization problems. In this paper, new insights into the regularization of the Wiener filter are provided, which is a must in real-world scenarios. A proper regularization technique is of great importance, especially in challenging conditions, e.g., when operating in noisy environments and/or when only a low quantity of data is available for the estimation of the statistics. Different regularization methods are investigated in this paper, including several new solutions that fit very well for the identification of sparse and low-rank systems. Experimental results support the theoretical developments and indicate the efficiency of the proposed techniques. [ABSTRACT FROM AUTHOR]

Anomaly target detection has been a hotspot of the hyperspectral imagery (HSI) processing in recent decades. One of the key research points in the HSI anomaly detection is the accurate descriptions of the background and anomaly targets. Considering this point, we propose a novel anomaly target detector in this article. Improving upon the low-rank and sparse matrix decomposition (LRaSMD) approach, the proposed method assumes that the low-rank component can be described as the parts-based representation. Parts refer to the various ground objects in HSI. A new update rule of the low-rank component and sparse component is proposed. The proposed approach can be divided into three main steps: first, further refining the low-rank component in the LRaSMD model as the parts-based representation. Then, the HSI is decomposed as three parts: the product of the basis matrix and coefficient matrix, sparse matrix, and noise. Second, the basis vectors matrix, coefficient matrix, and sparse matrix are solved by the new update rules. Third, since the anomaly targets exist in the sparse matrix, the sparse matrix is thus employed to detect the anomaly targets. The experiments implemented for five data sets demonstrate that the proposed algorithm achieved a better performance than the traditional algorithms.

[Introduction] The paper aims to establish a sparseness approach based sample distribution for LS-SVM models to solve the problem of excessive computation in the application of classical iterative shearing sparseness algorithm for the soft measurement model of the carbon content in flying ash. [Method] On the basis of calculating the feature space distance between the samples, global representative indicator is constructed by mixing together the density and dispersion. The original samples were sorted and pruned and the sparsenseness was realized according to the indicator. The LS-SVM soft measurement model of the carbon content in fly ash was applied to a 1 000 MW coal-fired power plant, the original training sample set was taken from the field operation data of the unit. [Result] The results show that the proposed algorithm can greatly reduce the capacity of the training set with tiny loss of the error performance and it can reduce the training and online prediction calculation work during the LS-SVM soft measurement model of the carbon content in fly ash. [Conclusion] The LS-SVM sparse algorithm proposed in this paper reduces the sample space from 90 to 30, which not only reduces the calculation scale, but also guarantees the calculation accuracy, while guaranteeing that the error is reduced by 0.01%. The algorithm can realize on-line soft measurement of carbon content in fly ash in industrial controllers with limited computing performance such as PLC, and can be extended to other parameters soft measurement systems in power plants.

Predictive coding provides a computational paradigm for modeling perceptual processing as the construction of representations accounting for causes of sensory inputs. Here, we developed a scalable, deep network architecture for predictive coding that is trained using a gated Hebbian learning rule and mimics the feedforward and feedback connectivity of the cortex. After training on image datasets, the models formed latent representations in higher areas that allowed reconstruction of the original images. We analyzed low- and high-level properties such as orientation selectivity, object selectivity and sparseness of neuronal populations in the model. As reported experimentally, image selectivity increased systematically across ascending areas in the model hierarchy. Depending on the strength of regularization factors, sparseness also increased from lower to higher areas. The results suggest a rationale as to why experimental results on sparseness across the cortical hierarchy have been inconsistent. Finally, representations for different object classes became more distinguishable from lower to higher areas. Thus, deep neural networks trained using a gated Hebbian formulation of predictive coding can reproduce several properties associated with neuronal responses along the visual cortical hierarchy.

Predictive coding provides a computational paradigm for modeling perceptual processing as the construction of representations accounting for causes of sensory inputs. Here, we developed a scalable, deep network architecture for predictive coding that is trained using a gated Hebbian learning rule and mimics the feedforward and feedback connectivity of the cortex. After training on image datasets, the models formed latent representations in higher areas that allowed reconstruction of the original images. We analyzed low- and high-level properties such as orientation selectivity, object selectivity and sparseness of neuronal populations in the model. As reported experimentally, image selectivity increased systematically across ascending areas in the model hierarchy. Depending on the strength of regularization factors, sparseness also increased from lower to higher areas. The results suggest a rationale as to why experimental results on sparseness across the cortical hierarchy have been inconsistent. Finally, representations for different object classes became more distinguishable from lower to higher areas. Thus, deep neural networks trained using a gated Hebbian formulation of predictive coding can reproduce several properties associated with neuronal responses along the visual cortical hierarchy. [ABSTRACT FROM AUTHOR]

Blind image separation is a method of recovering the original images from a set of image mixtures, with no information about the source images or about the mixing process. Blind source separation problem has been used to extract sources from one-dimensional mixture signals such as speech, whereas, application of source separation for images (two-dimensional signals) has been examined to a limited extent. The independent component analysis (ICA) method assumes statistical independence of the source signals and at least one of the source could be non-gaussian. These assumptions do not hold for image mixing conditions. An alternative approach is to use Kalman filter that operates on the noisy input data recursively to produce statistically optimal estimate of the underlying sources. In this paper, a robust filter on H-infinity norm is proposed and compared with for image separation of Kalman filter. The extension of the algorithms to a simple and block based approach wherein the image mixture is converted to a sparse representation is proposed. The image mixture is subdivided into blocks of known sizes and the sparseness of each block is measured using l_0 norm and the block with maximum sparseness is used for extraction of original sources. This reduces the computational complexity that exists with the large dimensions of images. The algorithms are validated for natural image data sets and also for window reflection images taken under different lighting conditions. The analysis results suggest that the proposed methods provide significantly good quality of separation as measured by the performance metrics. [ABSTRACT FROM AUTHOR]

The main focus of this paper is the separation of underdetermined convolutive blind speech in a multi-speaker environment. We present a method based on mask prediction in the time-frequency domain. Firstly, depending on the sparsity of signals in the time-frequency (TF) domain, we extimate speakers' masks by clustering the relative absolute and Hermitian angle features extracted from the frequency components of the mixtures. Speech separation algorithms that are based on the sparsity and disjoint orthogonality of the speech signals in the time-frequency domain are not efficient when more than one source is active. Hence, in this paper, the cluster centers are estimated mostly based on the TF units that probably have only one active source. The correlations between the estimated masks, belonging to adjacent frequency bins, are leveraged to solve the permutation problem. To increase the accuracy, we have zeroed the value of masks at the TF unit without any active source. Moreover, in clustering, we employ a weighting function to consider the parts of masks that probably contains just one active source. Finally, in order to decrease the musical noise of the separated signals and improve their quality, sparse filters in the time-domain are utilized to re-estimate the separated signals. Performance of the proposed method is evaluated by a number of simulated and real speech signals. The simulated experiments were performed using a public dataset and Roomsim simulator. Compared the proposed method with some conventional algorithms, we observed that our separation method is more accurate than other approaches. [ABSTRACT FROM AUTHOR]

When solving many regression problems, there exist a large number of input features. However, not all features are relevant for current regression, and sometimes, including irrelevant features may deteriorate the learning performance. Therefore, it is essential to select the most relevant features, especially for high‐dimensional regression. Feature selection is an effective way to solve this problem. It tries to represent original data by extracting relevant features that contain useful information. In this paper, aiming to effectively select useful features in least squares support vector regression (LSSVR), we propose a novel sparse LSSVR based on Lp‐norm (SLSSVR), 0

Finite-time stability (FTS) analysis and control problems are investigated in this paper for continuous-time networked dynamical systems constituted by many subsystems. Interactions among these subsystems are arbitrary and time-varying, and every subsystem has different and time-varying dynamics. Several differential linear matrix inequality (LMI) formed methods are presented for the FTS analysis and control, which avoid the inverse computation of matrices with large dimensions and efficiently utilize the block diagonal characteristic of system parameters and the sparseness of the subsystem connection matrix. Sufficient conditions only depending on the parameters of each individual subsystem are then derived. Furthermore, LMI-based conditions are also provided for the FTS analysis and control of linear time-invariant networked systems. Numerical simulations show that they are computationally effective to analyze and synthesize a large-scale networked dynamical systems. [ABSTRACT FROM AUTHOR]

The classical principal component analysis (PCA) is not sparse enough since it is based on the L2-norm that is also prone to be adversely affected by the presence of outliers and noises. In order to address the problem, a sparse robust PCA framework is proposed based on the min of zero-norm regularization and the max of Lp-norm (0 < p ≤ 2) PCA. Furthermore, we developed a continuous optimization method, DC (difference of convex functions) programming algorithm (DCA), to solve the proposed problem. The resulting algorithm (called DC-LpZSPCA) is convergent linearly. In addition, when choosing different p values, the model can keep robust and is applicable to different data types. Numerical simulations are simulated in artificial data sets and Yale face data sets. Experiment results show that the proposed method can maintain good sparsity and anti-outlier ability. [ABSTRACT FROM AUTHOR]

The data sparsity and cold start problems of the collaborative filtering recommendation algorithm affect the quality of the recommendation. Traditional user-item bipartite graph trust and the similarity calculation is negatively affected by the local individual point correlation factors. Firstly, this paper proposed a bipartite graph trust recommendation based on project popularity calculated by a grey correlational analysis by degree of balance and approach. On this basis, it proposed the user's preference enhanced bipartite graph direct trust degree mechanism, and used the JMSD correlation coefficient as the global trust recommendation. The comparison experiments under the MovieLens dataset show that the improved algorithm has a lower mean absolute error( MAE) compared with the benchmark algorithm, it improves the recommendation quality and alleviates the cold start problem. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Signal Processing is the property of Journal of Signal Processing and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Functional MRI (fMRI) is a valuable brain imaging technique. A significant problem, when analyzing fMRI time series, is the finding of functional brain networks when the brain is at rest, i.e. no external stimulus is applied to the subject. In this work, we present a probabilistic method to estimate the Resting State Networks (RSNs) using a model-based approach. More specifically, RSNs are assumed to be the result of a clustering procedure. In order to perform the clustering, a mixture of regression models are used. Furthermore, special care has been given in order to incorporate important functionalities, such as spatial and embedded sparsity enforcing properties, through the use of informative priors over the model parameters. Another interesting feature of the proposed scheme is the flexibility to handle all the brain time series at once, allowing more robust solutions. We provide comparative experimental results, using an artificial fMRI dataset and two real resting state fMRI datasets, that empirically illustrate the efficiency of the proposed regression mixture model. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel method for Multispectral document image binarization (MSdB) through the Non-negative Matrix Factorization (NMF) approach. We propose a three-step MSdB-NMF framework: i) NMF-based feature extraction algorithm by introducing a new optimization problem; ii) post-processing method iii); apply any existing gray/RGB binarization scheme. In the first step, we extract $N$ features out of $B$ spectral bands ($N < B$) and their corresponding coefficient matrix. We introduce a novel objective formulation that considers the robustness (related to the noise and various types of degradations) and sparseness (related to the ratio of text pixels versus the background). We employ the multiplicative updating rules to solve the proposed minimization problem and prove the convergence of the proposed feature extraction algorithm. In the next step, we select an appropriate feature vector, equivalently the corresponding coefficient vector. We propose to select it either visually or automatically via a post-processing method, which uses the benchmark binarization methods as baseline. In the last step, we apply some existing binarization methods such as Sauvola and Howe over the selected coefficient vector. Our proposed binarization framework is applicable for any kind of MS or hyperspectral (HS) document image without considering any prior knowledge such as the side information about the spectral bands of MS/HS document image. We evaluate our proposed binarization framework over two MS document image datasets. The experimental results confirm that our proposed framework outperforms several state-of-the-art binarization schemes including the winner of the contest in MS-TEx-2015. [ABSTRACT FROM AUTHOR]

To improve the sparseness of the base matrix in incremental non-negative matrix factorization, we in this paper present a new method, orthogonal incremental non-negative matrix factorization algorithm (OINMF), which combines the orthogonality constraint with incremental learning. OINMF adopts batch update in the process of incremental learning, and its iterative formulae are obtained using the gradient on the Stiefel manifold. The experiments on image classification show that the proposed method achieves much better sparseness and orthogonality, while retaining time efficiency of incremental learning. [ABSTRACT FROM AUTHOR]

Nonsmooth nonnegative matrix factorization (nsNMF) is capable of producing more localized, less overlapped feature representations than other variants of NMF while keeping satisfactory fit to data. However, nsNMF as well as other existing NMF methods are incompetent to learn hierarchical features of complex data due to its shallow structure. To fill this gap, we propose a deep nsNMF method coined by the fact that it possesses a deeper architecture compared with standard nsNMF. The deep nsNMF not only gives part-based features due to the nonnegativity constraints but also creates higher level, more abstract features by combing lower level ones. The in-depth description of how deep architecture can help to efficiently discover abstract features in dnsNMF is presented, suggesting that the proposed model inherits the major advantages from both deep learning and NMF. Extensive experiments demonstrate the standout performance of the proposed method in clustering analysis.

Efficiently solving a system identification problem represents an important step in numerous important applications. In this framework, some of the most popular solutions rely on the Wiener filter, which is widely used in practice. Moreover, it also represents a benchmark for other related optimization problems. In this paper, new insights into the regularization of the Wiener filter are provided, which is a must in real-world scenarios. A proper regularization technique is of great importance, especially in challenging conditions, e.g., when operating in noisy environments and/or when only a low quantity of data is available for the estimation of the statistics. Different regularization methods are investigated in this paper, including several new solutions that fit very well for the identification of sparse and low-rank systems. Experimental results support the theoretical developments and indicate the efficiency of the proposed techniques.

Artificial neural networks (ANNs) are important building blocks in technical applications. They rely on noiseless continuous signals in stark contrast to the discrete action potentials stochastically exchanged among the neurons in real brains. We propose to bridge this gap with Spike-by-Spike (SbS) networks which represent a compromise between non-spiking and spiking versions of generative models. What is missing, however, are algorithms for finding weight sets that would optimize the output performances of deep SbS networks with many layers. Here, a learning rule for feed-forward SbS networks is derived. The properties of this approach are investigated and its functionality is demonstrated by simulations. In particular, a Deep Convolutional SbS network for classifying handwritten digits achieves a classification performance of roughly 99.3% on the MNIST test data when the learning rule is applied together with an optimizer. Thereby it approaches the benchmark results of ANNs without extensive parameter optimization. We envision this learning rule for SBS networks to provide a new basis for research in neuroscience and for technical applications, especially when they become implemented on specialized computational hardware.

In this paper, we present a novel muscle synergy extraction method based on multivariate curve resolution–alternating least squares (MCR-ALS) to overcome the limitation of the nonnegative matrix factorization (NMF) method for extracting non-sparse muscle synergy, and we study its potential application for evaluating motor function of stroke survivors. Nonnegative matrix factorization (NMF) is the most widely used method for muscle synergy extraction. However, NMF is susceptible to components’ sparseness and usually provides inferior reliability, which significantly limits the promotion of muscle synergy. In this study, MCR-ALS was employed to extract muscle synergy from electromyography (EMG) data. Its performance was compared with two other matrix factorization algorithms, NMF and self-modeling mixture analysis (SMMA). Simulated data sets were utilized to explore the influences of the sparseness and noise on the extracted synergies. As a result, the synergies estimated by MCR-ALS were the most similar to true synergies as compared with SMMA and NMF. MCR-ALS was used to analyze the muscle synergy characteristics of upper limb movements performed by healthy (n = 11) and stroke (n = 5) subjects. The repeatability and intra-subject consistency were used to evaluate the performance of MCR-ALS. As a result, MCR-ALS provided much higher repeatability and intra-subject consistency as compared with NMF, which were important for the reliability of the motor function evaluation. The stroke subjects had lower intra-subject consistency and seemingly had more synergies as compared with the healthy subjects. Thus, MCR-ALS is a promising muscle synergy analysis method for motor function evaluation of stroke patients.

Conditional risk measuring plays an important role in financial regulation and depends on volatility estimation. A new class of parameter models called Generalized Autoregressive Score (GAS) model has been successfully applied for different error's densities and for different problems of time series prediction in particular for volatility modeling and VaR estimation. To improve the estimating accuracy of the GAS model, this study proposed a semi-parametric method, LS-SVR and FS-LS-SVR applied to the GAS model to estimate the conditional VaR. In particular, we fit the GAS(1,1) model to the return series using three different distributions. Then, LS-SVR and FS-LS-SVR approximate the GAS(1,1) model. An empirical research was performed to illustrate the effectiveness of the proposed method. More precisely, the experimental results from four stock indexes returns suggest that using hybrid models, GAS-LS-SVR and GAS-FS-LS-SVR provides improved performances in the VaR estimation. [ABSTRACT FROM AUTHOR]