Search

This paper investigates the dissipativity analysis of large-scale networked systems with linear time-invariant dynamics. The networked system is composed of a large number of subsystems whose connections are arbitrary, and each subsystem can have different dynamics. A sufficient and necessary condition for the strict dissipativity analysis of the networked system is derived, which takes advantage of the block-diagonal structure of the system parameter matrix and the sparseness characteristics of the subsystem interconnections. Then, a necessary condition and a sufficient condition that depend only on a single subsystem parameter are given separately. Numerical simulations illustrate that compared with the existing results, the conditions suggested in this paper have higher computational efficiency in the dissipative analysis of large-scale networked systems.

We extracted useful features from Mobile Spatial Data. Specifically, we applied a Non-negative Matrix Factorization (NMF), which is a method of decomposing the original data into two additive matrices, to extract features of daytime and nighttime demography. By applying Sparse Non-negative Matrix Factorization to the same data, we extracted features more clearly than NMF. Furthermore, we comprehensively analyzed the structure of the original data by examining the approximate residuals of each matrix decomposition model for the original data. We used the data on the day and night population of Hakui residents in Ishikawa Prefecture for 1461 days (March 1, 2014 to February 28, 2018) by age and sex (15－19, 20－29, 30－39, 40－49, 50－59, 60－69 and 70－79 years old×man and woman) . As a result of the analysis, when we set basis number as “3”, we obtained the characteristics that were interpreted as the “teenage male”, “teenage female” and “worker”. In addition, using Sparse NMF, many other attributes were “0” in the “teenage male” and “teenage female”, thus allowing for a clear interpretation. To summarize the characteristics of Hakui citizens, we found that there is an outflow of people who commute to work and school on weekdays, and outflow for those aged between 20 and 39 on holidays. In addition, the day and night population during the Obon holidays and New Year's holidays are very small, so it can be assumed that many citizens remain in Hakui during the same period.

Fecal coliform bacteria are commonly used as an indicator to reflect the fecal contamination level in the water. To manage a healthy and thriving aquaculture industry, the Canadian Shellfish Sanitation Program (CSSP) was established in 1948. As a part of the CSSP mandate, fecal coliform bacteria levels in shellfish growing habitat have been monitored at nearly 15, 000 shellfish harvesting sites across the six coastal provinces of Canada over 40 years (1980–2019). The irregular sparseness in the measurement data presented a critical challenge for reliable analysis of fecal contamination patterns along Canada's coastline. This paper illustrates a preprocessing approach to handle the irregular sparseness in the measurement data of fecal coliform bacteria levels and demonstrates the effectiveness of data filtering, pooling, binning, partitioning, and the application of functional principal component analysis. We managed to transform the irregularly sparse measurements at a site into surrogate variables without missing data that represent the differences in the site's contamination amplitude and seasonal variation from the average. The surrogate variables will be used for downstream analyses, such as associating contamination with climate change., 2022 IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB), August 15-17, 2022, Ottawa, ON, Canada

Earthquake early warning is an effective method to reduce casualties and losses. Based on the theory of compressive sensing, this paper proposes a strong earthquake signal processing architecture based on compressive sensing for difficulties of the acquisition, transmission and storage of massive strong earthquake data that the large-scale earthquake warning systems faced. Based on this, the observation vector of strong earthquake signal can be obtained with a predetermined compression ratio and used for transmission and storage. When necessary, the observation value is reconstructed to restore the original strong earthquake signal with a high probability. The paper analyzes the selection of sparse basis and observation matrix, discusses the reconstruction algorithms, base pursuit (BP) and orthogonal matching pursuit (OMP), and verifies the feasibility of the whole framework from compressed sampling to data reconstruction through experiments. The proposed framework has brought great convenience to the sampling, transmission, storage, and processing of the earthquake early warning system. It is foreseeable that replacing the traditional Nyquist sampling value with the observation values in the compressive sensing theory will cause a fundamental change in the signal characteristics, which will further affect the theory and technical system of the entire earthquake early warning system.

Depending on the scanning mode, existing short text stream clustering methods can be divided into the following two kinds of methods: one-pass-based and batch-based. The one-pass-based method handles each text only one time, but cannot deal with the sparseness problem very well. The batch-based method obtains better results by allowing multiple iterations of each batch, but the efficiency is relatively low. To overcome these problems, this paper presents Lifelong learning Augmented Short Text stream clustering method (LAST), which incorporates the episodic memory module and sparse experience replay module of lifelong learning into the clustering process. Specifically, LAST processes each text one time, but at a certain interval it randomly samples some previously seen texts of the episodic memory to update cluster features by performing sparse experience replay. Empirical studies on two public datasets demonstrate that the performance of the LAST-based method is on a par with the batch-based method, and runs close to the speed of the one-pass-based method.

In active sonar systems, detection always suffers from reverberation interference from multiple scatterers in oceanic environments; therefore, numerous studies have been conducted on reverberation suppression. Recently, a non-negative matrix factorization (NMF)-based method was proposed and successfully applied to reverberation suppression. However, the conventional NMF-based method makes convergence challenging because the frequency basis matrix is initialized without considering reverberation characteristic information from oceanic environments. To solve these problems, We propose an improved NMF-based reverberation suppression method adopting a pre-trained reverberation basis matrix and modified sparse update rule. The proposed method is evaluated by analyzing simulation and sea experiment data and the study confirmed that the detection performance was improved compared to the conventional method under various signal-to-reverberation ratio conditions. Several topics are also discussed to analyze the proposed method in detail.

Feature analysis is a fundamental research area in computer graphics, meanwhile, meaningful and part-aware feature bases are always demanding. This paper proposes a framework for conducting feature analysis on a three-dimensional (3D) model by introducing modified Non-negative Matrix Factorization (NMF) model into the graphical feature space and push forward further applications. By analyzing and utilizing the intrinsic ideas behind NMF, we propose conducting the factorization on feature matrices constructed based on descriptors or graphs, which provides a simple but effective way to raise compressed and scale-aware descriptors. In order to enable part-aware model analysis, we modify the NMF model to be sparse and constrained regarding to both bases and encodings, which gives rise to Sparse and Constrained Non-negative Matrix Factorization (SAC-NMF). Subsequently, by adapting the analytical components (including hidden variables, bases, and encodings) to design descriptors, several applications have been easily but effectively realized. The extensive experimental results demonstrate that the proposed framework has many attractive advantages, such as being efficient, extendable, and so forth.

Anomaly target detection has been a hotspot of the hyperspectral imagery (HSI) processing in recent decades. One of the key research points in the HSI anomaly detection is the accurate descriptions of the background and anomaly targets. Considering this point, we propose a novel anomaly target detector in this article. Improving upon the low-rank and sparse matrix decomposition (LRaSMD) approach, the proposed method assumes that the low-rank component can be described as the parts-based representation. Parts refer to the various ground objects in HSI. A new update rule of the low-rank component and sparse component is proposed. The proposed approach can be divided into three main steps: first, further refining the low-rank component in the LRaSMD model as the parts-based representation. Then, the HSI is decomposed as three parts: the product of the basis matrix and coefficient matrix, sparse matrix, and noise. Second, the basis vectors matrix, coefficient matrix, and sparse matrix are solved by the new update rules. Third, since the anomaly targets exist in the sparse matrix, the sparse matrix is thus employed to detect the anomaly targets. The experiments implemented for five data sets demonstrate that the proposed algorithm achieved a better performance than the traditional algorithms.

[Introduction] The paper aims to establish a sparseness approach based sample distribution for LS-SVM models to solve the problem of excessive computation in the application of classical iterative shearing sparseness algorithm for the soft measurement model of the carbon content in flying ash. [Method] On the basis of calculating the feature space distance between the samples, global representative indicator is constructed by mixing together the density and dispersion. The original samples were sorted and pruned and the sparsenseness was realized according to the indicator. The LS-SVM soft measurement model of the carbon content in fly ash was applied to a 1 000 MW coal-fired power plant, the original training sample set was taken from the field operation data of the unit. [Result] The results show that the proposed algorithm can greatly reduce the capacity of the training set with tiny loss of the error performance and it can reduce the training and online prediction calculation work during the LS-SVM soft measurement model of the carbon content in fly ash. [Conclusion] The LS-SVM sparse algorithm proposed in this paper reduces the sample space from 90 to 30, which not only reduces the calculation scale, but also guarantees the calculation accuracy, while guaranteeing that the error is reduced by 0.01%. The algorithm can realize on-line soft measurement of carbon content in fly ash in industrial controllers with limited computing performance such as PLC, and can be extended to other parameters soft measurement systems in power plants.

Efficiently solving a system identification problem represents an important step in numerous important applications. In this framework, some of the most popular solutions rely on the Wiener filter, which is widely used in practice. Moreover, it also represents a benchmark for other related optimization problems. In this paper, new insights into the regularization of the Wiener filter are provided, which is a must in real-world scenarios. A proper regularization technique is of great importance, especially in challenging conditions, e.g., when operating in noisy environments and/or when only a low quantity of data is available for the estimation of the statistics. Different regularization methods are investigated in this paper, including several new solutions that fit very well for the identification of sparse and low-rank systems. Experimental results support the theoretical developments and indicate the efficiency of the proposed techniques.

Predictive coding provides a computational paradigm for modeling perceptual processing as the construction of representations accounting for causes of sensory inputs. Here, we developed a scalable, deep network architecture for predictive coding that is trained using a gated Hebbian learning rule and mimics the feedforward and feedback connectivity of the cortex. After training on image datasets, the models formed latent representations in higher areas that allowed reconstruction of the original images. We analyzed low- and high-level properties such as orientation selectivity, object selectivity and sparseness of neuronal populations in the model. As reported experimentally, image selectivity increased systematically across ascending areas in the model hierarchy. Depending on the strength of regularization factors, sparseness also increased from lower to higher areas. The results suggest a rationale as to why experimental results on sparseness across the cortical hierarchy have been inconsistent. Finally, representations for different object classes became more distinguishable from lower to higher areas. Thus, deep neural networks trained using a gated Hebbian formulation of predictive coding can reproduce several properties associated with neuronal responses along the visual cortical hierarchy.

In this paper, we present a novel muscle synergy extraction method based on multivariate curve resolution–alternating least squares (MCR-ALS) to overcome the limitation of the nonnegative matrix factorization (NMF) method for extracting non-sparse muscle synergy, and we study its potential application for evaluating motor function of stroke survivors. Nonnegative matrix factorization (NMF) is the most widely used method for muscle synergy extraction. However, NMF is susceptible to components’ sparseness and usually provides inferior reliability, which significantly limits the promotion of muscle synergy. In this study, MCR-ALS was employed to extract muscle synergy from electromyography (EMG) data. Its performance was compared with two other matrix factorization algorithms, NMF and self-modeling mixture analysis (SMMA). Simulated data sets were utilized to explore the influences of the sparseness and noise on the extracted synergies. As a result, the synergies estimated by MCR-ALS were the most similar to true synergies as compared with SMMA and NMF. MCR-ALS was used to analyze the muscle synergy characteristics of upper limb movements performed by healthy (n = 11) and stroke (n = 5) subjects. The repeatability and intra-subject consistency were used to evaluate the performance of MCR-ALS. As a result, MCR-ALS provided much higher repeatability and intra-subject consistency as compared with NMF, which were important for the reliability of the motor function evaluation. The stroke subjects had lower intra-subject consistency and seemingly had more synergies as compared with the healthy subjects. Thus, MCR-ALS is a promising muscle synergy analysis method for motor function evaluation of stroke patients.

Significance Three-dimensional refinement is a critical component of cryo-EM single-particle reconstruction. In this paper, we report the development of a computational method, OPUS-SSRI, and its application to seven real cryo-EM datasets. Our data clearly demonstrated that OPUS-SSRI can improve the final resolutions and structural details in cryo-EM single-particle analysis., In this paper, we present a refinement method for cryo-electron microscopy (cryo-EM) single-particle reconstruction, termed as OPUS-SSRI (Sparseness and Smoothness Regularized Imaging). In OPUS-SSRI, spatially varying sparseness and smoothness priors are incorporated to improve the regularity of electron density map, and a type of real space penalty function is designed. Moreover, we define the back-projection step as a local kernel regression and propose a first-order method to solve the resulting optimization problem. On the seven cryo-EM datasets that we tested, the average improvement in resolution by OPUS-SSRI over that from RELION 3.0, the commonly used image-processing software for single-particle cryo-EM, was 0.64 Å, with the largest improvement being 1.25 Å. We expect OPUS-SSRI to be an invaluable tool to the broad field of cryo-EM single-particle analysis. The implementation of OPUS-SSRI can be found at https://github.com/alncat/cryoem.

In the absence of perfect reference standard, classical techniques result in biased diagnostic accuracy and prevalence estimates. By statistically defining the true disease status, latent class models (LCM) constitute a promising alternative. However, LCM is a complex method which relies on parametric assumptions, including usually a conditional independence between tests and might suffer from data sparseness. We carefully applied LCMs to assess new campylobacter infection detection tests for which bacteriological culture is an imperfect reference standard. Five diagnostic tests (culture, polymerase chain reaction and three immunoenzymatic tests) of campylobacter infection were collected in 623 patients from Bordeaux and Lyon Hospitals, France. Their diagnostic accuracy were estimated with standard and extended LCMs with a thorough examination of models goodness-of-fit. The model including a residual dependence specific to the immunoenzymatic tests best complied with LCM assumptions. Asymptotic results of goodness-of-fit statistics were substantially impaired by data sparseness and empirical distributions were preferred. Results confirmed moderate sensitivity of the culture and high performances of immunoenzymatic tests. LCMs can be used to estimate diagnostic tests accuracy in the absence of perfect reference standard. However, their implementation and assessment require specific attention due to data sparseness and limitations of existing software.

It is well-known that the class of sets that can be computed by polynomial size circuits is equal to the class of sets that are polynomial time reducible to a sparse set. It is widely believed, but unfortunately up to now unproven, that there are sets in (Formula presented.), or even in (Formula presented.) that are not computable by polynomial size circuits and hence are not reducible to a sparse set. In this paper we study this question in a more restricted setting: what is the computational complexity of sparse sets that are selfreducible? It follows from earlier work of Lozano and Toran (in: Mathematical systems theory, 1991) that (Formula presented.) does not have sparse selfreducible hard sets. We define a natural version of selfreduction, tree-selfreducibility, and show that (Formula presented.) does not have sparse tree-selfreducible hard sets. We also construct an oracle relative to which all of (Formula presented.) is reducible to a sparse tree-selfreducible set. These lower bounds are corollaries of more general results about the computational complexity of sparse sets that are selfreducible, and can be interpreted as super-polynomial circuit lower bounds for (Formula presented.).

Signal interactions in brain network communication have been little studied. We describe how nonlinear collision rules on simulated mammal brain networks can result in sparse activity dynamics characteristic of mammalian neural systems. We tested the effects of collisions in “information spreading” (IS) routing models and in standard random walk (RW) routing models. Simulations employed synchronous agents on tracer-based mesoscale mammal connectomes at a range of signal loads. We find that RW models have high average activity that increases with load. Activity in RW models is also densely distributed over nodes: a substantial fraction is highly active in a given time window, and this fraction increases with load. Surprisingly, while IS models make many more attempts to pass signals, they show lower net activity due to collisions compared to RW, and activity in IS increases little as function of load. Activity in IS also shows greater sparseness than RW, and sparseness decreases slowly with load. Results hold on two networks of the monkey cortex and one of the mouse whole-brain. We also find evidence that activity is lower and more sparse for empirical networks compared to degree-matched randomized networks under IS, suggesting that brain network topology supports IS-like routing strategies., Author Summary How do mammal brains control the communication of signals across their entire network? A fundamental goal for any large-scale communication system is managing signal interactions. Yet because brain dynamics are nonlinear and emergent, signal interactions on brain networks have been little studied. Here we investigate two forms of nonlinear signal interaction on the mammal mesoscale connectome: collisions and duplication (redundancy). Using explicit numerical simulations on high-accuracy connectomes of the mouse and monkey, we find evidence that mammal brain networks operate efficiently under a routing strategy that employs destructive collisions and the redundant spread of information. In contrast, standard random walk strategies with the same collision rule are less efficient in terms of global activity and sparseness of activity. Comparisons to randomized networks support these findings.

The formation of structure in the visual system, that is, of the connections between cellswithin neural populations, is by and large an unsupervised learning process. In the primary visualcortex of mammals, for example, one can observe during development the formation of cells selectiveto localized, oriented features, which results in the development of a representation in area V1 ofimages&rsquo, edges. This can be modeled using a sparse Hebbian learning algorithms which alternatea coding step to encode the information with a learning step to find the proper encoder. A majordifficulty of such algorithms is the joint problem of finding a good representation while knowingimmature encoders, and to learn good encoders with a nonoptimal representation. To solve thisproblem, this work introduces a new regulation process between learning and coding which ismotivated by the homeostasis processes observed in biology. Such an optimal homeostasis ruleis implemented by including an adaptation mechanism based on nonlinear functions that balancethe antagonistic processes that occur at the coding and learning time scales. It is compatible witha neuromimetic architecture and allows for a more efficient emergence of localized filters sensitiveto orientation. In addition, this homeostasis rule is simplified by implementing a simple heuristicon the probability of activation of neurons. Compared to the optimal homeostasis rule, numericalsimulations show that this heuristic allows to implement a faster unsupervised learning algorithmwhile retaining much of its effectiveness. These results demonstrate the potential application of sucha strategy in machine learning and this is illustrated by showing the effect of homeostasis in theemergence of edge-like filters for a convolutional neural network.

International audience; The formation of structure in the visual system, that is, of the connections between cells within neural populations, is by and large an unsupervised learning process. In the primary visual cortex of mammals, for example, one can observe during development the formation of cells selective to localized, oriented features, which results in the development of a representation in area V1 of images' edges. This can be modeled using a sparse Hebbian learning algorithms which alternate a coding step to encode the information with a learning step to find the proper encoder. A major difficulty of such algorithms is the joint problem of finding a good representation while knowing immature encoders, and to learn good encoders with a nonoptimal representation. To solve this problem, this work introduces a new regulation process between learning and coding which is motivated by the homeostasis processes observed in biology. Such an optimal homeostasis rule is implemented by including an adaptation mechanism based on nonlinear functions that balance the antagonistic processes that occur at the coding and learning time scales. It is compatible with a neuromimetic architecture and allows for a more efficient emergence of localized filters sensitive to orientation. In addition, this homeostasis rule is simplified by implementing a simple heuristic on the probability of activation of neurons. Compared to the optimal homeostasis rule, numerical simulations show that this heuristic allows to implement a faster unsupervised learning algorithm while retaining much of its effectiveness. These results demonstrate the potential application of such a strategy in machine learning and this is illustrated by showing the effect of homeostasis in the emergence of edge-like filters for a convolutional neural network.

In neuroscience, the structure of a circuit has often been used to intuit function—an inversion of Louis Kahn’s famous dictum, “Form follows function” (Kristan and Katz, 2006). However, different brain networks may use different network architectures to solve the same problem. The olfactory circuits of two insects, the locust,Schistocerca americana,and the fruit fly,Drosophila melanogaster, serve the same function—to identify and discriminate odors. The neural circuitry that achieves this shows marked structural differences. Projection neurons (PNs) in the antennal lobe innervate Kenyon cells (KCs) of the mushroom body. In locust, each KC receives inputs from ∼50% of PNs, a scheme that maximizes the difference between inputs to any two of ∼50,000 KCs. In contrast, inDrosophila, this number is only 5% and appears suboptimal. Using a computational model of the olfactory system, we show that the activity of KCs is sufficiently high-dimensional that it can separate similar odors regardless of the divergence of PN–KC connections. However, when temporal patterning encodes odor attributes, dense connectivity outperforms sparse connections. Increased separability comes at the cost of reliability. The disadvantage of sparse connectivity can be mitigated by incorporating other aspects of circuit architecture seen inDrosophila. Our simulations predict thatDrosophilaand locust circuits lie at different ends of a continuum where theDrosophilagives up on the ability to resolve similar odors to generalize across varying environments, while the locust separates odor representations but risks misclassifying noisy variants of the same odor.

1 Abstract Neural networks are important building blocks in artificial intelligence applications. These technical systems rely on noiseless continuous signals in stark contrast to the discrete action potentials exchanged among the neurons in real brains. A promising approach towards bridging this gap are the Spike-by-Spike (SbS) networks, which were demonstrated to reliably process information based on few stochastic spikes given suitable synaptic weights are provided. What is missing are algorithms for finding weight sets that would optimize the output performance of acyclically connected SbS networks. Here, a learning rule for hierarchically organized SbS networks is derived. It is inspired by the error back-propagation algorithms used in feed-forward neural networks. The properties of this approach are investigated and its functionality demonstrated by simulations. In particular, a Deep Convolutional SbS network for classifying handwritten digits (MNIST) is presented. The resulting classification performance reaches values achieved by standard deep networks that have a similar structure and also use only simple gradient descent. We expect that this new SbS back-prop learning rule will provide a basis for building spiking neuronal network models for research in neuroscience and technical applications, especially when it will become combined with state-of-the-art optimization techniques and implemented on specialized computational hardware.

Humans have flexible control over cognitive functions depending on the context. Several studies suggest that the prefrontal cortex (PFC) controls this cognitive flexibility, but the detailed underlying mechanisms remain unclear. Recent developments in machine learning techniques allow simple PFC models written as a recurrent neural network to perform various behavioral tasks like humans and animals. Computational modeling allows the estimation of neuronal parameters that are crucial for performing the tasks, which cannot be observed by biologic experiments. To identify salient neural-network features for flexible cognition tasks, we compared four PFC models using a context-dependent integration task. After training the neural networks with the task, we observed highly plastic synapses localized to a small neuronal population in all models. In three of the models, the neuronal units containing these highly plastic synapses contributed most to the performance. No common tendencies were observed in the distribution of synaptic strengths among the four models. These results suggest that task-dependent plastic synaptic changes are more important for accomplishing flexible cognitive tasks than the structures of the constructed synaptic networks.

High-level neurons processing complex, behaviorally relevant signals are sensitive to conjunctions of features. Characterizing the receptive fields of such neurons is difficult with standard statistical tools, however, and the principles governing their organization remain poorly understood. Here, we demonstrate multiple distinct receptive-field features in individual high-level auditory neurons in a songbird, European starling, in response to natural vocal signals (songs). We then show that receptive fields with similar characteristics can be reproduced by an unsupervised neural network trained to represent starling songs with a single learning rule that enforces sparseness and divisive normalization. We conclude that central auditory neurons have composite receptive fields that can arise through a combination of sparseness and normalization in neural circuits. Our results, along with descriptions of random, discontinuous receptive fields in the central olfactory neurons in mammals and insects, suggest general principles of neural computation across sensory systems and animal classes.

Nonsmooth Nonnegative Matrix Factorization (nsNMF) is capable of producing more localized, less overlapped feature representations than other variants of NMF while keeping satisfactory fit to data. However, nsNMF as well as other existing NMF methods is incompetent to learn hierarchical features of complex data due to its shallow structure. To fill this gap, we propose a deep nsNMF method coined by the fact that it possesses a deeper architecture compared with standard nsNMF. The deep nsNMF not only gives parts-based features due to the nonnegativity constraints, but also creates higher-level, more abstract features by combing lower-level ones. The in-depth description of how deep architecture can help to efficiently discover abstract features in dnsNMF is presented. And we also show that the deep nsNMF has close relationship with the deep autoencoder, suggesting that the proposed model inherits the major advantages from both deep learning and NMF. Extensive experiments demonstrate the standout performance of the proposed method in clustering analysis.

Brain decoding with multivariate classification and regression has provided a powerful framework for characterizing information encoded in population neural activity. Classification and regression models are respectively used to predict discrete and continuous variables of interest. However, cognitive and behavioral parameters that we wish to decode are often ordinal variables whose values are discrete but ordered, such as subjective ratings. To date, there is no established method of predicting ordinal variables in brain decoding. In this study, we present a new algorithm, sparse ordinal logistic regression (SOLR), that combines ordinal logistic regression with Bayesian sparse weight estimation. We found that, in both simulation and analyses using real functional magnetic resonance imaging data, SOLR outperformed ordinal logistic regression with non-sparse regularization, indicating that sparseness leads to better decoding performance. SOLR also outperformed classification and linear regression models with the same type of sparseness, indicating the advantage of the modeling tailored to ordinal outputs. Our results suggest that SOLR provides a principled and effective method of decoding ordinal variables.

Hyperspectral unmixing aims to estimate a set of endmembers and corresponding abundances in pixels. Nonnegative matrix factorization (NMF) and its extensions with various constraints have been widely applied to hyperspectral unmixing. L 1 / 2 and L 2 regularizers can be added to NMF to enforce sparseness and evenness, respectively. In practice, a region in a hyperspectral image may possess different sparsity levels across locations. The problem remains as to how to impose constraints accordingly when the level of sparsity varies. We propose a novel nonnegative matrix factorization with data-guided constraints (DGC-NMF). The DGC-NMF imposes on the unknown abundance vector of each pixel with either an L 1 / 2 constraint or an L 2 constraint according to its estimated mixture level. Experiments on the synthetic data and real hyperspectral data validate the proposed algorithm.

The recent "deep learning revolution" in artificial neural networks had strong impact and widespread deployment for engineering applications, but the use of deep learning for neurocomputational modeling has been so far limited. In this article we argue that unsupervised deep learning represents an important step forward for improving neurocomputational models of perception and cognition, because it emphasizes the role of generative learning as opposed to discriminative (supervised) learning. As a case study, we present a series of simulations investigating the emergence of neural coding of visual space for sensorimotor transformations. We compare different network architectures commonly used as building blocks for unsupervised deep learning by systematically testing the type of receptive fields and gain modulation developed by the hidden neurons. In particular, we compare Restricted Boltzmann Machines (RBMs), which are stochastic, generative networks with bidirectional connections trained using contrastive divergence, with autoencoders, which are deterministic networks trained using error backpropagation. For both learning architectures we also explore the role of sparse coding, which has been identified as a fundamental principle of neural computation. The unsupervised models are then compared with supervised, feed-forward networks that learn an explicit mapping between different spatial reference frames. Our simulations show that both architectural and learning constraints strongly influenced the emergent coding of visual space in terms of distribution of tuning functions at the level of single neurons. Unsupervised models, and particularly RBMs, were found to more closely adhere to neurophysiological data from single-cell recordings in the primate parietal cortex. These results provide new insights into how basic properties of artificial neural networks might be relevant for modeling neural information processing in biological systems.

Principal Components are usually hard to interpret. Sparseness is considered as one way to improve interpretability, and thus a trade-off between variance explained by the components and sparseness is frequently sought. In this note we address the problem of simultaneous maximization of variance explained and sparseness, and a heuristic method is proposed. It is shown that recent proposals in the literature may yield dominated solutions, in the sense that other components, found with our procedure, may exist which explain more variance and at the same time are sparser.

Principles of efficient coding suggest that the peripheral units of any sensory processing system are designed for efficient coding. The function of the lateral geniculate nucleus (LGN) as an early stage in the visual system is not well understood. Some findings indicate that similar to the retina that decorrelates input signals spatially, the LGN tends to perform a temporal decorrelation. There is evidence suggesting that corticogeniculate connections may account for this decorrelation in the LGN. In this study, we propose a computational model based on biological evidence reported by Wang et al. (2006), who demonstrated that the influence pattern of V1 feedback is phase-reversed. The output of our model shows how corticogeniculate connections decorrelate LGN responses and make an efficient representation. We evaluated our model using criteria that have previously been tested on LGN neurons through cell recording experiments, including sparseness, entropy, power spectra, and information transfer. We also considered the role of the LGN in higher-order visual object processing, comparing the categorization performance of human subjects with a cortical object recognition model in the presence and absence of our LGN input-stage model. Our results show that the new model that considers the role of the LGN, more closely follows the categorization performance of human subjects.

We propose a novel nonparallel classifier, called nonparallel support vector machine (NPSVM), for binary classification. Our NPSVM that is fully different from the existing nonparallel classifiers, such as the generalized eigenvalue proximal support vector machine (GEPSVM) and the twin support vector machine (TWSVM), has several incomparable advantages: 1) two primal problems are constructed implementing the structural risk minimization principle; 2) the dual problems of these two primal problems have the same advantages as that of the standard SVMs, so that the kernel trick can be applied directly, while existing TWSVMs have to construct another two primal problems for nonlinear cases based on the approximate kernel-generated surfaces, furthermore, their nonlinear problems cannot degenerate to the linear case even the linear kernel is used; 3) the dual problems have the same elegant formulation with that of standard SVMs and can certainly be solved efficiently by sequential minimization optimization algorithm, while existing GEPSVM or TWSVMs are not suitable for large scale problems; 4) it has the inherent sparseness as standard SVMs; 5) existing TWSVMs are only the special cases of the NPSVM when the parameters of which are appropriately chosen. Experimental results on lots of datasets show the effectiveness of our method in both sparseness and classification accuracy, and therefore, confirm the above conclusion further. In some sense, our NPSVM is a new starting point of nonparallel classifiers.

The use of sparseness in chemometrics is a concept that has increased in popularity. The advantage is, above all, a better interpretability of the results obtained. In this work, sparseness is implemented as a constraint in multivariate curve resolution - alternating least squares (MCR-ALS), which aims at reproducing raw (mixed) data by a bilinear model of chemically meaningful profiles. In many cases, the mixed raw data analyzed are not sparse by nature, but their decomposition profiles can be, as it is the case in some instrumental responses, such as mass spectra, or in concentration profiles linked to scattered distribution maps of powdered samples in hyperspectral images. To induce sparseness in the constrained profiles, one-dimensional and/or two-dimensional numerical arrays can be fitted using a basis of Gaussian functions with a penalty on the coefficients. In this work, a least squares regression framework with L0-norm penalty is applied. This L0-norm penalty constrains the number of non-null coefficients in the fit of the array constrained without having an a priori on the number and their positions. It has been shown that the sparseness constraint induces the suppression of values linked to uninformative channels and noise in MS spectra and improves the location of scattered compounds in distribution maps, resulting in a better interpretability of the constrained profiles. An additional benefit of the sparseness constraint is a lower ambiguity in the bilinear model, since the major presence of null coefficients in the constrained profiles also helps to limit the solutions for the profiles in the counterpart matrix of the MCR bilinear model. © 2017 Elsevier B.V., The authors thank Paul Eilers for his involvement in the development of the algorithms and appreciate the fruitful discussions on the topic. C. R. thanks the Agence National de la Recherche ( ANR-15-CE09-0020-01 Blink). C.B. and A.J have received funding from the European Research Council under the European Union's Seventh Framework Program ( FP/2007-2013 )/ ERC Grant Agreement n. 320737 (CHEMAGEB project). S.P. acknowledges the support of the Spanish government through project CTQ2015-66254-C2-2-P .

Natural images follow statistics inherited by the structure of our physical (visual) environment. In particular, a prominent facet of this structure is that images can be described by a relatively sparse number of features. To investigate the role of this sparseness in the efficiency of the neural code, we designed a new class of random textured stimuli with a controlled sparseness value inspired by measurements of natural images. Then, we tested the impact of this sparseness parameter on the firing pattern observed in a population of retinal ganglion cells recorded ex vivo in the retina of a rodent, the Octodon degus. These recordings showed in particular that the reliability of spike timings varies with respect to the sparseness with globally a similar trend than the distribution of sparseness statistics observed in natural images. These results suggest that the code represented in the spike pattern of ganglion cells may adapt to this aspect of the statistics of natural images., arXiv admin note: substantial text overlap with arXiv:1702.02485

In this paper, we propose a novel method for single-channel blind separation of nonoverlapped sources and, to the best of our knowledge, apply it for the first time to automatic segmentation of lung tumors in positron emission tomography (PET) images. Our approach first converts a 3-D PET image into a pseudo-multichannel image. Afterward, regularization free sparseness constrained non-negative matrix factorization is used to separate tumor from other tissues. By using complexity based criterion, we select tumor component as the one with minimal complexity. We have compared the proposed method with threshold based on 40% and 50% maximum standardized uptake value (SUV), graph cuts (GC), random walks (RW), and affinity propagation (AP) algorithms on 18 nonsmall cell lung cancer datasets with respect to ground truth (GT) provided by two radiologists. Dice similarity coefficient averaged with respect to two GTs is: 0.78 ± 0.12 by the proposed algorithm, 0.78 ± 0.1 by GC, 0.77 ± 0.13 by AP, 0.77 ± 0.07 by RW, and 0.75 ± 0.13 by 50% maximum SUV threshold. Since the proposed method achieved performance comparable with interactive methods, considering the unique challenges of lung tumor segmentation from PET images, our findings support possibility of using our fully automated method in routine clinics. The source codes will be available at www.mipav.net/English/research/research.html.

One of the most famous methods for recommendation is user-based Collaborative Filtering (CF). This system compares active user’s items rating with historical rating records of other users to find similar users and recommending items which seems interesting to these similar users and have not been rated by the active user. As a way of computing recommendations, the ultimate goal of the user-based collaborative filtering is recommending items with the high accuracy and coverage degree. Nevertheless, some famous limitations are obstacles to meet them. They are Scalability, Sparseness and new item problems. Scalability problem can be handled with the use of Data Mining techniques like clustering. However, use of this technique often leads to the lower recommendation accuracy. Nevertheless, two other problems still remain. Involving Semantic knowledge can increase the performance of recommendation in sparseness and New-Item Problem conditions as well. This paper presents a new approach to deal with the drawbacks of user-based CF systems for web pages recommendation by Combination of Semantic Knowledge with Web Usage Mining (WUM). Semantic knowledge of web pages are extracted and subsequently incorporated into the navigation patterns of each cluster which obtained from clustering the access sessions to get the Semantic Patterns of each cluster. The cluster with the most relevant semantic pattern is chosen with the comparison of semantic representation of the active user session with the semantic patterns and the proper web pages are recommended based on a switching recommendation engine. This engine recommends a list of appropriate recommendations. Results of the implementation of this hybrid web recommender system indicates that this combined approach yields better results in both accuracy and coverage metrics and also has a considerable capability to handle collaborative filtering recommender system for its typical shortcomings.

The nonlinear, nonnegative single-mixture blind source separation (BSS) problem consists of decomposing observed nonlinearly mixed multicomponent signal into nonnegative dependent component (source) signals. The problem is difficult and is a special case of the underdetermined BSS problem. However, it is practically relevant for the contemporary metabolic profiling of biological samples when only one sample is available for acquiring mass spectra ; afterwards, the pure components are extracted. Herein, we present a method for the blind separation of nonnegative dependent sources from a single, nonlinear mixture. First, an explicit feature map is used to map a single mixture into a pseudo multi-mixture. Second, an empirical kernel map is used for implicit mapping of a pseudo multi-mixture into a high-dimensional reproducible kernel Hilbert space (RKHS). Under sparse probabilistic conditions that were previously imposed on sources, the single-mixture nonlinear problem is converted into an equivalent linear, multiple-mixture problem that consists of the original sources and their higher order monomials. These monomials are suppressed by robust principal component analysis, hard-, soft- and trimmed thresholding. Sparseness constrained nonnegative matrix factorizations in RKHS yield sets of separated components. Afterwards, separated components are annotated with the pure components from the library using the maximal correlation criterion. The proposed method is depicted with a numerical example that is related to the extraction of 8 dependent components from 1 nonlinear mixture. The method is further demonstrated on 3 nonlinear chemical reactions of peptide synthesis in which 25, 19 and 28 dependent analytes are extracted from 1 nonlinear mixture mass spectra. The goal application of the proposed method is, in combination with other separation techniques, mass spectrometry-based non-targeted metabolic profiling, such as biomarker identification studies.

The dentate gyrus (DG) of mammals harbors neural stem cells that generate new dentate granule cells (DGCs) throughout life. Behavioral studies using the contextual fear discrimination paradigm have found that selectively augmenting or blocking adult hippocampal neurogenesis enhances or impairs discrimination under conditions of high, but not low, interference suggestive of a role in pattern separation. Although contextual discrimination engages population-based coding mechanisms underlying pattern separation such as global remapping in the DG and CA3, how adult hippocampal neurogenesis modulates pattern separation in the DG is poorly understood. Here, we propose a role for adult-born DGCs in re-activation coupled modulation of sparseness through feed-back inhibition to govern global remapping in the DG.

This paper proposes a novel method for MRI denoising that exploits both the sparseness and self-similarity properties of the MR images. The proposed method is a two-stage approach that first filters the noisy image using a non local PCA thresholding strategy by automatically estimating the local noise level present in the image and second uses this filtered image as a guide image within a rotationally invariant non-local means filter. The proposed method internally estimates the amount of local noise presents in the images that enables applying it automatically to images with spatially varying noise levels and also corrects the Rician noise induced bias locally. The proposed approach has been compared with related state-of-the-art methods showing competitive results in all the studied cases., We are grateful to Dr. Matteo Mangioni and Dr. Alessandro Foi for their help on running their BM4D method in our comparisons. We want also to thank Dr. Luis Marti-Bonmati and Dr. Angel Alberich-Bayarri from Quiron Hospital of Valencia for providing the real clinical data used in this paper. This study has been carried out with financial support from the French State, managed by the French National Research Agency (ANR) in the frame of the Investments for the future Programme IdEx Bordeaux (ANR-10-IDEX-03-02), Cluster of excellence CPU and TRAIL (HR-DTI ANR-10-LABX-57).

Many seismic data processing and imaging processes require densely and regularly sampled data, whereas the actual measurements are mostly irregularly and sparsely sampled. Therefore, seismic data reconstruction methods are utilised as a pre-processing step. Within the class of transformation-based reconstruction techniques, observed seismic data is decomposed into certain basis functions, such as plane waves, parabolas or curvelets. In the corresponding model space the aliasing noise is assumed to have different properties than the seismic signal and can be suppressed. However, in many cases subsurface information is available that cannot be used in these traditional reconstruction methods. Therefore, the double focal transformation was derived as a way to incorporate knowledge about the subsurface in the reconstruction algorithm. The basic principle of the double focal transformation is to focus seismic energy by a back-propagation of the seismic data at the source and receiver side to certain depth levels. As a result, the seismic data are represented by a limited number of samples in the focal domain in a localised area, whereas aliasing noise spreads out. By imposing a sparse solution in the focal domain, aliasing noise is suppressed and data reconstruction beyond aliasing is achieved. To facilitate the process, only a few effective depth levels need to be included, preferably along the major boundaries in the subsurface. Propagation operators from these boundaries to the surface (focal operators) serve as the basis functions of this data decomposition method. Including more depth levels allows a sparser data representation, and hence, increases the reconstruction capability. The more precise the subsurface information is known, the more accurate these propagators can be computed. However, very precise operators are not necessary for a good reconstruction result, because in the reconstruction step (the inverse focal transformation) the effect of these operators is again removed. The calculation of the double focal transformation requires a non-linear inversion process, where the samples in the focal domain are estimated such that they - after inverse transformation - match the input data at the measurement locations. Because the inversion process is under-determined, an extra constraint on the focal domain is applied, for which the minimum L1 norm is chosen. This forces the distribution in the focal domain to be sparse and - thereby - suppresses the aliasing noise. For the inversion a so-called spgl1 solver has been used that is guaranteed to converge to the desired minimum of the defined objective function. It utilises a steepest decent type iterative process, called Spectral Projected Gradient. Seismic data reconstruction via the double focal transform method appears to be robust against inaccuracies in the focal operators up to roughly ten percent velocity error. Furthermore, the method was extended to the full 3D case, where each focal transform sub-domain in principle contains a 5D data space. In addition to the basic focal transformation, the method can be combined with other transforms in order to increase data compression. As an example, the double focal transformation can be combined with the linear Radon transformation, such that the seismic data can be represented sparser and fewer focal operators are necessary. Satisfactory results of focal domain data reconstruction beyond aliasing on 2D and 3D synthetic and 2D field data illustrate the method’s virtues.

Many seismic data processing and imaging processes require densely and regularly sampled data, whereas the actual measurements are mostly irregularly and sparsely sampled. Therefore, seismic data reconstruction methods are utilised as a pre-processing step. Within the class of transformation-based reconstruction techniques, observed seismic data is decomposed into certain basis functions, such as plane waves, parabolas or curvelets. In the corresponding model space the aliasing noise is assumed to have different properties than the seismic signal and can be suppressed. However, in many cases subsurface information is available that cannot be used in these traditional reconstruction methods. Therefore, the double focal transformation was derived as a way to incorporate knowledge about the subsurface in the reconstruction algorithm. The basic principle of the double focal transformation is to focus seismic energy by a back-propagation of the seismic data at the source and receiver side to certain depth levels. As a result, the seismic data are represented by a limited number of samples in the focal domain in a localised area, whereas aliasing noise spreads out. By imposing a sparse solution in the focal domain, aliasing noise is suppressed and data reconstruction beyond aliasing is achieved. To facilitate the process, only a few effective depth levels need to be included, preferably along the major boundaries in the subsurface. Propagation operators from these boundaries to the surface (focal operators) serve as the basis functions of this data decomposition method. Including more depth levels allows a sparser data representation, and hence, increases the reconstruction capability. The more precise the subsurface information is known, the more accurate these propagators can be computed. However, very precise operators are not necessary for a good reconstruction result, because in the reconstruction step (the inverse focal transformation) the effect of these operators is again removed. The calculation of the double focal transformation requires a non-linear inversion process, where the samples in the focal domain are estimated such that they - after inverse transformation - match the input data at the measurement locations. Because the inversion process is under-determined, an extra constraint on the focal domain is applied, for which the minimum L1 norm is chosen. This forces the distribution in the focal domain to be sparse and - thereby - suppresses the aliasing noise. For the inversion a so-called spgl1 solver has been used that is guaranteed to converge to the desired minimum of the defined objective function. It utilises a steepest decent type iterative process, called Spectral Projected Gradient. Seismic data reconstruction via the double focal transform method appears to be robust against inaccuracies in the focal operators up to roughly ten percent velocity error. Furthermore, the method was extended to the full 3D case, where each focal transform sub-domain in principle contains a 5D data space. In addition to the basic focal transformation, the method can be combined with other transforms in order to increase data compression. As an example, the double focal transformation can be combined with the linear Radon transformation, such that the seismic data can be represented sparser and fewer focal operators are necessary. Satisfactory results of focal domain data reconstruction beyond aliasing on 2D and 3D synthetic and 2D field data illustrate the method’s virtues.

To perform regression analysis in high dimensions, lasso or ridge estimation are a common choice. However, it has been shown that these methods are not robust to outliers. Therefore, alternatives as penalized M-estimation or the sparse least trimmed squares (LTS) estimator have been proposed. The robustness of these regression methods can be measured with the influence function. It quantifies the effect of infinitesimal perturbations in the data. Furthermore it can be used to compute the asymptotic variance and the mean squared error. In this paper we compute the influence function, the asymptotic variance and the mean squared error for penalized M-estimators and the sparse LTS estimator. The asymptotic biasedness of the estimators make the calculations non standard. We show that only M-estimators with a loss function with a bounded derivative are robust against regression outliers. In particular, the lasso has an unbounded influence function. ispartof: FEB Research Report KBI_1319 nrpages: 35 status: published

Discrete wavelet transform (DWT) has been widely adopted in biomedical signal processing for denoising, compression and so on. Choosing a suitable decomposition level (DL) in DWT is of paramount importance to its performance. In this paper, we propose to exploit sparseness of the transformed signals to determine the appropriate DL. Simulation results have shown that the sparseness of transformed signals after DWT increases with the increasing DLs. Additional Monte-Carlo simulation results have verified the effectiveness of sparseness measure in determining the DL., {"references":["","S. K. Mitra, Digital Signal Processing: A Computer Based Approach.\nMcGraw-Hill, 2005.","S. S. Patil and M. K. Pawar, \"Quality advancement of EEG by wavelet\ndenoising for biomedical analysis,\" in Proc. Int. Conf. on Communication,\nInformation and Computing Technology, Oct. 2012.","X. Liu, Y. J. Zheng, M. W. Phyu, B. Zhao, and X. J. Yuan, \"Power\n& area efficient wavelet-based on-chip ECG processor for WBAN,\" in\nProc. IEEE Int. Conf. on Body Sensor Networks, 2010, pp. 124–130.","P. Karthikeyan, M. Murugappan, and S.Yaacob, \"ECG signal denoising\nusing wavelet thresholding techniques in human stress assessment,\"\nInternational Journal on Electrical Engineering and Informatics, vol. 4,\nno. 2, Jul. 2012.","Z. Zhao and P. Min, \"ECG denoising by sparse wavelet shrinkage,\" in\nProc. IEEE Conf. Bioinformatics and Biomedical Engineering, 2007,\npp. 786–789.","X. Liu, Y. J. Zheng, M. W. Phyu, B. Zhao, M.-K. Y. Je, and X. J. Yuan,\n\"Multiple functional ECG signal is processing for wearable applications\nfor long-term cardiac monitoring,\" IEEE Trans. Biomedical Engineering,\nvol. 58, pp. 380–389, 2011.","R. Cohen, \"Signal denoising using wavelets,\" Technion, Israel Institute\nof Technology, Tech. Rep., 2011.","D. Donoho, \"De-noising by soft-thresholding,\" IEEE Trans. Information\nTheory, vol. 41, pp. 613–627, 1995.","Y. H. Peng, \"De-noising by modified soft-thresholding,\" in Proc. IEEE\nAsia-Pacific Conf. Circuits and Systems, 2000, pp. 760–762.\n[10] D. W. Yan-Fang Sang and J.-C. Wu, \"Entropy-based method of choosing\nthe decomposition level in wavelet threshold de-noising,\" Journal of\nEntropy and Information Studies, Jun. 2010.\n[11] J. S. M. D. C. Robertson, O. I. Camps and W. B. Gish, \"Wavelets and\nelectromagnetic power system transient,\" IEEE Trans. Power Delivery,\nvol. 11-2, pp. 1050–1058, Apr. 1996.\n[12] A. M. R. Dixon, G. Allstot, D. Gangopadhyay, and D. J. Allstot,\n\"Compressed sensing system considerations for ECG and EMG wireless\nbiosensors,\" IEEE Trans. Biomedical Circuits and Systems, vol. 6, no. 2,\npp. 155–166, Apr. 2012.\n[13] Online Available: http://gerstner.felk.cvut.cz/biolab/newbiolab/teach\n/mitdat.htm"]}

One of the most basic and general tasks faced by all nervous systems is extracting relevant information from the organism's surrounding world. While physical signals available to sensory systems are often continuous, variable, overlapping, and noisy, high-level neuronal representations used for decision-making tend to be discrete, specific, invariant, and highly separable. This study addresses the question of how neuronal specificity is generated. Inspired by experimental findings on network architecture in the olfactory system of the locust, I construct a highly simplified theoretical framework which allows for analytic solution of its key properties. For generalized feed-forward systems, I show that an intermediate range of connectivity values between source- and target-populations leads to a combinatorial explosion of wiring possibilities, resulting in input spaces which are, by their very nature, exquisitely sparsely populated. In particular, connection probability ½, as found in the locust antennal-lobe–mushroom-body circuit, serves to maximize separation of neuronal representations across the target Kenyon cells (KCs), and explains their specific and reliable responses. This analysis yields a function expressing response specificity in terms of lower network parameters; together with appropriate gain control this leads to a simple neuronal algorithm for generating arbitrarily sparse and selective codes and linking network architecture and neural coding. I suggest a straightforward way to construct ecologically meaningful representations from this code.

A method is proposed for unsupervised 3D (volume) segmentation of registered multichannel medical images. To this end, multichannel image is treated as 4D tensor represented by a multilinear mixture model, i.e. the image is modeled as weighted linear combination of 3D intensity distributions of organs (tissues) present in the image. Interpretation of this model suggests that 3D segmentation of organs (tissues) can be implemented through sparseness constrained factorization of the nonnegative matrix obtained by mode-4 unfolding of the 4D image tensor. Sparseness constraint implies that only one organ (tissue) is dominantly present at each pixel or voxel element. The method is preliminary validated, in term of Dice's coefficient, on extraction of brain tumor from synthetic multispectral magnetic resonance image obtained from the TumorSim database.

One of the main goals in neuroscience is to understand how the brain processes sensory information in order to drive proper behaviors. In this way, sensory systems constitute the first step for this complex computation. They integrate information about the external world and provide an appropriate signal that eventually leads to a behavioral response. The olfactory system, in particular the olfactory bulb (OB), represents a good model to study the sequential processes of sensory integration. This thesis attempts to elucidate the neural network properties that the OB displays to encode and discriminate odor-related sensory information. It further attempts to understand how these network features are related to odor perception ability in the behaving animal.

The concept of sparseness is harnessed to learn a low dimensional representation of high dimensional data. This sparseness assumption is exploited in multiple ways. In the Bayesian Elastic Net, a small number of correlated features are identified for the response variable. In the sparse Factor Analysis for biomarker trajectories, the high dimensional gene expression data is reduced to a small number of latent factors, each with a prototypical dynamic trajectory. In the Bayesian Graphical LASSO, the inverse covariance matrix of the data distribution is assumed to be sparse, inducing a sparsely connected Gaussian graph. In the nonparametric Mixture of Factor Analyzers, the covariance matrices in the Gaussian Mixture Model are forced to be low-rank, which is closely related to the concept of block sparsity. Finally in the information-theoretic projection design, a linear projection matrix is explicitly sought for information-preserving dimensionality reduction. All the methods mentioned above prove to be effective in learning both simulated and real high dimensional datasets.

peer-reviewed We compare improved classical backward elimination and forward selection methods of model selection in sparse contingency tables with methods based on a regularisation approach involving the least absolute shrinkage and selection operator (LASSO) and the Smooth LASSO. The results show that the modified classical methods outperform the regularisation methods, by producing sparser models which are always hierarchical. Curiously, models selected by the regularisation methods often include effects which are known to be inestimable in the classical paradigm. Our findings support the use of classical methodology.

This thesis consists of four papers that deal with several aspects of the measurement of model fit for categorical data. In all papers, special attention is paid to situations with sparse data. The first paper concerns the computational burden of calculating Pearson's goodness-of-fit statistic for situations where many response patterns have observed frequencies that equal zero. A simple solution is presented that allows for the computation of the total value of Pearson's goodness-of-fit statistic when the expected frequencies of response patterns with observed frequencies of zero are unknown. In the second paper, a new fit statistic is presented that is a modification of Pearson's statistic but that is not adversely affected by response patterns with very small expected frequencies. It is shown that the new statistic is asymptotically equivalent to Pearson's goodness-of-fit statistic and hence, asymptotically chi-square distributed. In the third paper, comprehensive simulation studies are conducted that compare seven asymptotically equivalent fit statistics, including the new statistic. Situations that are considered concern both multinomial sampling and factor analysis. Tests for the goodness-of-fit are conducted by means of the asymptotic and the bootstrap approach both under the null hypothesis and when there is a certain degree of misfit in the data. Results indicate that recommendations on the use of a fit statistic can be dependent on the investigated situation and on the purpose of the model test. Power varies substantially between the fit statistics and the cause of the misfit of the model. Findings indicate further that the new statistic proposed in this thesis shows rather stable results and compared to the other fit statistics, no disadvantageous characteristics of the fit statistic are found. Finally, in the fourth paper, the potential necessity of determining the goodness-of-fit by two sided model testing is adverted. A simulation study is conducted that investigates differences between the one sided and the two sided approach of model testing. Situations are identified for which two sided model testing has advantages over the one sided approach.

One of the most basic and general tasks faced by all nervous systems is extracting relevant information from the organism's surrounding world. While physical signals available to sensory systems are often continuous, variable, overlapping, and noisy, high-level neuronal representations used for decision-making tend to be discrete, specific, invariant, and highly separable. This study addresses the question of how neuronal specificity is generated. Inspired by experimental findings on network architecture in the olfactory system of the locust, I construct a highly simplified theoretical framework which allows for analytic solution of its key properties. For generalized feed-forward systems, I show that an intermediate range of connectivity values between source- and target-populations leads to a combinatorial explosion of wiring possibilities, resulting in input spaces which are, by their very nature, exquisitely sparsely populated. In particular, connection probability ½, as found in the locust antennal-lobe–mushroom-body circuit, serves to maximize separation of neuronal representations across the target Kenyon cells (KCs), and explains their specific and reliable responses. This analysis yields a function expressing response specificity in terms of lower network parameters; together with appropriate gain control this leads to a simple neuronal algorithm for generating arbitrarily sparse and selective codes and linking network architecture and neural coding. I suggest a straightforward way to construct ecologically meaningful representations from this code.

International audience; This paper proposes two new methods for the three-dimensional denoising of magnetic resonance images that exploit the sparseness and self-similarity properties of the images. The proposed methods are based on a three-dimensional moving-window discrete cosine transform hard thresholding and a three-dimensional rotationally invariant version of the well-known nonlocal means filter. The proposed approaches were compared with related state-of-the-art methods and produced very competitive results. Both methods run in less than a minute, making them usable in most clinical and research settings.

This paper discusses the role of primal and (Lagrange) dual model representations in problems of supervised and unsupervised learning. The specification of the estimation problem is conceived at the primal level as a constrained optimization problem. The constraints relate to the model which is expressed in terms of the feature map. From the conditions for optimality one jointly finds the optimal model representation and the model estimate. At the dual level the model is expressed in terms of a positive definite kernel function, which is characteristic for a support vector machine methodology. It is discussed how least squares support vector machines are playing a central role as core models across problems of regression, classification, principal component analysis, spectral clustering, canonical correlation analysis, dimensionality reduction and data visualization.

Recently projected gradient (PG) approaches have found many applications in solving the minimization problems underlying nonnegative matrix factorization (NMF). NMF is a linear representation of data that could lead to sparse result of natural images. To improve the parts-based representation of data some sparseness constraints have been proposed. In this paper the efficiency and execution time of five different PG algorithms and the basic multiplicative algorithm for NMF are compared. The factorization is done for an existing and proposed sparse NMF and the results are compared for all these PG methods. To compare the algorithms the resulted factorizations are used for a hand-written digit classifier QC 20110613