Search

Abstract A smooth group regularization method is proposed to identify and remove the redundant input nodes of feedforward neural networks, or equivalently the redundant dimensions of the input data of a given data set. This is achieved by introducing a smooth group L 1/2 regularizer with respect to the input nodes into the error function to drive some weight vectors of the input nodes to zero. The main advantage of the method is that it can remove not only the redundant nodes, but also some redundant weights of the surviving nodes. As a comparison, the L 1 regularization (Lasso) is mainly designed for removing the redundant weights, and it is not very good at removing the redundant nodes. And the group Lasso can remove the redundant nodes, but not any weight of the surviving nodes. Another advantage of the proposed method is that it uses a smooth function to replace the non-smooth absolute value function in the common L 1/2 regularizer, and thus it reduces the oscillation caused by the non-smoothness and enables us to prove the convergence properties of the proposed training algorithm. Numerical simulations are performed to illustrate the efficiency of the algorithm. [ABSTRACT FROM AUTHOR]

Unlike the first and the second generation artificial neural networks, spiking neural networks (SNNs) model the human brain by incorporating not only synaptic state but also a temporal component into their operating model. However, their intrinsic properties require expensive computation during training. This paper presents a novel algorithm to SpikeProp for SNN by introducing smoothing L 1 ∕ 2 regularization term into the error function. This algorithm makes the network structure sparse, with some smaller weights that can be eventually removed. Meanwhile, the convergence of this algorithm is proved under some reasonable conditions. The proposed algorithms have been tested for the convergence speed, the convergence rate and the generalization on the classical XOR-problem, Iris problem and Wisconsin Breast Cancer classification. [ABSTRACT FROM AUTHOR]

This paper proposes new techniques for data representation in the context of deep learning using agglomerative clustering. Existing autoencoder-based data representation techniques tend to produce a number of encoding and decoding receptive fields of layered autoencoders that are duplicative, thereby leading to extraction of similar features, thus resulting in filtering redundancy. We propose a way to address this problem and show that such redundancy can be eliminated. This yields smaller networks and produces unique receptive fields that extract distinct features. It is also shown that autoencoders with nonnegativity constraints on weights are capable of extracting fewer redundant features than conventional sparse autoencoders. The concept is illustrated using conventional sparse autoencoder and nonnegativity-constrained autoencoders with MNIST digits recognition, NORB normalized-uniform object data and Yale face dataset. [ABSTRACT FROM AUTHOR]

Accurate lung segmentation from large-size 3-D chest-computed tomography images is crucial for computer-assisted cancer diagnostics. To efficiently segment a 3-D lung, we extract voxel-wise features of spatial image contexts by unsupervised learning with a proposed incremental constrained nonnegative matrix factorization (ICNMF). The method applies smoothness constraints to learn the features, which are more robust to lung tissue inhomogeneities, and thus, help to better segment internal lung pathologies than the known state-of-the-art techniques. Compared to the latter, the ICNMF depends less on the domain expert knowledge and is more easily tuned due to only a few control parameters. Also, the proposed slice-wise incremental learning with due regard for interslice signal dependencies decreases the computational complexity of the NMF-based segmentation and is scalable to very large 3-D lung images. The method is quantitatively validated on simulated realistic lung phantoms that mimic different lung pathologies (seven datasets), in vivo datasets for 17 subjects, and 55 datasets from the Lobe and Lung Analysis 2011 (LOLA11) study. For the in vivo data, the accuracy of our segmentation w.r.t. the ground truth is 0.96 by the Dice similarity coefficient, 9.0 mm by the modified Hausdorff distance, and 0.87% by the absolute lung volume difference, which is significantly better than for the NMF-based segmentation. In spite of not being designed for lungs with severe pathologies and of no agreement between radiologists on the ground truth in such cases, the ICNMF with its total accuracy of 0.965 was ranked fifth among all others in the LOLA11. After excluding the nine too pathological cases from the LOLA11 dataset, the ICNMF accuracy increased to 0.986. [ABSTRACT FROM AUTHOR]

This paper discusses the stability condition for discrete-time multi-valued recurrent neural networks (MVNRNNs) in asynchronous update mode. In the existing research literature, an MVNRNN in asynchronous update mode has been found convergent if its weight matrix is Hermitian with nonnegative diagonal entries. However, our finding has been that the weight matrix with zero diagonal entries cannot guarantee the network stability. Furthermore, the new stability condition and proof is offered to allow diagonal entries to be complex-valued, which extends previous theoretical result. Simulation results are used to illustrate the theory. [ABSTRACT FROM AUTHOR]

Abstract: Minimization of the training regularization term has been recognized as an important objective for sparse modeling and generalization in feedforward neural networks. Most of the studies so far have been focused on the popular L 2 regularization penalty. In this paper, we consider the convergence of online gradient method with smoothing regularization term. For normal regularization, the objective function is the sum of a non-convex, non-smooth, and non-Lipschitz function, which causes oscillation of the error function and the norm of gradient. However, using the smoothing approximation techniques, the deficiency of the normal regularization term can be addressed. This paper shows the strong convergence results for the smoothing regularization. Furthermore, we prove the boundedness of the weights during the network training. The assumption that weights are bounded is no longer needed for the proof of convergence. Simulation results support the theoretical findings and demonstrate that our algorithm has better performance than two other algorithms with L 2 and normal regularizations respectively. [Copyright &y& Elsevier]

Abstract: This paper discusses a Competitive Layer Model (CLM) for a class of recurrent Cellular Neural Networks (CNNs) from continuous-time type to discrete-time type. The objective of the CLM is to partition a set of input features into salient groups. The complete convergence of such networks in continuous-time type has been discussed first. We give a necessary condition, and a necessary and sufficient condition, which allow the CLM performance existence in our networks. We also discuss the properties of such networks of discrete-time type, and propose a novel CLM iteration method. Such method shows similar performance and storage allocation but faster convergence compared with the previous CLM iteration method (). Especially for a large scale network with many features and layers, it can significantly reduce the computing time. Examples and simulation results are used to illustrate the developed theory, the comparison between two CLM iteration methods, and the application in image segmentation. [Copyright &y& Elsevier]

Abstract: This paper discusses a class of discrete time recurrent neural networks with multivalued neurons (MVN), which have complex-valued weights and an activation function defined as a function of the argument of a weighted sum. Complementing state-of-the-art of such networks, our research focuses on the convergence analysis of the networks in synchronous update mode. Two related theorems are presented and simulation results are used to illustrate the theory. [Copyright &y& Elsevier]

Ensemble classifiers have been shown efficient in multiple applications. In this article, the authors explore the effectiveness of ensemble classifiers in a case-based computer-aided diagnosis system for detection of masses in mammograms. They evaluate two general ways of constructing subclassifiers by resampling of the available development dataset: Random division and random selection. Furthermore, they discuss the problem of selecting the ensemble size and propose two adaptive incremental techniques that automatically select the size for the problem at hand. All the techniques are evaluated with respect to a previously proposed information-theoretic CAD system (IT-CAD). The experimental results show that the examined ensemble techniques provide a statistically significant improvement (AUC=0.905±0.024) in performance as compared to the original IT-CAD system (AUC=0.865±0.029). Some of the techniques allow for a notable reduction in the total number of examples stored in the case base (to 1.3% of the original size), which, in turn, results in lower storage requirements and a shorter response time of the system. Among the methods examined in this article, the two proposed adaptive techniques are by far the most effective for this purpose. Furthermore, the authors provide some discussion and guidance for choosing the ensemble parameters. [ABSTRACT FROM AUTHOR]

Abstract: The paper describes an application of artificial neural networks (ANN) for natural language text reasoning. The task of knowledge discovery in text from a database, represented with a database file consisting of sentences with similar meanings but different lexico-grammatical patterns, was solved with ANNs which recognize the meaning of the text using training files with limited dictionary. The paper features recognition algorithms of text meaning from a selected source using 3-layer ANNs. Tests of the new method have also been described. [Copyright &y& Elsevier]

We present a technique that enhances computer-assisted decision (CAD) systems with the ability to assess the reliability of each individual decision they make. Reliability assessment is achieved by measuring the accuracy of a CAD system with known cases similar to the one in question. The proposed technique analyzes the feature space neighborhood of the query case to dynamically select an input-dependent set of known cases relevant to the query. This set is used to assess the local (query-specific) accuracy of the CAD system. The estimated local accuracy is utilized as a reliability measure of the CAD response to the query case. The underlying hypothesis of the study is that CAD decisions with higher reliability are more accurate. The above hypothesis was tested using a mammographic database of 1337 regions of interest (ROIs) with biopsy-proven ground truth (681 with masses, 656 with normal parenchyma). Three types of decision models, (i) a back-propagation neural network (BPNN), (ii) a generalized regression neural network (GRNN), and (iii) a support vector machine (SVM), were developed to detect masses based on eight morphological features automatically extracted from each ROI. The performance of all decision models was evaluated using the Receiver Operating Characteristic (ROC) analysis. The study showed that the proposed reliability measure is a strong predictor of the CAD system’s case-specific accuracy. Specifically, the ROC area index for CAD predictions with high reliability was significantly better than for those with low reliability values. This result was consistent across all decision models investigated in the study. The proposed case-specific reliability analysis technique could be used to alert the CAD user when an opinion that is unlikely to be reliable is offered. The technique can be easily deployed in the clinical environment because it is applicable with a wide range of classifiers regardless of their structure and it requires neither additional training nor building multiple decision models to assess the case-specific CAD accuracy. [ABSTRACT FROM AUTHOR]

Estimation of the dynamic spinal forces from kinematics data is very complicated because it involves the handling, of the relationship between kinematic variables and electromyography (EMG) signals, as well as the relationship between EMG signals and the forces. A recurrent fuzzy neural network (RFNN) model is proposed to establish the kinematics-EMG—force relationship and model the dynamics of muscular activities. The EMG signals are used as an intermediate output and are fed back to the input layer. Since EMG is a direct reflection of muscular activities, the feedback of this model has a physical meaning. It expresses the dynamics of muscular activities in a straightforward way and takes advantage from the recurrent property. The trained model can then have the forces predicted directly from kinematic variables while bypassing the costly procedure of measuring EMG signals and avoiding the use of a biomechanics model. A learning algorithm is derived for the RFNN model. [ABSTRACT FROM AUTHOR]

Abstract: Superposition of radial basis functions centered at given prototype patterns constitutes one of the most suitable energy forms for gradient systems that perform nearest neighbor classification with real-valued static prototypes. It is shown in this paper that a continuous-time dynamical neural network model, employing a radial basis function and a sigmoid multi-layer perceptron sub-networks, is capable of maximizing such an energy form locally, thus performing almost perfectly nearest neighbor classification, when initiated by a distorted pattern. The proposed design scheme allows for explicit representation of prototype patterns as network parameters, as well as augmenting additional or forgetting existing memory patterns. The dynamical classification scheme implemented by the network eliminates all comparisons, which are the vital steps of the conventional nearest neighbor classification process. The performance of the proposed network model is demonstrated on binary and gray-scale image reconstruction applications. [Copyright &y& Elsevier]

The ability of bifurcating processing units and their networks to rapidly switch between different dynamic modes has been used in recent research efforts to model new computational properties of neural systems. In this spirit, we devise a bifurcating neuron based on control of chaos collapsing to a period-3 orbit in the dynamics of a quadratic logistic map (QLM). Proposed QLM3 neuron is constructed with the third iterate of QLM and uses an external input, which governs its dynamics. The input shifts the neuron''s dynamics from chaos to one of the stable fixed points. This way the inputs from certain ranges (clusters) are mapped to stable fixed points, while the rest of the inputs is mapped to chaotic or periodic output dynamics. It has been shown that QLM3 neuron is able to learn a specific mapping by adaptively adjusting its bifurcation parameter, the idea of which is based on the principles of parametric control of logistic maps [Proceedings of the International Symposium on Nonlinear Theory and its Applications (NOLTA''97), Honolulu, HI, 1997; Proceedings of SPIE, 2000]. Learning algorithm for the bifurcation parameter is proposed, which employs the error gradient descent method. [Copyright &y& Elsevier]

The author says that managing a publication of this size such as "IEEE Transactions on Neural Networks," is a monumental task. It requires a huge investment in time and careful awareness of policies and procedures. For a large part, it directs the editor's attention to the big picture of the discipline of neural networks and its associated publishing strategies. The journal has undergone many changes. Following the recommendations from the Society Review Committee in 2001, submissions of briefs have been discontinued.

This paper provides an overview of research focused on the utilization of neurocomputing technology to model critical in-process integrated circuit material and device characteristics. Artificial neural networks are employed to develop models of complex relationships between material and device characteristics at critical stages of the semiconductor fabrication process. Measurements taken and subsequently used in modeling include doping concentrations, layer thicknesses, planar geometries, resistivities, device voltages, and currents. The neural network architecture utilized in this research is the multilayer perceptron neural network (MLPNN). The MLPNN is trained in the supervised mode using the generalized delta learning rule. The MLPNN has demonstrated with good results the ability to model these characteristics, and provide an effective tool for parametric yield prediction and whole wafer characterization in semiconductor manufacturing. ©1999 John Wiley & Sons, Inc. Int J RF and Microwave CAE 9: 241–253, 1999. [ABSTRACT FROM AUTHOR]

Presents a generalization of the analog-to-digital converter concept to versatile computational circuits. Ability of a neural network structure to perform the operations of analogue and binary addition; Computation of conductance values for an analogue adder.

Describes the structural learning with forgetting algorithm, with focus on the development of a continuous dynamical system model of regularization in which the associated regularization parameter is generalized to be a time-varying function. Features of the algorithm; Implications of the results; Conclusions reached.

The article discusses various reports published within the issue, including one on the modelling of typicality in terms of Atanassov's IFSs (A-IFSs), another on an application of neural networks for medical diagnostics of recurrent breast cancer and another on the factorisation of systems of fuzzy sets by similarity.

This paper presents an efficient technique to reduce the inference cost of deep and/or wide convolutional neural network models by pruning redundant features (or filters). Previous studies have shown that over-sized deep neural network models tend to produce a lot of redundant features that are either shifted version of one another or are very similar and show little or no variations, thus resulting in filtering redundancy. We propose to prune these redundant features along with their related feature maps according to their relative cosine distances in the feature space, thus leading to smaller networks with reduced post-training inference computational costs and competitive performance. We empirically show on select models (VGG-16, ResNet-56, ResNet-110, and ResNet-34) and dataset (MNIST Handwritten digits, CIFAR-10, and ImageNet) that inference costs (in FLOPS) can be significantly reduced while overall performance is still competitive with the state-of-the-art. [ABSTRACT FROM AUTHOR]

To enhance the sparseness of the network, improve its generalization ability and accelerate its training, we propose a novel pruning approach for sigma-pi-sigma neural network (SPSNN) under the relaxed condition by adding smoothing group L 1 / 2 regularization and adaptive momentum. The main strength of this method is that it can prune both the redundant nodes between groups in the network, and also the redundant weights of the non-redundant nodes within the group, so as to achieve the sparseness of the network. Another strength is that the non-smooth absolute value function in the traditional L 1 / 2 regularization method is replaced by a smooth function. This reduces the oscillations of learning and enables us to more effectively prove the convergence of the proposed algorithm. Finally, the numerical simulation results demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Editorial. Comments on the changes made to the publication and acceptance of articles for the journal 'IEEE Transactions on Neural Networks.' Choices for authors with regard to the submission of articles for review and publication; Guidelines for the submission of papers in electronic format; Topics which will be given focus in future issues of the journal in 2002.

Abstract: This paper presents how commonly used machine learning classifiers can be analyzed using a common framework of convex optimization. Four classifier models, the Support Vector Machine (SVM), the Least-Squares SVM (LSSVM), the Extreme Learning Machine (ELM), and the Margin Loss ELM (MLELM) are discussed to demonstrate how specific parametrizations of a general problem statement affect the classifier design and performance, and how ideas from the four different classifiers can be mixed and used together. Furthermore, 21 public domain benchmark datasets are used to experimentally evaluate five performance metrics of each model and corroborate the theoretical analysis. Comparison of classification accuracies under a nested cross-validation evaluation shows that with an exception all four models perform similarly on the evaluated datasets. However, the four classifiers command different amounts of computational resources for both testing and training. These requirements are directly linked to their formulations as different convex optimization problems. [Copyright &y& Elsevier]

Abstract: The aim of this paper is to develop a novel method to prune feedforward neural networks by introducing an regularization term into the error function. This procedure forces weights to become smaller during the training and can eventually removed after the training. The usual regularization term involves absolute values and is not differentiable at the origin, which typically causes oscillation of the gradient of the error function during the training. A key point of this paper is to modify the usual regularization term by smoothing it at the origin. This approach offers the following three advantages: First, it removes the oscillation of the gradient value. Secondly, it gives better pruning, namely the final weights to be removed are smaller than those produced through the usual regularization. Thirdly, it makes it possible to prove the convergence of the training. Supporting numerical examples are also provided. [Copyright &y& Elsevier]

Abstract: Weight decay method as one of classical complexity regularization methods is simple and appears to work well in some applications for backpropagation neural networks (BPNN). This paper shows results for the weak and strong convergence for cyclic and almost cyclic learning BPNN with penalty term (CBP-P and ACBP-P). The convergence is guaranteed under certain relaxed conditions for activation functions, learning rate and under the assumption for the stationary set of error function. Furthermore, the boundedness of the weights in the training procedure is obtained in a simple and clear way. Numerical simulations are implemented to support our theoretical results and demonstrate that ACBP-P has better performance than CBP-P on both convergence speed and generalization ability. [Copyright &y& Elsevier]

Abstract: Conjugate gradient methods have many advantages in real numerical experiments, such as fast convergence and low memory requirements. This paper considers a class of conjugate gradient learning methods for backpropagation neural networks with three layers. We propose a new learning algorithm for almost cyclic learning of neural networks based on PRP conjugate gradient method. We then establish the deterministic convergence properties for three different learning modes, i.e., batch mode, cyclic and almost cyclic learning. The two deterministic convergence properties are weak and strong convergence that indicate that the gradient of the error function goes to zero and the weight sequence goes to a fixed point, respectively. It is shown that the deterministic convergence results are based on different learning modes and dependent on different selection strategies of learning rate. Illustrative numerical examples are given to support the theoretical analysis. [Copyright &y& Elsevier]

A multilayer neural network based on multivalued neurons (MLMVN) is a neural network with a traditional feedforward architecture. At the same time, this network has a number of specific different features. Its backpropagation learning algorithm is derivative-free. The functionality of MLMVN is superior to that of the traditional feedforward neural networks and of a variety kernel-based networks. Its higher flexibility and faster adaptation to the target mapping enables to model complex problems using simpler networks. In this paper, the MLMVN is used to identify both type and parameters of the point spread function, whose precise identification is of crucial importance for the image deblurring. The simulation results show the high efficiency of the proposed approach. It is confirmed that the MLMVN is a powerful tool for solving classification problems, especially multiclass ones. [ABSTRACT FROM AUTHOR]

Abstract: This study investigates the effect of class imbalance in training data when developing neural network classifiers for computer-aided medical diagnosis. The investigation is performed in the presence of other characteristics that are typical among medical data, namely small training sample size, large number of features, and correlations between features. Two methods of neural network training are explored: classical backpropagation (BP) and particle swarm optimization (PSO) with clinically relevant training criteria. An experimental study is performed using simulated data and the conclusions are further validated on real clinical data for breast cancer diagnosis. The results show that classifier performance deteriorates with even modest class imbalance in the training data. Further, it is shown that BP is generally preferable over PSO for imbalanced training data especially with small data sample and large number of features. Finally, it is shown that there is no clear preference between oversampling and no compensation approach and some guidance is provided regarding a proper selection. [Copyright &y& Elsevier]

Single-class minimax probability machines (MPMs) offer robust novelty detection with distribution-free worst case bounds on the probability that a pattern will fall inside the normal region. However, in practice, they are too cautious in labeling pat- terns as outlying and so have a high false negative rate (FNR). In this paper, we propose a more aggressive version of the single-class MPM that bounds the best case probability that a pattern will fall inside the normal region. These two MPMs can then be used together to delimit the solution space. By using the hyperplane lying in the middle of this pair of MPMs, a better compromise between false positives (FPs) and false negatives (FNs), and between recall and precision can be obtained. Experiments on the real-world data sets show encouraging results. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we present a new approach of designing adaptive neural network controllers for uncertain systems containing nonsmooth nonlinearities in the actuator device. The controllers are designed by introducing certain well-defined sign functions and neural network approximations as well as by using the backstepping technique. The salient feature of the approach is that no knowledge is assumed on unknown system parameters and nonlinearities. It is shown that the proposed controller not only can guarantee semi-global stability, but also excellent transient performance can be achieved. [Copyright &y& Elsevier]

Kernel methods, such as the support vector machine (SVM), are often formulated as quadratic programming (QP) problems. However, given m training patterns, a naive implementation of the QP solver takes O(m3) training time and at least O(m2) space. Hence, scaling up these QPs is a major stumbling block in applying kernel methods on very large data sets, and a replacement of the naive method for finding the QP solutions is highly desirable. Recently, by using approximation algorithms for the minimum enclosing ball (MEB) problem, we proposed the core vector machine (CVM) algorithm that is much faster and can handle much larger data sets than existing SVM implementations. However, the CVM can only be used with certain kernel functions and kernel methods. For example, the very popular support vector regression (SVR) cannot be used with the CVM. In this paper, we introduce the center-constrained MEB problem and subsequently extend the CVM algorithm. The generalized CVM algorithm can now be used with any linear/nonlinear kernel and can also be applied to kernel methods such as SVR and the ranking SVM. Moreover, like the original CVM, its asymptotic time complexity is again linear in m and its space complexity is independent of m. Experiments show that the generalized CVM has comparable performance with state-of-the-art SVM and SVR implementations, but is faster and produces fewer support vectors on very large data sets. [ABSTRACT FROM AUTHOR]

An energy function-based autoassociative memory design method to store a given set of unipolar binary memory vectors as attractive fixed points of an asynchronous discrete Hopfield network (DHN) is presented. The discrete quadratic energy function whose local minima correspond to the attractive fixed points of the network is constructed via solving a system of linear inequalities derived from the strict local minimality conditions. The weights and the thresholds are then calculated using this energy function. If the inequality system is infeasible, we conclude that no such asynchronous DHN exists, and extend the method to design a discrete piecewise quadratic energy function, which can be minimized by a generalized version of the conventional DHN, also proposed herein. In spite of its computational complexity, computer simulations indicate that the original method performs better than the conventional design methods in the sense that the memory can store, and provide the attractiveness for almost all memory sets whose cardinality is less than or equal to the dimension of its elements. The overall method, together with its extension, guarantees the storage of an arbitrary collection of memory vectors, which are mutually at least two Hamming distances away from each other, in the resulting network. [ABSTRACT FROM AUTHOR]

The olfactory system is a very efficient biological setup capable of odor information processing with neural signals. The nature of neural signals restricts the information representation to multidimensional temporal sequences o. spikes. The information is contained in the interspike intervals within each individual neural signal and interspike intervals between multiple signals. A mechanism of interactions between random excitations evoked by odorants in the olfactory receptors of the epithelium and deterministic operation of the olfactory bulb is proposed in this paper. Inverse Frobenius-Perron models of the bulb's temporal sequences are fitted to the interspike distributions of temporally modulated receptor signals. Ultimately, such pattern matching results in ability to recognize odors and offer a hypothetic model for signal processing occurring in the primary stage of the olfactory system. [ABSTRACT FROM AUTHOR]

Analysis of backscatter in the ultrasound echo envelope, in conjunction with ultrasound B-scans, can provide important information for tissue characterization and pathology diagnosis. Statistical models have often proven useful in modeling backscatter. In this paper, an innovative approach to backscatter analysis based on generalized entropies and neural function approximation is presented. Entropy measures are shown to provide accurate estimates of scatterer density, regularity, and SNR of the amplitude distribution. Specific scattering distributions need not be assumed. Experimental results on ground truth envelopes show that generalized entropies can be used to accurately estimate backscatter properties. [Copyright &y& Elsevier]

Focuses on a study that presented a design method for the complex-valued multistate Hopfield associative memory neural network. Design procedure; Simulation results; Conclusion.

This paper proposes temporal-to-spatial dynamic mapping inspired by neural dynamics of the olfactory cortex. In our model the temporal structure of olfactory-bulb patterns is mapped to the spatial dynamics of the ensemble of cortical neurons. This mapping is based on the following biological mechanism: while anterior part of piriform cortex can be excited by the afferent input alone, the posterior areas require both afferent and association signals, which are temporally correlated in a specific way. One of the functional types of the neurons in our model corresponds to the cortical spatial dynamics and encodes odor components, and another represents temporal activity of association-fiber signals, which, we suggest, may be relevant to the encoding of odor concentrations. The temporal-to-spatial mapping and distributed representation of the model enable simultaneous rough cluster classification and fine recognition of patterns within a cluster as parts of the same dynamic process. The model is able to extract and segment the components of complex odor patterns which are spatiotemporal sequences of neural activity. [ABSTRACT FROM AUTHOR]

Presents information on the use of artificial neural networks (ANN) to estimate the shape parameter of K distribution, a specific backscattering model. Application of ultrasonic backscatter models on different clinical analysis; Features of ANN; Information on the K distribution model.

Presents a study that tested an approach for extracting rules from trained neural networks for nonlinear regression. Overview of related works for function approximation, pruning and rule extraction; Description of the neural training and pruning algorithm; Examination of the effectiveness of the proposed approach for function approximation.

We propose an embedded/integrated feature selection method based on neural networks with Group Lasso penalty. Group Lasso regularization is considered to produce sparsity on the inputs to the network, i.e., for selection of useful features. Lasso based feature selection using a multi-layer perceptron usually requires an additional set of weights, while our Group Lasso formulation does not require that. However, Group Lasso penalty is non-differentiable at the origin. This may lead to oscillations in numerical simulations and make it difficult to analyze theoretically. To address this issue, four smoothing Group Lasso penalties are introduced. A rigorous proof for the convergence of the proposed algorithm is presented under suitable assumptions. To verify the effectiveness, a three-step algorithmic architecture is adopted in implementation. Experimental results on several datasets validate the theoretical results and demonstrate the competitive performance of the proposed method. [ABSTRACT FROM AUTHOR]

Background and Objective Anemia is a common comorbidity in patients with chronic kidney disease (CKD) and is frequently associated with decreased physical component of quality of life, as well as adverse cardiovascular events. Current treatment methods for renal anemia are mostly population-based approaches treating individual patients with a one-size-fits-all model. However, FDA recommendations stipulate individualized anemia treatment with precise control of the hemoglobin concentration and minimal drug utilization. In accordance with these recommendations, this work presents an individualized drug dosing approach to anemia management by leveraging the theory of optimal control. Methods A Multiple Receding Horizon Control (MRHC) approach based on the RBF-Galerkin optimization method is proposed for individualized anemia management in CKD patients. Recently developed by the authors, the RBF-Galerkin method uses the radial basis function approximation along with the Galerkin error projection to solve constrained optimal control problems numerically. The proposed approach is applied to generate optimal dosing recommendations for individual patients. Results Performance of the proposed approach (MRHC) is compared in silico to that of a population-based anemia management protocol and an individualized multiple model predictive control method for two case scenarios: hemoglobin measurement with and without observational errors. In silico comparison indicates that hemoglobin concentration with MRHC method has less variation among the methods, especially in presence of measurement errors. In addition, the average achieved hemoglobin level from the MRHC is significantly closer to the target hemoglobin than that of the other two methods, according to the analysis of variance (ANOVA) statistical test. Furthermore, drug dosages recommended by the MRHC are more stable and accurate and reach the steady-state value notably faster than those generated by the other two methods. Conclusions The proposed method is highly efficient for the control of hemoglobin level, yet provides accurate dosage adjustments in the treatment of CKD anemia. [ABSTRACT FROM AUTHOR]

Weight elimination offers a simple and efficient improvement of training algorithm of feedforward neural networks. It is a general regularization technique in terms of the flexible scaling parameters. Actually, the weight elimination technique also contains the weight decay regularization for a large scaling parameter. Many applications of this technique and its improvements have been reported. However, there is little research concentrated on its convergence behavior. In this paper, we theoretically analyze the weight elimination for cyclic learning method and determine the conditions for the uniform boundedness of weight sequence, and weak and strong convergence. Based on the assumed network parameters, the optimal choice for the scaling parameter can also be determined. Moreover, two illustrative simulations have been done to support the theoretical explorations as well. [ABSTRACT FROM AUTHOR]