Search

Prototype-based learning (PbL) using a winner-take-all (WTA) network based on minimum Euclidean distance (ED-WTA) is an intuitive approach to multiclass classification. By constructing meaningful class centers, PbL provides higher interpretability and generalization than hyperplane-based learning (HbL) methods based on maximum inner product (IP-WTA) and can efficiently detect and reject samples that do not belong to any classes. In this article, we first prove the equivalence of IP-WTA and ED-WTA from a representational power perspective. Then, we show that naively using this equivalence leads to unintuitive ED-WTA networks in which the centers have high distances to data that they represent. We propose ±ED-WTA that models each neuron with two prototypes: one positive prototype, representing samples modeled by that neuron, and a negative prototype, representing the samples erroneously won by that neuron during training. We propose a novel training algorithm for the ±ED-WTA network, which cleverly switches between updating the positive and negative prototypes and is essential to the emergence of interpretable prototypes. Unexpectedly, we observed that the negative prototype of each neuron is indistinguishably similar to the positive one. The rationale behind this observation is that the training data that are mistaken for a prototype are indeed similar to it. The main finding of this article is this interpretation of the functionality of neurons as computing the difference between the distances to a positive and a negative prototype, which is in agreement with the BCM theory. Our experiments show that the proposed ±ED-WTA method constructs highly interpretable prototypes that can be successfully used for explaining the functionality of deep neural networks (DNNs), and detecting outlier and adversarial examples.

PCANet is an unsupervised Convolutional Neural Network (CNN), which uses Principal Component Analysis (PCA) to learn convolutional filters. One drawback of PCANet is that linear PCA cannot capture nonlinear structures within data. To address this problem, a straightforward approach is utilizing kernel methods by equipping the PCA method in PCANet with a kernel function. However, this practice leads to a network having cubic complexity with respect to the number of training image patches. In this paper, we propose a network called Randomized Nonlinear PCANet (RNPCANet), which uses an explicit kernel PCA to learn the convolutional filters. Although RNPCANet utilizes kernel methods for nonlinear processing of data, using kernel approximation techniques to define an explicit feature space in each stage, we theoretically show that the complexity of this model is not much higher than that of PCANet. We also show that our method links PCANets to Convolutional Kernel Networks (CKNs) as the proposed model maps the patches to a kernel feature space similar to CKNs. We evaluate our model on image recognition tasks including Coil-20, Coil-100, ETH-80, Caltech-101, MNIST, and C-Cube datasets. The experimental results show that the proposed method has superiority over PCANet and CKNs in terms of recognition accuracy.

After Minsky and Papert (Perceptrons, MIT Press, Cambridge, 1969) showed the inability of perceptrons in solving nonlinearly separable problems, for several decades people misinterpreted it as an inherent weakness that is common to all single-layer neural networks. The introduction of the backpropagation algorithm reinforced this misinterpretation as its success in solving nonlinearly separable problems passed through the training of multilayer neural networks. Recently, Conaway and Kurtz (Neural Comput 29(3):861–866, 2017) proposed a single-layer network in which the number of output units for each class is the same as input units and showed that it could solve some nonlinearly separable problems. They used the MSE (Mean Square Error) between the input units and the output units of the actual class as the objective function for training the network. They showed that their method could solve the XOR and M&S’81 problems, but it could not do any better than random guessing on the 3-bit parity problem. In this paper, we use a soft competitive approach to generalize the CE (Cross-Entropy) loss, which is a widely accepted criterion for multiclass classification, to networks that have several output units for each class, calling the resulting measure the CCE (Competitive cross-entropy) loss. In contrast to Conaway and Kurtz (2017), in our method, the number of output units for each class can be chosen arbitrarily. We show that the proposed method can successfully solve the 3-bit parity problem, in addition to the XOR and M&S’81 problems. Furthermore, we perform experiments on several datasets for multiclass classification, comparing a single-layer network trained with the proposed CCE loss against LVQ, linear SVM, a single-layer network trained with the CE loss, and the method of Conaway and Kurtz (2017). The results show that the CCE loss performs remarkably better than existing algorithms for training single-layer neural networks.

Gabor functions have wide-spread applications both in analyzing the visual cortex of mammalians and in designing machine vision algorithms. It is known that the receptive field of neurons of V1 layer in the visual cortex can be accurately modeled by Gabor functions. In addition, Gabor functions are extensively used for feature extraction in machine vision tasks. In this paper, we prove that Gabor functions are translation-invariant positive-definite kernels and show that the problem of image representation with Gabor functions can be formulated as infinite kernel learning regression. Specifically, we use the stabilized infinite kernel learning regression algorithm that has already been introduced for learning translation-invariant positive-definite kernels and has enough flexibility and generality to embrace the class of Gabor kernels. The algorithm yields a representation of the image as a support vector expansion with a compound kernel that is a finite mixture of Gabor functions. The problem with this representation is that all Gabor functions are present at all support vector pixels. Using LASSO, we propose a method for sparse representation of an image with Gabor functions in which each Gabor function is positioned at a very sparse set of pixels. As a practical application, we introduce a novel method for learning a dataset-specific set of Gabor filters that can be used subsequently for feature extraction. Our experiments on CMU-PIE and Extended Yale B datasets show that use of the learned Gabor filters significantly improves the recognition accuracy of a recently introduced face recognition algorithm.

Runtime reconfiguration is a fundamental requirement of many telecommunication applications which also has been addressed by management standards like CMIP, 3GPP TS 32.602, and NETCONF. Two basic commands considered by these standards are CREATE and DELETE which operate on managed objects inside an application. The available configuration management technologies, like JMX, OSGi, and Fractal, do not support the CREATE and DELETE reconfiguration commands of the telecommunication standards. In this paper, we introduce a novel technology, called RSCM, for development of runtime reconfigurable applications complying with the telecommunication standards. The RSCM subagent takes the responsibility of loading the application from the configuration file, executing the runtime reconfiguration commands (including CREATE and DELETE), enforcing validity of the configuration state, and updating the configuration file according to the latest reconfiguration commands. We exploit the modular and object oriented features of the XML technology for storing the configuration state of a program in a configuration file. The software development process is tailored such that the design of XML schemas of managed classes is performed parallel to the design of software classes. In addition, a novel programming approach based on indirect referencing is proposed which allows safe and almost immediate deletion of managed objects at runtime. This indirect referencing mechanism affects the implementation of associations in class diagrams and prevents methods of a class to use the this pointer freely. The RSCM technology has been successfully used in several commercial telecommunication applications; including an SMS service center, an SMS gateway, and an SMS hub. A technology is proposed for developing runtime reconfigurable telecommunication applications.The CREATE and DELETE runtime-reconfiguration commands are handled automatically by the RSCM subagent.Provisioning the development of manageable software libraries with built-in CREATE and DELETE runtime-reconfigurability.Providing a mechanism for safe and almost immediate deletion of managed objects.Supporting inheritance relation among managed objects in the configuration file.Providing a mechanism for modular/component-based design of the XML schema during development of software components. Each component is accompanied with its own XML schema.Complete handling of the configuration file by the RSCM subagent.

Background and objectives Diabetic retinopathy (DR) is the leading cause of blindness worldwide, and therefore its early detection is important in order to reduce disease-related eye injuries. DR is diagnosed by inspecting fundus images. Since microaneurysms (MA) are one of the main symptoms of the disease, distinguishing this complication within the fundus images facilitates early DR detection. In this paper, an automatic analysis of retinal images using convolutional neural network (CNN) is presented. Methods Our method incorporates a novel technique utilizing a two-stage process with two online datasets which results in accurate detection while solving the imbalance data problem and decreasing training time in comparison with previous studies. We have implemented our proposed CNNs using the Keras library. Results In order to evaluate our proposed method, an experiment was conducted on two standard publicly available datasets, i.e., Retinopathy Online Challenge dataset and E-Ophtha-MA dataset. Our results demonstrated a promising sensitivity value of about 0.8 for an average of >6 false positives per image, which is competitive with state of the art approaches. Conclusion Our method indicates significant improvement in MA-detection using retinal fundus images for monitoring diabetic retinopathy.

Most sequence processing tasks can be cast as a problem of mapping a sequence of observations into a sequence of labels. This is a very difficult problem since the association between input data sequences and output label sequences is not given at the frame level. Recurrent neural networks (RNNs) equipped with connectionist temporal classification (CTC) are among the best tools devised to handle this problem and have been used to achieve state of the art results in many handwritten and speech recognition tasks. The reason that RNNs are used instead of feedforward networks in combination with CTC is that CTC does not model the dynamics of sequences. Specifically, the long short term memory (LSTM) RNN, which is excellent at memorizing information for a long time, is used in combination with CTC to overcome the limitations of CTC in modeling the dynamics of sequences. In this paper, we propose to model each label with a sequence of hidden sub-labels at the CTC level. The proposed framework allows CTC to learn the intra-label relations which transfers part of the load of learning dynamical sequences from RNN to CTC. Our experiments on handwriting recognition tasks show that the proposed method outperforms standard CTC in terms of accuracy.

Convolutional Neural Networks (CNNs) are multi-layer deep structures that have been very successful in visual recognition tasks. These networks basically consist of the convolution, pooling, and the nonlinearity layers, each of which operates on the representation produced by the preceding layer and generates a new representation. Convolution layers naturally compute some inner product between a plane represented by the weight parameters and input patches. Recently, Generalized Convolutional Neural Networks (GCNN) have been introduced which justify the use of some kernels or distance functions in place of the inner product operator inside the convolution layers. Although GCNNs gained interesting results on the MNIST dataset, their application to more challenging datasets is hindered by lack of an efficient implementation. In this paper, we focus on a specific generalized convolution operator which is based on the weighted L2 norm distance (WL2Dist). By replacing the nonlinear part with three convolution operators and using effective matrix-matrix multiplications, we were able to efficiently compute the WL2Dist convolution layer both on CPU and GPU. Our experiments show that, on CPU (GPU), the proposed implementation of the WL2Dist layer achieves a 5.5x (21x) speed-up over the initial BLAS-based (CUDA-based) implementations.

In this paper we propose a new method for extending 1-D step edge detection filters to two dimensions via complex-valued filtering. Complex-valued filtering allows us to obtain edge magnitude and direction simultaneously. Our method can be viewed either as an extension of n-directional complex filtering of Paplinski to infinite directions or as a variant of Canny's gradient-based approach. In the second view, the real part of our filter computes the gradient in the x direction and the imaginary part computes the gradient in the y direction. Paplinski claimed that n-directional filtering is an improvement over the gradient-based method, which computes gradient only in two directions. We show that our omnidirectional and Canny's gradient-based extensions of the 1-D DoG coincide. In contrast to Paplinski's claim, this coincidence shows that both approaches suffer from being confined to the subspace of two 2-D filters, even though n-directional filtering hides these filters in a single complex-valued filter. Aside from these theoretical results, the omnidirectional method has practical advantages over both n-directional and gradient-based approaches. Our experiments on synthetic and real-world images show the superiority of omnidirectional and gradient-based methods over n-directional approach. In comparison with the gradient-based method, the advantage of omnidirectional method lies mostly in freeing the user from specifying the smoothing window and its parameter. Since the omnidirectional and Canny's gradient-based extensions of the 1-D DoG coincide, we have based our experiments on extending the 1-D Demigny filter. This filter has been proposed by Demigny as the optimal edge detection filter in sampled images.