Search

One of the significant challenges in Brain–Computer Interface (BCI) is to develop a classifier that can decode users' mental states based on electroencephalogram (EEG) data collected from independent subjects. The focus of such subject-independent (SI) classification is justified because it can lead to BCIs that do not require a user-specific calibration process. In recent years, the emergence of deep neural networks (DNNs) has significantly improved the performance of EEG classification. Among various deep learning techniques, the training efficiency and the performance of Convolutional Neural Networks (CNNs), in particular, have led to several state-of-the-art architectures for accurate classification of EEG. Not surprisingly, the efforts to improve the performance of these architectures for EEG classification have been ramped up in recent years. In this regard, a trivial approach is to train and tune a large number of architectures and hyperparameters and hope to improve upon the existing results. In contrast with this ad hoc approach, here we put forward a systematic method inspired by the jackknife estimation to improve the performance of existing CNN architectures. Using EEGNet and ShallowConvNet as archetypical, our empirical results show that the proposed "Delete-a-Subject Jackknife" (DASJ) technique can potentially improve the performance of existing CNN architectures for SI classification of EEG. • Proposes DASJ-CNN which is a jackknife-inspired approach with Convolutional Neural Network (CNN) base classifiers. • Presents an analytical motivation of the DASJ ensemble classification. • Evaluates the effectiveness of the method using two state-of-the-art CNN architectures for EEG classification. • Demonstrates the capability of the method to improve the performance of existing CNN architectures. [ABSTRACT FROM AUTHOR]

• A novel application of hybrid method based on EDA for CNN training in the text categorization task. • The CNN training is divided into two parts, allowing the combination of several algorithms for weights optimization. • Accuracy of optimization via EDA improves in comparison with two gradient-based methods. Convolutional Neural Networks for text categorization allows the extraction of features from the text represented through word embedding. The high dimensionality of the texts themselves implies a larger number of network parameters and a more complex optimization surface. Artificial neural network training is an NP-Hard optimization problem, which has been addressed by methods based on partial derivatives of the objective function and presents several theoretical and practical limitations, such as the probability of convergence to local minimums. In this work, we propose a hybrid method based on the Estimation of Distribution Algorithms for training a Convolutional Neural Network. For this, we train together gradient-based methods with the Estimation of Multivariate Normal Algorithm and Univariate Marginal Distribution Algorithm by dividing the training process into two stages. The different variants obtained with the proposed method are compared with gradient-based methods on public benchmark datasets and statistical differences are analyzed by nonparametric tests. The proposed method increases the accuracy of the convolutional network applied to the text categorization task and overcome in about 0.22%–24% the state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

• A CNN-based exemplar prediction model is designed to enhance the original Siam-RPN. • Both the temporal and spatial information around an object are introduced to the prediction model. • The proposed tracker performs better than the existing trackers on the PTB-TIR benchmark. • The proposed tracker runs in real-time. Tracking pedestrian targets over a thermal infrared (TIR) image sequence is a hot topic in visual tracking. The imagery characteristics of TIR targets such as low target-background contrast and far imaging distance make TIR object tracking very difficult. In this paper, based on a convolutional neural network (CNN) and the siamese region proposal network (SiamRPN), we design an improved TIR pedestrian tracker. By fully considering the temporal and spatial information around an object, we firstly construct a CNN-based prediction model to produce the exemplar of a pedestrian target. Then the predicted exemplar is combined with SiamRPN to form an improved real-time TIR pedestrian tracker. The proposed tracker is evaluated on the TIR pedestrian tracking benchmark dataset PTB-TIR. Our experimental results demonstrate that the proposed tracker achieves promising tracking performance. In terms of tracking success rate and precision, our tracker outperforms traditional trackers such as KCF, and state-of-the-art trackers such as SiamRPN, SRDCF, and DSST. Moreover, similar to other siamese-network-based trackers, our tracker runs in real-time. [ABSTRACT FROM AUTHOR]

• A model to detect distraction of drivers. • Utilizes Convolutional Neural Networks to detect the activity being performed by the driver. • The model is able to differentiate between the types of distractions as well. • High detection accuracy of 99% achieved over a large dataset. • The model proposed, takes significantly less training time and delivers high test accuracy. With the intervention of social media and internet technology, people are getting more and more careless and distracted while driving which is having a severe detrimental effect on the safety of the driver and his fellow passengers. To provide an effective solution, this paper puts forward a Machine Learning model using Convolutional Neural Networks to not only detect the distracted driver but also identify the cause of his distraction by analyzing the images obtained using the camera module installed inside the vehicle. Convolutional Neural Networks are known to learn spatial features from images, which can be further examined by fully connected neural networks. The experimental results show a 99% average accuracy in distraction recognition and hence strongly support that our Convolutional Neural Networks model can be used to identify distraction among the drivers. [ABSTRACT FROM AUTHOR]

• We propose a novel Hybrid CNN architecture to address the problem of sketch recognition. • S-Net extracts shape features, which is invariant for sketch rotation and transformation. • Hybrid CNN achieves state-of-the-art on sketch classification and sketch-based image retrieval tasks. • The classification accuracy is 84.42% and 82.74% on TU-Berlin and SketchX datasets, respectively. • The MAP of SBIR is 57.4% and 28.74% on SketchX and Flickr15K-Large datasets, respectively. With the popularity of touch-screen devices, it is becoming increasingly important to understand users' free-hand sketches in computer vision and human-computer interaction. Most of existing sketch recognition methods employ the similar strategies used in image recognition, relying on appearance information represented by hand-crafted features or deep features from convolutional neural networks. We believe that sketch recognition can benefit from learning both appearance and shape representation. In this paper, we propose a novel architecture, named Hybrid CNN, which is composed of A-Net and S-Net. They describe appearance information and shape information, respectively. Hybrid CNN is then comprehensively evaluated in the sketch classification and retrieval tasks on different datasets, including TU-Berlin, Sketchy and Flickr15k. Experimental results demonstrate that the Hybrid CNN achieves competitive accuracy compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• We propose a category-wise residual attention learning framework (CRAL) for multi-label chest X-ray image classification. • CRAL benefits from both relevant and irrelevant pathologies. • Residual attention learning improves the performance of CNN. • A new state of the art is yielded by the proposed method on the Chest X-ray14 dataset. This paper considers the problem of multi-label thorax disease classification on chest X-ray images. Identifying one or more pathologies from a chest X-ray image is often hindered by the pathologies unrelated to the targets. In this paper, we address the above problem by proposing a category-wise residual attention learning (CRAL) framework. CRAL predicts the presence of multiple pathologies in a class-specific attentive view. It aims to suppress the obstacles of irrelevant classes by endowing small weights to the corresponding feature representation. Meanwhile, the relevant features would be strengthened by assigning larger weights. Specifically, the proposed framework consists of two modules: feature embedding module and attention learning module. The feature embedding module learns high-level features with a convolutional neural network (CNN) while the attention learning module focuses on exploring the assignment scheme of different categories. The attention module can be flexibly integrated into any feature embedding networks with end-to-end training. The comprehensive experiments are conducted on the Chest X-ray14 dataset. CRAL yields the average AUC score of 0.816 which is a new state of the art. [ABSTRACT FROM AUTHOR]

• We propose a unified algorithm to compress the CNN for IAHCCR. • We achieve 12.4 × compression and 1.7 × acceleration with only 0.17% accuracy loss. • The experiments are conducted on IAHCC-UCAS2016, ICDAR-2013, and MNIST. • The state-of-the-art performance is achieved on IAHCC-UCAS2016. Since the convolutional neural network (CNN) has brought great breakthroughs in the field of computer vision, it recently has been introduced to the in-air handwritten Chinese character recognition (IAHCCR) to achieve better recognition performance. However, the CNN is typically over-parameterized and contains lots of redundant filters or parameters. This leads the CNN to suffer from huge computation cost and considerable storage usage, limiting its deployments to resource-constrained devices like mobile phones and intelligent TVs. In this paper, we propose a unified algorithm to effectively compress the CNN for IAHCCR with little accuracy loss. Specifically, we first utilize the channel pruning strategy to simplify the network structure, and then adopt the network quantization technique to represent parameters with lower precision. We conduct experiments on the in-air handwriting dataset IAHCC-UCAS2016, where the baseline CNN achieves the state-of-the-art accuracy of 95.33% with 15.5 MB of storage. After the compression, we achieve 12.4 × storage saving and 1.7 × theoretical acceleration with only 0.17% accuracy loss. Moreover, evaluations on other benchmark datasets including ICDAR-2013 and MNIST further demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

• We propose a novel method termed SFCM to fuse CNNs features of different layers. • Our SFCM can be implemented by direct connection and residual connection. • Our SFCM can be used in many existing frameworks to achieve better performance. • We evaluate the SFCM on many different challenging computer vision tasks. Different layers of deep convolutional neural networks(CNNs) can encode different-level information. High-layer features always contain more semantic information, and low-layer features contain more detail information. However, low-layer features suffer from the background clutter and semantic ambiguity. During visual recognition, the feature combination of the low-layer and high-level features plays an important role in context modulation. If directly combining the high-layer and low-layer features, the background clutter and semantic ambiguity may be caused due to the introduction of detailed information. In this paper, we propose a general network architecture to concatenate CNN features of different layers in a simple and effective way, called Selective Feature Connection Mechanism (SFCM). Low-level features are selectively linked to high-level features with a feature selector which is generated by high-level features. The proposed connection mechanism can effectively overcome the above-mentioned drawbacks. We demonstrate the effectiveness, superiority, and universal applicability of this method on multiple challenging computer vision tasks, including image classification, scene text detection, and image-to-image translation. [ABSTRACT FROM AUTHOR]

• A novel two-stream CNN that uses audiovisual data to estimate affective dimensions. • Selection of CNN-based features using minimum redundancy and maximum relevancy rule. • Regression of selected CNN-based features outperforms the end-to-end CNN mapping. • Proposed model shows robust performance of estimation on generalization experiments. Automatic estimation of emotional state has been of great interest as emotion is an important component in user-oriented interactive technologies. This paper investigates the usage of feed-forward convolutional neural network (CNN) and features extracted from such networks for predicting dimensions of continuous-level emotional states. In this context, a two-stream CNN architecture wherein the video and audio data are learned simultaneously, is proposed. End-to-end mapping of audiovisual data to emotional dimensions reveals that the two-stream network performs better than its single-stream counterpart. The representations learned by the CNNs are refined through a minimum redundancy maximum relevance statistical selection method. Then, the support vector regression applied to selected CNN-based features estimates the instantaneous values of emotional dimensions. The proposed method is trained and tested using the audiovisual conversations of well-known RECOLA and SEMAINE databases. Experimentally it is verified that the regression of the CNN-based features outperforms the traditional audiovisual affective features as well as the end-to-end CNN mapping. Through generalization experiments, it is also observed that the learned representations are robust enough to provide an acceptable prediction performance, when the settings of training and testing datasets are widely different. [ABSTRACT FROM AUTHOR]

Highlights • We propose the use of pooling and deconvolution operations in CNNs in order to greatly increase their receptive fields. • We highlight characteristics specific to the stereo matching problem and how they can relate to the way that CNNs operate. • We use a simple feature space transformation that allow us to better evaluate the quality of the features extracted. • We present several easy to train models capable of accurate stereo matching, even without spatial regularization. Abstract Computational stereo is one of the classical problems in computer vision. Numerous algorithms and solutions have been reported in recent years focusing on developing methods for computing similarity, aggregating it to obtain spatial support and finally optimizing an energy function to find the final disparity. In this paper, we focus on the feature extraction component of stereo matching architecture and we show standard CNNs operation can be used to improve the quality of the features used to find point correspondences. Furthermore, we use a simple space aggregation that hugely simplifies the correlation learning problem, allowing us to better evaluate the quality of the features extracted. Our results on benchmark data are compelling and show promising potential even without refining the solution. [ABSTRACT FROM AUTHOR]

Highlights • We decompose sEMG image into multiple streams by the electrodes layout to model correlation between muscles and gestures. • A two-stage multi-stream CNN framework is proposed for sEMG based gesture recognition in muscle computer interface. • Comprehensive evaluations on 3 benchmark datasets shows that it outperformed the single-stream CNN for gesture recognition. Graphical abstract Abstract In muscle-computer interface (MCI), deep learning is a promising technology to build-up classifiers for recognizing gestures from surface electromyography (sEMG) signals. Motivated by the observation that a small group of muscles play significant roles in specific hand movements, we propose a multi-stream convolutional neural network (CNN) framework to improve the recognition accuracy of gestures by learning the correlation between individual muscles and specific gestures with a "divide-and-conquer" strategy. Its pipeline consists of two stages, namely the multi-stream decomposition stage and the fusion stage. During the multi-stream decomposition stage, it first decomposes the original sEMG image into equal-sized patches (streams) by the layout of electrodes on muscles, and for each stream, it independently learns representative features by a CNN. Then during the fusion stage, it fuses the features learned from all streams into a unified feature map, which is subsequently fed into a fusion network to recognize gestures. Evaluations on three benchmark sEMG databases showed that our proposed multi-stream CNN framework outperformed the state-of-the-arts on sEMG-based gesture recognition. [ABSTRACT FROM AUTHOR]

Highlights • The paper is theoretically well founded and motivated by recent research on the physiological aspects of visual recognition. • The proposed idea is simple yet effective. It provides a practical way of training CNN for cross-domain 3D shape retrieval. • High versatility, a single framework for all the three investigated tasks. • A new selected-view matching strategy based on-line learning. • The experiments are sufficient with significantly improved results. Graphical abstract Abstract Recent vision-brain physiological experiments [65] have demonstrated that contours, and in particular, contour junctions present in the 2D images are very informative for revealing the 3D structure of the object. Inspired by this observation, we take 2D sketches (or 2D views of 3D sketches) and edge maps of 2D views of 3D models as a unified domain to train the Convolutional Neural Network (CNN). The CNN features are then used for 3D object representation. We show that the CNN can successfully learn the object structure from different types of clues. The performance of the proposed method demonstrates that the semantic gap between the 2D/3D sketches and the 3D models can be bridged without any cross-domain similarity learning. Experiments show that our approach significantly outperforms the state-of-the-art 2D/3D sketch-based 3D retrieval methods. [ABSTRACT FROM AUTHOR]

Highlights • Motion blur kernel size is a significant input parameter of deblurring algorithms. • A regression CNN is proposed to estimate the kernel size of a blurry image. • The kernel size for training is labeled in a relative way. • The kernel size to be estimated can be arbitrary. • The estimated kernel size helps existing deblurring algorithms yielding satisfactory results. Abstract Deblurring is to restore a latent clear image as well as to estimate an underlying blur kernel from a single blurry image. Motion blur kernel size is a significant input parameter of existing deblurring algorithms. Setting the size manually, which is a tedious trial-and-error process, can hardly obtain a satisfactory result directly and effectively. In this paper, we formulate the kernel size estimation as a regression problem and construct a convolutional neural network (CNN) to resolve it. After training the CNN by a large number of simulated blurry images with labels, which are produced by a relative way, we can estimate the kernel sizes of a given blurry image without size limitation. Both quantitative and qualitative results of experiments show that our method can not only estimate the motion blur kernel size accurately but also help to yield better pictures with estimated kernel sizes as input parameters. [ABSTRACT FROM AUTHOR]

Highlights • An UP-CNN architecture is introduced for boosting its discriminative capability. • A ratio un-pooling operation is proposed to reconstruct activations of original size. • The cross-layer interaction mechanism is employed among different sub-networks. • It can alleviate the information attenuation in both forward and backward processes. • Experimental results well verify the effectiveness of our proposed UP-CNN. Abstract Convolutional neural network (CNN) has shown remarkable performance in various visual recognition tasks. Most of existing CNN is a purely bottom-up and feed-forward architecture, we argue that it fails to consider the interaction between low-level fine details and high-level semantic information. In this paper, a novel "Un-Pooling augmented Convolutional Neural Network" (UP-CNN) is proposed to boost the discriminative capability of the CNN with the following three distinctive properties: (1) UP-CNN is a deeper network, which is comprised of a bottom-up, a top-down and then a bottom-up sub-networks, associating to different level information that jointly improves its discriminative capability. (2) With the mixture of pooling and un-pooling layers, UP-CNN easily allows the interaction across convolutional layers with the same size from different sub-networks. This architecture effectively depresses the attenuation of important information including both activations in the forward process and gradient information in the back-propagation process. (3) UP-CNN employs the ratio un-pooling operation to reconstruct activations of the original size in the top-down sub-network, where the spatial information that is lost during pooling can be preserved within a receptive field. The experiments on four benchmark datasets (including the CIFAR-10, CIFAR-100, MNIST and SVHN datasets) well demonstrate that the proposed UP-CNN architecture considerably outperforms other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Highlights • It is for the first time the pure binarized convolutional layers are incorporated into Gabor CNNs. • HGCNs can reduce the size of required storage space of any given convolutional filters by a factor of 32. • HGCNs can achieve the comparable performance to the full-precision model. Abstract Despite great effectiveness of very wide and deep convolutional neural networks (DCNNs) in various computer vision tasks, the significant cost in terms of storage requirement of such networks impedes the deployment on computationally limited devices. Therefore, the network of resource and accuracy trade-offs always have been paid much attention in the popular DCNNs. A new deep model, Hybrid Gabor Convolutional Networks (HGCNs), is proposed in this paper. Incorporating binarized feature map and binarized filters into DCNNs in alternating way, HGCNs alleviate the accuracy loss while reduce the memory storage by 32. Also, HGCNs can be easily implemented and compatible with any other popular deep learning architecture by manipulating the basic structure elements of the DCNNs (i. e. , the convolution operator) based on the binarization filter. Most importantly, HGCNs achieve a comparable performance while largely reducing the storage memory compared with state-of-the-art networks such as Resnet. The source code will be public soon. [ABSTRACT FROM AUTHOR]

Highlights • We propose a personality prediction method based on image scene perception. • Users with the same personality have similar preferences for image scenes. • A number of image scenes are recognized to solve a user-centric prediction problem. Graphical abstract Abstract People tend to have different preferences for image scenes, which are indicative of their personality traits. Inspired by the relationship between users' attributed personality traits and preferred image scenes, this letter proposes a personality prediction method based on image scene perception probability. Firstly, image scenes are recognized based on the convolutional neural network. Secondly, the probability distribution of perceived image scenes is calculated. Finally, a regression model is trained using the scene probability distribution features to predict the attributed personality traits. The experimental results on the annotated PshycoFlickr dataset show that the proposed method is superior to the state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Highlights • We propose a novel two-stream ConvNet model for first person action recognition. • Our model effectively captures the temporal structure of actions. • We introduce novel long-term pooling operators for appearance and motion information. • We analyze the effect of the proposed architecture with extensive experiments. • We achieve the state-of-the-art performance for first person action recognition. Abstract First person action recognition is an active research area with increasingly popular wearable devices. Action classification for first person video (FPV) is more challenging than conventional action classification due to strong egocentric motions, frequent changes of viewpoints, and diverse global motion patterns. To tackle these challenges, we introduce a two-stream convolutional neural network that improves action recognition via long-term fusion pooling operators. The proposed method effectively captures the temporal structure of actions by leveraging a series of frame-wise features of both appearance and motion in actions. Our experiments validate the effect of the feature pooling operators, and show that the proposed method achieves state-of-the-art performance on standard action datasets. [ABSTRACT FROM AUTHOR]

Depth information has been discovered to improve the performance of computer vision tasks, such as semantic segmentation and object recognition. However, careful acquisition of depth data needs highly developed depth sensors which are expensive. As a classic computer vision task, depth estimation from a single image has obtained promising results based on supervised learning methods. In this paper, we investigate the extension of color images with corresponding deep-regressed depth images in boosting the performance of semantic segmentation. Furthermore, the usage of combining color channels with the estimated depth or the ground truth depth channel is compared. Specifically, there are two stages in our work. Firstly, we adopt the framework of convolutional neural networks (CNN) for the depth estimation by combing the global depth network and the depth gradient network. After refining based on these two networks, the depth image map can be estimated in a deep-regressed manner. Secondly, after augmenting the color images with the predicted depth images, fully convolutional networks (FCN) are further used to implement the pixel-level semantic labeling. In the experiments, we employ two popular RGBD datasets, i.e., SUNRGBD and NYUDv2, for 37 and 40-class semantic segmentation, respectively. By comparing with the ground truth depth images, experimental results demonstrate that the networks trained on the estimated depth images can achieve comparable performance on facilitating the accuracy of semantic segmentation task. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks have made great progress in recent handwritten character recognition (HCR) by learning discriminative features from large amounts of labeled data. However, the large variance of handwriting styles across writers is still a big challenge to the robust HCR. To alleviate this issue, an intuitional idea is to extract writer-independent semantic features from handwritten characters, while standard printed characters are writer-independent stencils for handwritten characters. They could be used as prior knowledge to guide models to exploit writer-independent semantic features for HCR. In this paper, we propose a novel adversarial feature learning (AFL) model to incorporate the prior knowledge of printed data and writer-independent semantic features to improve the performance of HCR on limited training data. Different from available handcrafted features methods, the proposed AFL model exploits writer-independent semantic features automatically, and standard printed data as prior knowledge is learnt objectively. Systematic experiments on MNIST and CASIA–HWDB show that the proposed model is competitive with the state-of-the-art methods on the offline HCR task. [ABSTRACT FROM AUTHOR]

In this work, a novel deep learning technique for the recognition of handwritten Bangla isolated compound character is presented and a new benchmarkof recognition accuracy on the CMATERdb 3.1.3.3 dataset is reported. Greedy layer wise training of Deep Neural Network has helped to make significant strides in various pattern recognition problems. We employ layerwise training to Deep Convolutional Neural Networks (DCNN) in a supervised fashion and augment the training process with the RMSProp algorithm to achieve faster convergence. We compare results with those obtained from standard shallow learning methods with predefined features, as well as standard DCNNs. Supervised layerwise trained DCNNs are found to outperform standard shallow learning models such as Support Vector Machines as well as regular DCNNs of similar architecture by achieving error rate of 9.67% thereby setting a new benchmark on the CMATERdb 3.1.3.3 with recognition accuracy of 90.33%, representing an improvement of nearly 10%. [ABSTRACT FROM AUTHOR]

Extracting high level information from digital images and videos is a hard problem frequently faced by the computer vision and machine learning communities. Modern surveillance systems can monitor people, cars or objects by using computer vision methods. The objective of this work is to propose a method for identifying soft-biometrics, in the form of clothing and gender, from images containing people, as a previous step for further identifying people themselves. We propose a solution to this classification problem using a Convolutional Neural Network, working as an all-in-one feature extractor and classifier. This method allows the development of a high-level end-to-end clothing/gender classifier. Experiments were done comparing the CNN with hand-designed classifiers. Also, two different operating modes of CNN are proposed and compared each other. The results obtained were very promising, showing that is possible to extract soft-biometrics attributes using an end-to-end CNN classifier. The proposed method achieved a good generalization capability, classifying the three different attributes with good accuracy. This suggests the possibility to search images using soft biometrics as search terms. [ABSTRACT FROM AUTHOR]