Search

In this paper, a three-step model based on the integration of Deep Neural Networks (DNN) and Decision Models is introduced for image classification which is inspired by the human visual system. To make a decision about an object, many actions should be done in a hierarchical process in the brain. First, the retina receives visual stimuli and transfers them to the visual cortex in the brain. The information extracted in the visual cortex, is accumulated over time to select an appropriate response. Many of the current decision-making models do not show how each image is converted into useful information for the decision model. Some models have used neural networks to convert each image into the information needed in the decision-making model; however, the role of the retina is ignored among these models. In this paper, a combination of retina inspired filters, CNN-based description and accumulator-based decision model is used to classify images. This model's structure resembles the human brain due to the usage of the DoG filter bank as retina inspired filter in the first stage of it. This model shows a significant improvement in accuracy in comparison to other models; furthermore, its performance is acceptable even with the small sample training set. [ABSTRACT FROM AUTHOR]

Pruning is a useful technique for decreasing the memory consumption and floating point operations (FLOPs) of deep convolutional neural network (CNN) models. Nevertheless, at modest pruning levels, current structured pruning approaches often lead to considerable declines in accuracy. Furthermore, existing approaches often treat pruning rates as super parameters, neglecting the sensitivity of different convolution layers. In this study, we propose a novel sensitivity-based method for channel pruning that utilizes second-order sensitivity as a criterion. The essential concept is to prune insensitive filters while retaining sensitive ones. We quantify the sensitivity of the filter using the sum of the sensitivities of all weights in the filter, rather than the magnitude-based metric frequently applied in the literature. Furthermore, a layer sensitivity approach based on the Hessian eigenvalues of each layer is introduced into the process of automatically choosing the most appropriate pruning rate for each layer. Experiments on a variety of modern CNN architectures demonstrate that we can considerably enhance the pruning rate while sacrificing a small amount of accuracy, resulting in a reduction of more than 60% in FLOPs on CIFAR-10. Notably, on ImageNet, pruning based on ResNet50 decreased the FLOPs by 56.3% while losing only 0.92% of accuracy. [ABSTRACT FROM AUTHOR]

In personal identity recognition systems, detecting a person's age, gender, and language using voice signal characteristics is a crucial issue, especially because of the importance of security considerations. Age, gender, and language classification problems are important in signal processing because they are used to analyze and understand human behavior, interactions, and preferences. This can be especially useful in the fields of human-computer interaction, psychology, and social science research. In this paper, a new system for detecting a speaker's age, gender, and language based on deep learning models is presented. Deep learning models have shown great efficacy in various fields of signal processing. For this paper, a range of deep models were tested, including convolutional neural networks (CNNs), recurrent neural network (RNN), and a fine-tuning ResNet34 architecture. Additionally, techniques such as transfer learning were applied to improve the efficiency of the proposed system. The input voice signals are preprocessed by applying the spectro-temporal transform to obtain additional features that can be fed to the ResNet34 model, which is designed specifically for the task of voice signal processing. The dataset used in this paper was sourced from the Mozilla common voice initiative, which is dedicated to advancing speech recognition and language identification technologies. The performance of the proposed algorithm was evaluated in the presence of Gaussian noise to determine its robustness. The experimental results demonstrated that the proposed algorithm significantly outperformed basic algorithms and other deep neural networks in terms of age and gender recognition from voice signals. [ABSTRACT FROM AUTHOR]

• Neurobiological-inspired CNN models accurately predict sound location. • Models of binaural integration in human brainstem simulate human sound localization. • Objective functions minimizing angular or Euclidean distance lead to distinct errors. • Feature representations of shallow layers resemble brainstem neuronal spatial tuning. • Neurobiological-inspired CNN models generate testable predictions for neuroscience. The human brain effortlessly solves the complex computational task of sound localization using a mixture of spatial cues. How the brain performs this task in naturalistic listening environments (e.g. with reverberation) is not well understood. In the present paper, we build on the success of deep neural networks at solving complex and high-dimensional problems [1] to develop goal-driven, neurobiological-inspired convolutional neural network (CNN) models of human spatial hearing. After training, we visualize and quantify feature representations in intermediate layers to gain insights into the representational mechanisms underlying sound location encoding in CNNs. Our results show that neurobiological-inspired CNN models trained on real-life sounds spatialized with human binaural hearing characteristics can accurately predict sound location in the horizontal plane. CNN localization acuity across the azimuth resembles human sound localization acuity, but CNN models outperform human sound localization in the back. Training models with different objective functions - that is, minimizing either Euclidean or angular distance - modulates localization acuity in particular ways. Moreover, different implementations of binaural integration result in unique patterns of localization errors that resemble behavioral observations in humans. Finally, feature representations reveal a gradient of spatial selectivity across network layers, starting with broad spatial representations in early layers and progressing to sparse, highly selective spatial representations in deeper layers. In sum, our results show that neurobiological-inspired CNNs are a valid approach to modeling human spatial hearing. This work paves the way for future studies combining neural network models with empirical measurements of neural activity to unravel the complex computational mechanisms underlying neural sound location encoding in the human auditory pathway. [ABSTRACT FROM AUTHOR]

• Developing a new recursive convolutional layer (RCL) able to stop the iteration when the hidden state stabilize. • First network to have different depth for different batch of samples. • A comprehensive study on the best locations and numbers of RCL in a multilayer network called C-FRPN. To overcome problems with the design of large networks, particularly with respect to the depth of the network, this paper presents a new model of convolutional neural networks (CNN) which features fully recursive convolutional layers (RCLs). An RCL is a generalization of the classic one-stage feedforward convolutional layer (CL) to fully direct feedback connections between the outputs of the CL and its inputs. A traditional deep CNN consisting of many CLs, can then be generalized to include some CLs, and some RCLs in the intermediate stages. We call the corresponding network a Convolutional Neural Network with Fully Recursive Perceptron Network (C-FRPN). Through an analysis of results obtained from applications of the C-FRPN to three benchmark image classification datasets: CIFAR-10, SVHN, ISIC, it is found that (i) in general, the performance of a C-FRPN, even with only one RCL, is better than the performance of the corresponding deep CNN with all CLs, under the constraint of having the same number of unknown parameters; (ii) the performance of the C-FRPN varies with respect to (a) where the RCLs are located, and (b) the number of RCLs in the C-FRPN; and, (iii) the effectiveness of the RCLs depends on the size of the training dataset. The results suggest that: (a) it is advisable to use RCLs particularly when training very large sets of data, (b) it is best to prioritize placement of RCLs close to the input layer of the C-FRPN, and (c) it is advisable to increase the number of RCLs as long as the training dataset can sustain without overfitting being observed. [ABSTRACT FROM AUTHOR]

Despite the superiority of convolutional neural networks demonstrated in time series modeling and forecasting, it has not been fully explored on the design of the neural network architecture and the tuning of the hyper-parameters. Inspired by the incremental construction strategy for building a random multilayer perceptron, we propose a novel Error-feedback Stochastic Modeling (ESM) strategy to construct a random Convolutional Neural Network (ESM-CNN) for time series forecasting task, which builds the network architecture adaptively. The ESM strategy suggests that random filters and neurons of the error-feedback fully connected layer are incrementally added to steadily compensate the prediction error during the construction process, and then a filter selection strategy is introduced to enable ESM-CNN to extract the different size of temporal features, providing helpful information at each iterative process for the prediction. The performance of ESM-CNN is justified on its prediction accuracy of one-step-ahead and multi-step-ahead forecasting tasks respectively. Comprehensive experiments on both the synthetic and real-world datasets show that the proposed ESM-CNN not only outperforms the state-of-art random neural networks, but also exhibits stronger predictive power and less computing overhead in comparison to trained state-of-art deep neural network models. [ABSTRACT FROM AUTHOR]

• Developing a new recursive convolutional layer (RCL) able to stop the iteration when the hidden state stabilize. • First network to have different depth for different batch of samples. • A comprehensive study on the best locations and numbers of RCL in a multilayer network called C-FRPN. To overcome problems with the design of large networks, particularly with respect to the depth of the network, this paper presents a new model of convolutional neural networks (CNN) which features fully recursive convolutional layers (RCLs). An RCL is a generalization of the classic one-stage feedforward convolutional layer (CL) to fully direct feedback connections between the outputs of the CL and its inputs. A traditional deep CNN consisting of many CLs, can then be generalized to include some CLs, and some RCLs in the intermediate stages. We call the corresponding network a Convolutional Neural Network with Fully Recursive Perceptron Network (C-FRPN). Through an analysis of results obtained from applications of the C-FRPN to three benchmark image classification datasets: CIFAR-10, SVHN, ISIC, it is found that (i) in general, the performance of a C-FRPN, even with only one RCL, is better than the performance of the corresponding deep CNN with all CLs, under the constraint of having the same number of unknown parameters; (ii) the performance of the C-FRPN varies with respect to (a) where the RCLs are located, and (b) the number of RCLs in the C-FRPN; and, (iii) the effectiveness of the RCLs depends on the size of the training dataset. The results suggest that: (a) it is advisable to use RCLs particularly when training very large sets of data, (b) it is best to prioritize placement of RCLs close to the input layer of the C-FRPN, and (c) it is advisable to increase the number of RCLs as long as the training dataset can sustain without overfitting being observed. [ABSTRACT FROM AUTHOR]

• HELP improves the efficiency of hyperparameter tunning with random exploration. • HELP avoids bad hyperparameters in future exploration by learning with an RNN. • HELP finds the optimal hyperparameters at iteration by information during training. Hyperparameter selection is very important for the success of deep neural network training. Random search of hyperparameters for deep neural networks may take a long time to converge and yield good results because the training of deep neural networks with a huge number of parameters for every selected hyperparameter is very time-consuming. In this work, we propose the Hyperparameter Exploration LSTM-Predictor (HELP) which is an improved random exploring method using a probability-based exploration with an LSTM-based prediction. The HELP has a higher probability to find a better hyperparameter with less time. The HELP uses a series of hyperparameters in a time period as input and predicts the fitness values of these hyperparameters. Then, exploration directions in the hyperparameter space yielding higher fitness values will have higher probabilities to be explored in the next turn. Experimental results for training both the Generative Adversarial Net and the Convolution Neural Network show that the HELP finds hyperparameters yielding better results and converges faster. [ABSTRACT FROM AUTHOR]

Deep neural networks (DNN), such as convolutional neural networks (CNN) have been widely used for object recognition. However, they are usually unable to ensure the required intra-class compactness and inter-class separability in the kernel space. These are known to be important in pattern recognition for achieving both robustness and accuracy. In this paper, we propose to integrate a kernelized Min-Max objective in the DNN training in order to explicitly enforce both kernelized within-class compactness and between-class margin. The involved kernel space is implicitly mapped from the feature space associated with a certain upper layer of DNN by exploiting a kernel trick, while the Min-Max objective in this space is interpolated with the original DNN loss function and finally optimized in the training phase. With a very small additional computation cost, the proposed strategy can be easily integrated in different DNN models without changing any other part of the original model. The comparative recognition accuracy of the proposed method is evaluated with multiple DNN models (including shallow CNN, deep CNN and deep residual neural network models) on two benchmark datasets: CIFAR-10 and CIFAR-100. Extensive experimental results demonstrate that the integration of kernelized Min-Max objective in the training of DNN models can achieve better results compared to state-of-the-art models, without incurring additional model complexity. [ABSTRACT FROM AUTHOR]

Hashing has been widely used for large-scale multimedia retrieval because of its advantages in storage and retrieval efficiency. Traditional supervised hash methods represent an image as a feature vector and then perform a separate quantization step to generate a binary code. Due to the difficulty of discrete optimization of hash codes, continuous relaxation is generally used to replace discrete optimization. However, the process of continuous relaxation leads to inevitable quantization error. To avoid this drawback, a deep balanced discrete hashing method is proposed, which uses discrete gradient propagation with the straight-through estimator. The proposed method does not use the traditional continuous relaxation strategy, thereby reducing the quantization error caused by continuous relaxation. And the proposed method uses supervised information to directly guide the discrete coding and deep feature learning process. In the proposed method, the last layer of the Convolutional Neural Network (CNN) outputs the binary code directly. In the loss function, discrete values are calculated by combining the pairwise loss and a balance controlling term. The learned binary hash code maintains the similar relationship and label consistency at the same time. While maintaining the pairwise similarity, the proposed method keeps the balance of hash codes to improve retrieval performance. Extensive experiments show that the proposed method outperforms the state-of-the-art hashing methods on four image retrieval benchmark datasets. [ABSTRACT FROM AUTHOR]

The deep neural network model based on spatial pyramid pooling for multi-scale context information has been widely adopted in image semantic segmentation. However, this structure reduces the feature resolution during the inference process, which should be recovered by bilinear interpolation. We find that the performance of boundary location heavily depends on the interpolation operation. Assuming that the foreground object has a boundary with uneven curvature changes, it will be difficult to preserve details in the lower resolution feature map. In this case, if the model achieves near-optimal fitting, the prediction of the boundary point needs to satisfy strict constraints. But this condition is very difficult to meet for the model. For this purpose, we propose a constraint balance factor (CBF) relaxation method to improve the fitting ability of the model. The constraint balance factor is learned end-to-end by a sub-network. This sub-network can be easily embedded into a semantic segmentation network that requires interpolation without too many extra parameter increments. The experimental results illustrate that the proposed CBFNet performs well on PASCAL VOC 2012 and Cityscapes datasets. [ABSTRACT FROM AUTHOR]

Visual cortex is able to process information in multiple pathways and integrate various forms of representations. This paper proposed a bio-inspired method that utilizes the line-segment-based representation to perform a dedicated channel for the geometric feature learning process. The extracted geometric information can be integrated with the original pixel-based information and implemented on both the convolutional neural networks (SegCNN) and the stacked autoencoders (SegSAE). Segment-based operations such as segConvolve and segPooling are designed to further process the extracted geometric features. The proposed models are verified on the MNIST dataset, Caltech 101 dataset and QuickDraw dataset for image classification. According to the experimental results, the proposed models can facilitate the classification accuracies especially when the sizes of the training set are limited. Particularly, the method based on multiple representations is found to be effective for classifying the hand-drawn sketches. [ABSTRACT FROM AUTHOR]

Single-image super-resolution (SISR) has been an important topic due to the demand for high-quality virtual images in the field of visual artificial intelligence. Methods based on deep learning have achieved great success based on the excellent capability of grasping complicated features of deep convolutional networks. The performance can be improved slightly but not obviously by simply widening or deepening the network. In this paper, we propose a merged convolutional network for super-resolution, which extracts more adequate details to restore high-resolution images. We used dense blocks for feature extraction to concatenate deep features with shallow features in depth. We also designed two sub-nets with distinct convolution kernels as different branches of the network, which can widen the network and improve the performance of the system. Finally, we employed sub-pixel layers to avoid feature distortion for up-sampling at the very end. Our method was evaluated using several standard benchmark datasets. The results demonstrate superior performance and good robustness compared with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Video-based crowd counting can leverage the spatial-temporal information between neighboring frames, and thus this information would improve the robustness of crowd counting. Therefore, this solution is more practical than single image-based crowd counting in real applications. Since severe occlusions, translation, rotation, and scaling of persons will give rise to the change of density map of heads between neighboring frames, video-based crowd counting is a very challenging task. To alleviate these issues in video crowd counting, a Multi-Level Feature Fusion Based Locality-Constrained Spatial Transformer Network (MLSTN) is proposed, which consists of two components, namely density map regression module and Locality-Constrained Spatial Transformer (LST) module. Specifically, we first estimate the density map of each frame by utilizing the combination of the low-level, middle-level and high-level features of the Convolutional Neural Networks. This is because the low-level features may be more effective in the extraction of small head information, while the middle and high level features are more effective in the extraction of medium and large head information. Then to measure the relationship of the density maps between neighboring frames, the LST module is proposed, which estimates the density map of the next frame by concatenating several regression density maps. To facilitate the performance evaluation for video crowd counting, we have collected and labeled a large-scale video crowd counting dataset which includes 100 five-second-long sequences with 394,081 annotated heads from 13 different scenes. As far as we know, it is the largest video crowd counting dataset. Extensive experiments show the effectiveness of our proposed approach for crowd counting on our dataset and other video-based crowd counting datasets. All our dataset are released online. 1 1 https://github.com/sweetyy83/Lstn_fdst_dataset. [ABSTRACT FROM AUTHOR]

MRI is the leading method of evaluation in traumatic shoulder pathologies ranging from soft tissue or bone edema to rupture of tendons or ligaments and subtle fractures of bone. MRI has the power of evaluating both of the soft tissues and bone at the same time. PD weighted MRI sequences are very powerful in demonstrating bone trauma in terms of edema and cortical disruption in the shoulder pathologies. The alterations in the intensity of the metaphyseal bone may be used to predict the presence of the bone trauma or pathologies. Although it is superior to many other imaging modalities to uncover the effects of trauma on the each anatomical structure of shoulder, PD weighted MRI has innate obstacles to image processing like low signal to noise ratio besides close anatomical relations and unclear bony borders. We proposed an innovative representation of a learning framework for diagnosis of bone lesions from PD MRI sequences that integrates deep learning techniques to automatically learn discriminative features while avoiding the design of specific hand crafted image based feature descriptors. In this study automatically segmented PD weighted shoulder images were evaluated by the proposed convolutional neural network (CNN) to extract features and classify humeral head in three groups as normal, edematous and Hill -Sachs lesion with a success rate of 98.43%. Compared to the state of art methods, our proposed CNN based diagnosis system is very promising to assist radiologists and orthopedists in decision making. [ABSTRACT FROM AUTHOR]

High-resolution (HR) magnetic resonance (MR) imaging is an important diagnostic technique in clinical practice. However, hardware limitations and time constraints often result in the acquisition of anisotropic MR images. It is highly desirable but very challenging to enhance image spatial resolution in medical image analysis for disease diagnosis. Recently, studies have shown that deep convolutional neural networks (CNN) can significantly boost the performance of MR image super-resolution (SR) reconstruction. In this paper, we present a novel CNN-based anisotropic MR image reconstruction method based on residual learning with long and short skip connections. The proposed network can effectively alleviate the vanishing gradient problem of deep networks and learn to restore high-frequency details of MR images. To reduce computational complexity and memory usage, the proposed network utilizes cross-plane self-similarity of 3D T1-weighted (T1w) MR images. Based on experiments on simulated and clinical brain MR images, we demonstrate that the proposed network can significantly improve the spatial resolution of anisotropic MR images with high computational efficiency. The network trained on T1w MR images is able to effectively reconstruct both SR T1w and T2-weighted (T2w) images, exploiting image features for multi-modality reconstruction. Moreover, the experimental results show that the proposed method outperforms classical interpolation methods, non-local means method (NLM), and sparse coding based algorithm in terms of peak signal-to-noise-ratio, structural similarity image index, intensity profile, and small structures. The proposed method can be efficiently applied to SR reconstruction of thick-slice MR images in the out-of-plane views for radiological assessment and post-acquisition processing. [ABSTRACT FROM AUTHOR]

Automatic heart sound auscultation is one of the common used techniques for cardiovascular diseases detection. In this paper, a novel heart sound classification method based on deep learning technologies for cardiovascular disease prediction is introduced, which is mainly comprised three parts: pre-processing, 1-D waveform heart sound patches classification using a deep convolutional neural network (CNN) with attention mechanism, and majority voting for final prediction of heart sound recordings. In order to enhance the information flow of the CNNs, a block-stacked style architecture with clique blocks is employed, and in each clique block a bidirectional connection structure is introduced in the proposed CNN. By using the stacked clique and transition blocks, the proposed CNN achieves both spatial and channel attention leading a promising classification performance. Moreover, a novel separable convolution with inverted bottleneck is utilized to decouple the spatial and channel-wise relevancy of features efficiently. Experiments on PhysioNet/CinC 2016 show that the proposed method obtains a superior classification results and excels in consumption of parameter comparing to state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Extracting brain parenchyma from computed tomography (CT) images of the head is an important prerequisite step in a number of image processing applications, as it improves the computational speed and accuracy of quantitative analyses and image co-registration. In this study, we present a robust method based on fully convolutional neural networks (CNN) to remove non-brain tissues from head CT scans in a computationally efficient manner. The method includes an encoding part, which has sequential convolutional filters that produce feature representation of the input image in low dimensional space, and a decoding part, which consists of convolutional filters that reconstruct the input image from the reduced representation. We trained several CNN models on 122 volumetric head CT scans and tested our models on 22 withheld volumetric CT head scans based on two experts' manual brain segmentation. The performance of our best CNN model on the test set is: Dice Coefficient = 0.998 ± 0.001 (mean ± standard deviation), recall = 0.999 ± 0.001, precision = 0.998 ± 0.001, and accuracy = 1. Our method extracts complete volumetric brain from head CT images in about 2 s which is substantially faster than currently available methods. To the best of our knowledge, this is the first study using CNN to perform brain extraction from CT images. In conclusion, the proposed approach based on CNN provides accurate extraction of brain tissue from head CT images in a computationally efficient manner. [ABSTRACT FROM AUTHOR]

Early diagnosis of breast cancer, when it is small and has not spread, can make the disease easier to treat which increases the patient's chances of survival. The recent proposed methods for the early diagnosis of breast cancer, and while showing great success in achieving this goal, rely on one of the indicators in the mammogram to diagnose the patient's condition. Whether it is identifying differences in shapes and patterns of the findings (i.e. masses, calcifications,...etc.) or assessing the breast density as a risk indicator, these Computer-aided Diagnosis (CAD) systems by using single-label classification, fail to exploit the intrinsic useful correlation information among data from correlated domains. Rather than learning to identify the disease based on one of the indicators, we propose the joint learning of the tasks using multi-label image classification. Furthermore, we introduce a new fine-tuning strategy for using transfer learning, that takes advantage of the end-to-end image representation learning when adapting the pre-trained Convolutional Neural Network (CNN) to the new task. We also propose a customized label decision scheme, adapted to this problem, which estimates the optimal confidence for each visual concept. We demonstrate the effectiveness of our approach on four benchmark datasets, CBIS-DDSM, BCDR, INBreast and MIAS, obtaining better results compared to other commonly used baselines. [ABSTRACT FROM AUTHOR]

Deep learning technologies have played more and more important roles in Computer Aided Diagnosis (CAD) in medicine. In this paper, we tackled the problem of automatic prediction of Alzheimer's Disease (AD) based on Magnetic Resonance Imaging (MRI) images, and propose a fully unsupervised deep learning technology for AD diagnosis. We first implement the unsupervised Convolutional Neural Networks (CNNs) for feature extraction, and then utilize the unsupervised predictor to achieve the final diagnosis. In the proposed method, two kinds of data forms, one slice and three orthogonal panels (TOP) of MRI image, are employed as the input data respectively. Experimental results run on all the 1075 subjects in database of the Alzheimer's Disease Neuroimaging Initiative (ADNI 1 1.5T) show that the proposed method with one slice data yields the promising prediction results for AD vs. MCI (accuracy 95.52%) and MCI vs. NC (accuracy 90.63%), and the proposed methods with TOP data yields the best overall prediction results for AD vs. MCI (accuracy 97.01%) and MCI vs. NC (accuracy 92.6%). [ABSTRACT FROM AUTHOR]

In order to make financial investment more stable and more profitable, convolutional neural network (CNN) and deep learning technology are used to quantify financial investment, so as to obtain more robust investment and returns. With the continuous development of in-depth learning technology, people are applying it more and more widely. Deep learning is put forward on the basis of neural network. It contains more hidden layers, shows more powerful learning ability, and can abstract data at a higher level, so as to obtain more accurate data. CNN is a multi-layer network structure which simulates the operation mechanism of biological vision system. Its special structure can obtain more useful feature descriptions from original data and is very effective in extracting data. Therefore, in this study, the two technologies are combined to quantify financial investment. The results show that the convolution neural network and deep learning algorithm can obtain relatively accurate investment strategies, thus ensuring investment returns and reducing investment risks. [ABSTRACT FROM AUTHOR]

Autonomous vehicles have become a hot spot of the automotive industry, many cities have claimed that autonomous vehicles should be capable of recognizing gestures used by traffic police. Traditional traffic police gesture recognition methods rely on depth-sensor or wearable-devices, which limits their availability in the domain of the intelligent vehicle. Vision-based methods have fewer requirements for distance, but the modeling process is challenging due to the complexity of the visual scenes. Inspired by the recent success in vision-based pose estimation networks such as Convolutional Pose Machine (CPM), in this paper, we propose a novel vision-based human-machine interface to recognize eight kinds of Chinese traffic police gestures and apply it in the real-time recognition tasks. This method integrates a modified CPM network and two kinds of handcrafted features: Relative Bone Length and Angle with Gravity as spatial domain features, and adopt a Long short-term memory (LSTM) network to extract temporal domain features. To train and validate our method, we create a gestures dataset with two hours of traffic police gesture videos, which has 3354 gesture instances. The experiment results show that the proposed method is capable of recognizing traffic police gestures, and is fast enough for online gesture prediction. [ABSTRACT FROM AUTHOR]

This paper proposes a new haze layer based single image dehazing algorithm. The residual images, which exists between the hazy images and the clear images, will be firstly obtained by haze layers through an end-to-end mapping from the original hazy images. Then the designed convolutional neural network can remove the residual image from the given hazy image to obtain a recovered dehazed image. In order to remove the halos and block artefacts in the dehazed images, a guided filter is applied before generating the final dehazed images, which will also increase the reality of the result images. Since the proposed haze layer based dehazing algorithm directly learns the residual images, it enjoys a relatively high learning rate and low computation. We tested and compared the proposed algorithm with other dehazing methods in various variables, such as the density of the fog and the testing scenes, e.g. real-world images or synthetic images. Experimental results showed the advancement of the proposed method in many metrics. [ABSTRACT FROM AUTHOR]

Applying machine learning methods to various modality medical images and clinical data for early diagnosis of Alzheimer's disease (AD) and its prodromal stage has many significant results. So far, the image data input to classifier mainly focus on 2D or 3D images. Although some functional imaging technologies, such as functional magnetic resonance imaging (fMRI), generate 4D data which contain both spatial and time-varying information of the brain, for the lack of suitable 4D image processing algorithm, these 4D data were always used by transforming them into functional connectivity or slicing them into 2D/3D pictures which causing apparent information loss. In this paper, we present a 4D deep learning model (C3d-LSTM) for AD discrimination, which is able to utilize the spatial and time-varying information simultaneously by dealing with 4D fMRI data directly. The proposed C3d-LSTM combines a series of 3D convolutional neural networks (CNNs) to extract spatial features from each volume of 3D static image in fMRI image sequence. Then the feature maps obtained were put into the long short-term memory (LSTM) network to capture the time-varying information contained in the data. Because of the design of structure, C3d-LSTM became an end-to-end data-driven model, which was more convenient for processing 4D fMRI data. The model proposed conducted on the AD Neuroimaging Initiative dataset for algorithm evaluation compared with controlled experiments. Results showed that using 4D fMRI data directly with the proposed method did make a far better result for AD detection than the methods using functional connectivity, 2D, or 3D fMRI data. It demonstrated our assumption that making the most of the natural spatial and temporal information preserved in 4D fMRI data is significant for AD detection and can increase the performance of the classifier. Meanwhile the result implied that the C3d-LSTM model proposed is a suitable model for processing 4D fMRI data and extracting the spatio-temporal property of fMRI data fully for diagnosis of AD. [ABSTRACT FROM AUTHOR]

Video-based convolutional neural networks (CNNs) involve a large amount of training parameters, leading to the enormous computational complexity, which thereby delays the network convergence. Therefore, training successful CNN for action recognition rapidly is non-trivial. In this paper, a novel encoding called nonlinear gated channels unit (NGCU) is proposed to facilitate network training by encoding the global channel-level relationship. Based on this, nonlinear gated channels network (NGCN) is constructed for the end-to-end encoding, and the corresponding convergence performance is evaluated on the standard benchmarks UCF101 and HMDB51. Experimental results demonstrate that the proposed method is conducive to the convergence process of CNN based action recognition models. [ABSTRACT FROM AUTHOR]

Unlike natural image captured with cameras, screen content image (SCI) is a composite image including textual and pictorial regions. The different characteristics lead to many difficulties in image quality assessment (IQA). Most existing models based on the convolutional neural network (CNN) divide large SCIs into image patches to increase training samples for CNN training. This brings two problems: (1) a single image patch can not represent the quality of the entire image, especially in IQA of SCI; (2) SCI patches of an entire image degraded by the same distortion type and strength may have drastically different quality. In addition, these models adopt the mean square error (MSE) between the predicted quality and the subjective differential mean opinion score (DMOS) to train the CNN, without considering quality ranking between different SCIs. In this paper, we propose a novel no-reference (NR) IQA model based on the convolutional neural network (CNN). The contributions of our algorithm can be concluded as follows: (1) considering a large difference exists in different regions in a SCI, the pseudo global features generated with multi-region local features are utilized for quality evaluation, which better reflect image quality than local features of each image patch; (2) the noise classification task is used as an auxiliary task which aids the quality score prediction task to improve the representation ability; (3) the Siamese networks are used to predict the quality scores of two different SCIs, and a new ranking loss is proposed to rank the predicted scores, aiming to enhance the ability of the model to rank image in terms of quality. Experimental results verify that our model outperforms all test NR IQA methods and full-reference (FR) IQA methods on the screen content image quality assessment database (SIQAD). [ABSTRACT FROM AUTHOR]

• Unsupervised framework for depth estimation and camera motion prediction. • Depth CNN and pose CNN are trained jointly and can be used respectively. • The supervision signal with a left-right consistency is constructed by spatial and temporal geometry constraints. • Results outperform previous unsupervised methods and some supervised methods. • A model which is trained on the Euroc dataset is used to test the algorithm's generalization capability. Depth estimation from monocular video plays a crucial role in scene perception. The significant drawback of supervised learning models is the need for vast amounts of manually labeled data (ground truth) for training. To overcome this limitation, unsupervised learning strategies without the requirement for ground truth have achieved extensive attention from researchers in the past few years. This paper presents a novel unsupervised framework for estimating single-view depth and predicting camera motion jointly. Stereo image sequences are used to train the model while monocular images are required for inference. The presented framework is composed of two CNNs (depth CNN and pose CNN) which are trained concurrently and tested independently. The objective function is constructed on the basis of the epipolar geometry constraints between stereo image sequences. To improve the accuracy of the model, a left-right consistency loss is added to the objective function. The use of stereo image sequences enables us to utilize both spatial information between stereo images and temporal photometric warp error from image sequences. Experimental results on the KITTI and Cityscapes datasets show that our model not only outperforms prior unsupervised approaches but also achieving better results comparable with several supervised methods. Moreover, we also train our model on the Euroc dataset which is captured in an indoor environment. Experiments in indoor and outdoor scenes are conducted to test the generalization capability of the model. [ABSTRACT FROM AUTHOR]

• We propose a deep convolutional neural network for improving the quality of the segmentation WMHs. • This network consists of two subnets which uses several architectural innovations to provide very accurate segmentation. • We also propose a novel loss function based on lesion similarity and background similarity. • Our network outperforms current segmentation methods on two public segmentation challenges. White matter hyperintensities (WMHs) appear as regions of abnormally signal intensity on magnetic resonance imaging (MRI) images, that can be identified in MRI images of elderly people and ischemic stroke patients. However, manual segmentation and quantification of images with WMHs is laborious and time-consuming. Moreover, ischemic stroke lesion and WMHs appear as similar signals in MRI images, making it difficult to accurately segment the WMHs. Analysis of WMH-containing images is important for clinical diagnosis, and thus several segmentation methods have been proposed. However, these methods cannot accurately differentiate WMH and ischemic stroke lesions. We propose a deep convolutional neural network, M2DCNN, that can accurately segment WMHs and distinguish them from ischemic stroke lesions. M2DCNN consists of two subnets that rely on a set of novel multi-scale features and a novel architecture (inclusion of dense and dilated blocks). Our model is trained and evaluated on two public segmentation challenges with multi-modality MRI images. Empirical tests demonstrate that M2DCNN outperforms current segmentation methods. We empirically demonstrate that M2DCNN effectively separates WMHs from stroke lesions. Finally, ablation experiments reveal that both multi-scale features and architectural elements in our method contribute to the improved predictive performance. [ABSTRACT FROM AUTHOR]

Compressed sensing MRI (CS-MRI) is considered as a powerful technique for decreasing the scan time of MRI while ensuring the image quality. However, state of the art reconstruction algorithms are still subjected to two challenges including terrible parameters tuning and image details loss resulted from over-smoothing. In this paper, we propose a deep frequency-division network (DFDN) to face these two image reconstruction issues. The proposed DFDN approach applies a deep iterative reconstruction network (DIRN) to replace the regularization terms and the corresponding parameters by a stacked convolution neural network (CNN). And then multiple DIRN blocks are cascaded continuously as one deeper neural network. Data consistency (DC) layer is incorporated after each DIRN block to correct the k -space data of intermediate results. Image content loss is computed after each DC layer and frequency-division loss is gained by weighting the high frequency loss and low frequency loss after each DIRN block. The combination of image content loss and frequency-division loss is considered as the total loss for constraining the network training procedure. Validations of the proposed method have been performed on two brain datasets. Visual results and quantitative evaluations show that the proposed DFDN algorithm has better performance in sparse MRI reconstruction than other comparative methods. [ABSTRACT FROM AUTHOR]

Fine-grained visual categorization often suffers from the challenges that the subordinate categories within an entry-level category can only be distinguished by subtler discriminations. It demands an effective algorithm to learn a multiple-perspective density distribution for precious categorization. To shape a coarse-to-fine object perception, a hierarchical convolutional neural network (CNN) denoting the skip-connections convolutional neural network (S-CCNN) was proposed, focusing on a butterfly domain at sub-species level due to the fine-grained structure of the category taxonomy. Specifically, based on the serial backbone, three skip-connections with Grating layers are established to link the earlier layers and upper layers of network, and integrated with DropConnect, exponential linear unit (ELU), and local response normalization (LRN) to alleviate over-fitting and vanishing gradient. Benefitting from the long-span skip-connections, coarse-grained context with orientation descriptions and finer-grained context with semantic representations can be both took into consideration, and they are jointly incorporated into framework. S-CCNN can achieve the cross-utilization of object-level and part-level representations, and its rationality were evidenced with both theory and practice. For effectness verification, a total of 24,836 lab-made images of butterfly specimens spanning 56 sub-species are utilized as testing samples, while 173,852 augmented images are employed for model training. S-CCNN delivers a consistent and significant boost in performance, i.e., validation accuracy achieved 94.17% and testing accuracy achieved 93.36%, which outperformed state-of-the-arts. S-CCNN can easily relish accuracy gains from skip-connections in fine-grained visual categorization of butterfly sub-species, without any bells and whistles. [ABSTRACT FROM AUTHOR]

Crowd counting is a challenging problem due to the scene complexity and scale variation. Although deep learning has achieved great improvement in crowd counting, scene complexity affects the judgement of these methods and they usually regard some objects as people mistakenly; causing potentially enormous errors in the crowd counting result. To address the problem, we propose a novel end-to-end model called Crowd Attention Convolutional Neural Network (CAT-CNN). Our CAT-CNN can adaptively assess the importance of a human head at each pixel location by automatically encoding a confidence map. With the guidance of the confidence map, the position of human head in estimated density map gets more attention to encode the final density map, which can avoid enormous misjudgements effectively. The crowd count can be obtained by integrating the final density map. To encode a highly refined density map, the total crowd count of each image is classified in a designed classification task and we first explicitly map the prior of the population-level category to feature maps. To verify the efficiency of our proposed method, extensive experiments are conducted on three highly challenging datasets. Results establish the superiority of our method over many state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Recommender Systems (RS) are used to give customized recommendations about specific items to users in a variety of applications including social web sites applications, media recommendation and commerce sites etc. Collaborative Filtering (CF) along with the Content Based Filtering (CBF) are two widely used methods which are being efficiently applied in RS implementation. CF suffers with sparseness problem where user-to-item ratings are amply sparse. On the other hand, CBF performance rely on methods of feature extraction for efficient use of items' description. The sparseness of user-to-item ratings and features extraction impede the performance of RS. Quality of rating prediction and item recommendation further degrades due to the negative values present in users/items latent factors. This paper proposes a novel RS that is built upon the semantics based items' content embedding model, enriched with contextual features extracted through Convolutional Neural Network (CNN). Non-negative Matrix Factorization (NMF), supplied with improvised embedding is used as CF technique. Embedding model captures the item details, thus resolving the sparsity, whereas, NMF caters for information loss due to negative values in latent factors. The proposed RS with contextually enriched NMF (Contx-NMF) simultaneously overcomes both the issues of sparseness and loss due to negative values, thus enhancing the rating prediction accuracy. The proposed model is evaluated on three public datasets (MovieLens 1M, MovieLens 10M)and Amazon Instant Video (AIV). The results demonstrate significant improvement in performance of Contx-NMF over state of the art RS models for sparse user-to-item ratings. [ABSTRACT FROM AUTHOR]

• A new core building block, i.e., recurrent convolutional layer, is built to learn the temporal dependencies from time-series sensor data, which enables the prognostics network to effectively memorize useful degradation information over time and thus enhances its representation ability. • A probabilistic RUL prediction result is obtained based on variational inference, which breaks the inherent limitation of convolutional neural networks and is beneficial to maintenance decision making. • A systematic prognostics framework named recurrent convolutional neural network is proposed for RUL prediction of machinery, and its effectiveness and superiority are verified by two case studies, i.e., RUL prediction of rolling element bearings and RUL prediction of milling cutters. Deep learning is becoming more appealing in remaining useful life (RUL) prediction of machines, because it is able to automatically build the mapping relationship between the raw data and the corresponding RUL by representation learning. Among deep learning models, convolutional neural networks (CNNs) are gaining special attention because of its powerful ability in dealing with time-series signals, and have achieved promising results in current studies. These studies, however, suffer from the two limitations: (1) The temporal dependencies of different degradation states are not considered during network construction; and (2) The uncertainty of RUL prediction results cannot be obtained. To overcome the above-mentioned limitations, a new framework named recurrent convolutional neural network (RCNN) is proposed in this paper for RUL prediction of machinery. In RCNN, recurrent convolutional layers are first constructed to model the temporal dependencies of different degradation states. Then, variational inference is used to quantify the uncertainty of RCNN in RUL prediction. The proposed RCNN is evaluated using vibration data from accelerated degradation tests of rolling element bearings and sensor data from life testing of milling cutters, and compared with some state-of-the-art prognostics approaches. Experimental results demonstrate the effectiveness and superiority of RCNN in improving the accuracy and convergence of RUL prediction. More importantly, RCNN is able to provide a probabilistic RUL prediction result, which breaks the inherent limitation of CNNs and facilitates maintenance decision making. [ABSTRACT FROM AUTHOR]

• An effective plane assignment cost based on sparse 3D points, structure priors and CNNs is proposed. • A new piecewise planar stereo method that jointly optimizes image regions and their associated planes is proposed. • The problems (i.e., inaccurate superpixels, unreliable regularization) existing traditional methods are effectively resolved. Piecewise planar stereo methods can approximately reconstruct the complete structures of a scene by overcoming challenging difficulties (e.g., poorly textured regions) that pixel-level stereo methods cannot resolve. In this paper, a novel plane assignment cost is first constructed by incorporating scene structure priors and high-level image features obtained by convolutional neural network (CNN). Then, the piecewise planar scene structures are reconstructed in a progressive manner that jointly optimizes image regions (or superpixels) and their associated planes, followed by a global plane assignment optimization under a Markov Random Field (MRF) framework. Experimental results on a variety of urban scenes confirm that the proposed method can effectively reconstruct the complete structures of a scene from only sparse three-dimensional (3D) points with high efficiency and accuracy and can achieve superior results compared with state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• This paper proposes a CNN-based segmentation framework multiscale dense CNN (MDCNN) to automatically segment cerebral vessel in DSA images. • In the training process, a patch selection strategy is put forward to get enough training data from the limited quantity of annotated cerebrovascular DSA images. • In order to verify the validity of the proposed method, we label a DSA cerebrovascular dataset DCVessel. Accurate cerebrovascular segmentation in Digital Subtraction Angiography (DSA) image, as an indispensable step, can help doctors appropriately estimate the degree of cerebrovascular lesions to avoid misdiagnosis. Because of the complexity of cerebrovascular structure and the uneven distribution of contrast media in DSA, automatic segmentation is a challenging task in clinical diagnosis. In recent years, deep convolutional neural networks (CNN) have outperformed the state-of-art methods and shown great potential for medical image segmentation. This paper proposes a CNN-based segmentation framework Multiscale Dense CNN (MDCNN) to automatically segment cerebral vessel in DSA images. Inspired by U-net, this proposed MDCNN structure is designed as encoder-decoder architecture. Considering that the diameters of blood vessel in cerebrovascular are various, we define a multiscale module to segment cerebral vessel with different diameters. Meanwhile, we redesign the skip connections between encoder and decoder stage to utilize more features from encoder stage. To improve the capability of extracting high-level features, improved dense blocks are introduced. In terms of simplifying parameters, we refer to the idea of deep supervision to make pruning possible. The proposed framework is tested on DCVessel (a DSA cerebrovascular dataset made by our lab). Our proposed method reaches 0.8813, 0.9784, 0.8775, 0.9886, 0.9944 in F1 score, accuracy (Acc), sensitivity (Sen), specificity (Spe) and AUC respectively, outperforming the state-of-art methods. Meanwhile, we evaluate our MDCNN framework on benchmark retinal vessel dataset DRIVE. The promising experiment results demonstrate that proposed MDCNN model is extensible for various vessel segmentation tasks. [ABSTRACT FROM AUTHOR]

Convolutional neural networks have been successfully applied to detect salient objects in an image. However, how to better use convolutional features for co-saliency detection, which is an emerging branch of saliency detection, is not fully explored. This paper proposes a convolutional neural network based co-saliency detection model, which consists of two key parts including the integration of multi-layer convolutional features extracted from a group of images and the inter-image saliency propagation. Firstly, the input image and its four co-images belonging to the same image category are passed through the VGG16 model, to obtain the multi-layer convolutional features of these images. Secondly, multi-scale synthesized feature maps, which contain both internal features and correlative features, are generated by integrating the multi-layer convolutional features. Thirdly, via the integration of low-level boundary features and high-level semantic features, the multi-scale synthesized feature maps are enhanced and fused together to generate the initial co-saliency map. Finally, an inter-image saliency propagation method is utilized to refine the initial co-saliency map, yielding the final co-saliency map with the improved quality. Experimental results on two public datasets demonstrate that the proposed model achieves the best performance compared to the state-of-the-art co-saliency detection models. [ABSTRACT FROM AUTHOR]

Anomaly detection in crowded scenes plays a crucial role in automatic video surveillance to avert any casualty in the areas witnessing the high amount of footfalls. The key challenge for automatically classifying the anomalies in crowd image is the usage of feature set and techniques which can be replicated in every crowded scenario. In this paper, we propose a novel concept of Aggregation of Ensembles (AOE) for detecting an anomaly in video data showing crowded scenes, which leverage the existing capability of pre-trained ConvNets and a pool of classifiers. The proposed approach uses an ensemble of different fine-tuned Convolutional Neural Networks (CNN) based on the hypothesis that different CNN architectures learn different levels of semantic representation from crowd videos and thus an ensemble of CNNs will enable enriched feature sets to be extracted. The proposed AOE concept utilizes the fine-tuned ConvNets as fixed feature extractors to train variants of SVM classifier and then the posterior probabilities are fused to predict the anomaly in the crowd frame sequences. The experimental results show that the proposed Aggregation of Ensembles fine-tuned CNNs of various architectures achieve a higher accuracy in comparison with other established methods on benchmark datasets. [ABSTRACT FROM AUTHOR]

Recent advances in single-image super-resolution are mostly based on convolutional neural networks. However, the high computational cost is a critical obstacle to achieve real-time performance(24 fps). In this paper, we proposed the Stacked Dense Adaptive Network (SDANet) to reduce the computational cost and improve the computational density of CPUs/GPUs. Especially, SDANet consists of (1) a dense adaptive learning (DAL) module, which harnesses adaptive gate units (AGU) to retain the high-frequency information in the prior layers. (2) a stacked bottleneck (STB) module, that stacks the information of all previous layers to improve the computational density and reduce the computation dramatically. (3) a spatial-channel transform (SCT) module, which transforms bilinear spatial information to channels to improve the computational density via parallel computing. With the proposed three modules, our algorithm takes full advantage of the high-frequency information of all the layers, resulting in reducing the computational complexity dramatically. Extensive experiments verify the necessity of the SDANet, not only yielding the state-of-the-art results with a considerable margin on five benchmark datasets in terms of PSNR and visual quality but also achieving real-time performance(> 24 fps) with a generic CPU. [ABSTRACT FROM AUTHOR]

Color laser printers have fast printing speed and high resolution, and forgeries using color laser printers can cause significant harm to society. A source printer identification technique can be employed as a countermeasure to those forgeries. This paper presents a color laser printer identification method based on cascaded learning of deep neural networks. First, the refiner network is trained by adversarial training to refine the synthetic dataset for halftone color decomposition. Then, the halftone color decomposing ConvNet is trained with the refined dataset. The trained knowledge about halftone color decomposition is transferred to the printer identifying ConvNet to enhance the identification accuracy. Training of the printer identifying ConvNet is carried out with real halftone images printed from candidate source printers. The robustness about rotation and scaling is considered in training process, which is not considered in existing methods. Experiments are performed on eight color laser printers, and the performance is compared with several existing methods. The experimental results clearly show that the proposed method outperforms existing source color laser printer identification methods. [ABSTRACT FROM AUTHOR]

With the scale expansion and structural upgrading of pipeline network, the detection methods based on both ends of the pipeline pressure have appeared the limitations of judging pipeline status in the multi-mode and complex network system. To overcome the limitation of early methods, a pipeline network status detection method based on data transformation convolutional neural network (DT-CNN) is proposed in this paper. Firstly, the difference among the eigenvalue distribution of data covariance matrices is calculated to detect the pipeline status by Kullback-Leibler divergence (KLD). If the eigenvalue distribution deviates from the normal status, the KLD will exceed the given threshold. Furthermore, an improved CNN model is proposed to judge pipeline status by converting the largest eigenvectors of data covariance matrices to extract features. The effectiveness of the proposed detection method is demonstrated through the simulation results of a practical pipeline network. [ABSTRACT FROM AUTHOR]

Deep learning-based salient object detection techniques have shown impressive results compared to conventional saliency detection by handcrafted features. Integrating hierarchical features of Convolutional Neural Networks (CNN) to achieve fine-grained saliency detection is a current trend, and various deep architectures are proposed by researchers, including "skip-layer" architecture, "top-down" architecture, "short-connection" architecture and so on. While these architectures have achieved progressive improvement on detection accuracy, it is still unclear about the underlying distinctions and connections between these schemes. In this paper, we review and draw underlying connections between these architectures, and show that they actually could be unified into a general framework, which simply just has side structures with different depths. Based on the idea of designing deeper side structures for better detection accuracy, we propose a unified framework called Deepside that can be deeply supervised to incorporate hierarchical CNN features. Additionally, to fuse multiple side outputs from the network, we propose a novel fusion technique based on segmentation-based pooling, which severs as a built-in component in the CNN architecture and guarantees more accurate boundary details of detected salient objects. The effectiveness of the proposed Deepside scheme against state-of-the-art models is validated on 8 benchmark datasets. [ABSTRACT FROM AUTHOR]

Facial expression recognition (FER) in the wild is a novel and challenging topic in the field of human emotion perception. Different kinds of convolutional neural network (CNN) approaches have been applied to this topic, but few of them ever considered what kind of architecture was better for the FER research. In this paper, we proposed three novel CNN models with different architectures. The first one is a shallow network, named the Light-CNN, which is a fully convolutional neural network consisting of six depthwise separable residual convolution modules to solve the problem of complex topology and over-fitting. The second one is a dual-branch CNN which extracts traditional LBP features and deep learning features in parallel. The third one is a pre-trained CNN which is designed by transfer learning technique to overcome the shortage of training samples. Extensive evaluations on three popular datasets (public CK+, multi-view BU-3DEF and FER2013 datasets) demonstrated that our models were competitive and representative in the field of FER in the wild research. We achieved significant better results with comparisons to plenty of state-of-the-art approaches. Moreover, we provided discussions on the effectiveness and practicability of CNNs with different feature types and architectures for FER in the wild as well. [ABSTRACT FROM AUTHOR]

Visual saliency detection, aimed at the simulation of human visual system (HVS), has drawn wide attention in recent decades. Reconstruction based saliency detection models are established methods for saliency detection, which predict unexpected regions via linear combination or auto-encoder network. However, these models are ineffective in dealing with images due to the loss of spatial information caused by the conversion from images to vectors. In this paper, a novel approach is proposed to solve this problem. The core is a deep reconstruction model, i.e., convolutional neural network for reconstruction stacked with auto-encoder (CNNR for short). On the one hand, the use of CNN is able to directly take two-dimensional data as input rather than having to convert the matrix to a series of vectors as in conventional reconstruction based saliency detection methods. On the other hand, the training process of CNN is augmented with the initialization obtained by an unsupervised learning process of convolutional auto-encoder (CAE). By this way, our CNNR model can be trained on limited labeled data, with the weights of the CNN being meaningfully initialized by CAE instead of random initialization. Performance evaluations are conducted through comprehensive experiments on four benchmark datasets and the comparisons with eight state-of-the-art saliency detection models show that our proposed deep reconstruction model outperforms most of the eight state-of-the-art saliency detection models. [ABSTRACT FROM AUTHOR]

Nowadays, with the rapid development of high bit-depth (HBD) monitors, the demands for high quality image visualization have been raised. However, a prominent problem is the inconsistency between existing low bit-depth (LBD) images and HBD monitors. When LBD images are simply de-quantized to HBD ones, there will be severe false contour artifacts in smooth gradient areas, degrading image visual quality. Therefore, bit-depth enhancement plays a key role in viewing LBD images on HBD monitors. In this paper, motivated by the promising results of deep Convolutional Neural Network (CNN) in generating realistic high-quality images, we proposed a novel algorithm to recover photo-realistic HBD images. To the best of our knowledge, CNN is introduced to bit-depth enhancement task for the first time. A novel neural network is proposed with summation and concatenation skip connections among transposed convolutional layers to cope with the gradient vanishing problem. Besides, different from traditional pixel-wise loss functions, perceptual loss is adopted to reconstruct images with higher visual quality and structural similarity to original HBD sources. Experiments performed on three datasets demonstrate that the proposed method outperforms state-of-the-art algorithms objectively and subjectively with suppressed false contour artifacts and preserved textures. [ABSTRACT FROM AUTHOR]

Highlights • The re-identification problem is re-formulate as a multi-instance verification problem, to meet the requirement of more realistic real-world applications. • Two paradigms IS and ES are developed to effectively perform multiinstance learning in an end-to-end way, which takes pairs of bags as input and directly output bag labels. • The proposed method relaxes the requirements for strictly accurate annotations, accordingly increases the availability of large re-identification datasets. • The proposed MICNN can achieve superior performance against single instance CNN framework. Abstract This paper tackles the challenging problem of multi-shot person re-identification with Convolutional Neural Network (CNN). As no prior information about how importance each instance plays, it is non-trivial to exploit the interaction information shared by the multi-shot images to help identification. Traditional CNN is in single-shot architecture, then how to utilize the interaction information provided by multi-shot images becomes an important problem to solve. Furthermore, as data augmentation methods are not strictly label-preserving, it increases the difficulty to select discriminative instance for CNN training. In this paper, we propose a weakly supervised CNN framework named Multi-Instance Convolutional Neural Network (MICNN) to solve the aforementioned problem. We develop two paradigms, i.e., Embedding-Space paradigm and Instance-Space paradigm, which re-formulate the person re-identification problem as a multi-instance verification problem with part-based features extracted by neural network. We respectively devise a specific bag-level loss function which incorporates the characteristics of the multi-instance problem for each paradigm. Experiments show that the proposed IS method outperforms many related state-of-the-art techniques on four benchmark datasets: CUHK03, SYSUm, RAiD and Market-1501. [ABSTRACT FROM AUTHOR]

Abstract Head pose estimation has many wide applications such as driver monitoring, attention recognition and multi-view facial analysis. Most of the previous works routinely utilize detected face regions to further estimate head pose with hard labels, which limits to explore more discriminative texture information and tends to over-fit. In this paper, we present a novel framework to alleviate this problem, which takes entire images as input and constructs soft labels using a Gaussian distribution function as supervision information, and then introduces a regularized convolutional neural network architecture that is optimized by two types of similarity measure functions: Kullback–Leibler divergence loss and Jeffreys divergence loss. The regularized architecture includes four modules: one backbone net for learning common features, two parallel branches named sub-net1 and sub-net2 for learning complementary features and one feature fusion module, namely, fused net. The architecture is trained in an alternately training fashion, making the learned model more robust and stable. Extensive experiments have been carried out on three public datasets: Pointing04, CAS-PEAL-R1 and CMU Multi-PIE. The results show that our method achieves a significant improvement in performance compared to the state of the art. The best accuracy on the three datasets we achieve are 85.77%, 99.19% and 99.88%, respectively. [ABSTRACT FROM AUTHOR]

Abstract This paper proposes an object detection model named Feature Enriching Object Detection Framework with Weak Segmentation Loss (FEOD) based on convolutional neural networks, which is an improvement of You Only Look Once (YOLO). To overcome the shortcoming of insensitivity to small objects, a novel feature enriching module is proposed to augment the semantic information of the feature in the shallow layer within a typical deep detector. Meanwhile, Focal Loss is also introduced to our model to further improve the algorithm performance. To obtain features more suitable for object detection, a more powerful feature extractor – DetNet is used as the backbone. An mAP of 83.8 in VOC2007 test and an mAP of 82.1 in VOC2012 test are achieved with a FPS of 11.3 with a NVIDIA GTX1080Ti GPU. For a lower resolution version, we achieve an mAP of 81.9 and 80.9 respectively in VOC2007 and VOC2012 tests with FPS of 17. Comparisons with other object detection algorithms have shown that our method works well and achieves a trade-off of accuracy and speed. [ABSTRACT FROM AUTHOR]

Abstract Different convolutional neural networks (CNNs) may learn different levels of discriminative features to represent the raw face data. To enhance the discrimination of deeply learned face features, we propose a customized weighted discriminative loss (CWD loss) to seek a customized constraint for mitigating the large perturbations caused by imbalanced distribution of correctly-classified features and misclassified features. It focuses on mapping the raw data into a feature space such that deeply learned face features can achieve a high discrimination for representation, by restraining the intra-class variations and the inter-class variations, simultaneously. Extensive experiments carried out on several famous face recognition benchmarks, including LFW, YTF, FGLFW and BLUFR, demonstrate that the proposed approach can achieve superior performance over the related approaches. [ABSTRACT FROM AUTHOR]

Abstract The number of kernels in a convolutional neural network (CNN) can have a significant impact on the performance of the CNN in the aspects of accuracy and computation efficiency. However, existing approaches to determining the number of convolution kernels are mainly conducted through a manual process, which suffers from the problems of potential overfitting, instability and inefficiency. In response to these problems, this paper presents a corner radiation area adaptation (CRAA) based method to automatically determine the number of convolution kernels. CRAA is evaluated in comparison with three representative methods on multiple public data sets. Experimental results show that CRAA is robust to the number of convolution kernels which is well adapted to a specific data set and achieves a higher level of classification accuracy by 3% when spending the same period of time in classification. More importantly, CRAA reduces the computational time by 15% in comparison with the three representative approaches when reaching the same level of accuracy in classification. [ABSTRACT FROM AUTHOR]

Abstract The recognition of license plates is very important for intelligent transportation systems. Generally, the performance of an intelligent recognition algorithm is greatly affected by different shooting angles, illumination conditions and backgrounds of the license plate images. This paper presents a sequence recognition approach for intelligent recognition of Chinese license plates. Firstly, a spatial transformer network (STN) is employed to adjust the inclined and deformed license plates such that all the plates have a uniform orientation and thus are easier to be recognized. Then, an improved convolutional neural network (CNN) is designed to extract sequence features of the rectified license plates. The features of different convolutional layers are integrated as input to a bi-directional recurrent neural network (BRNN), where the character segmentation is not needed. Finally, the recognition is accomplished by the BRNN and connectionist temporal classification (CTC). Due to the lack of adequate Chinese license plates, an effective training method is presented in which the network is pre-trained by sufficiently enough synthetic license plates and is fine-tuned by our collected real Chinese license plates. The experimental results show that our model achieves better recognition accuracy and lower average edit distance than some existing methods. [ABSTRACT FROM AUTHOR]