Search

• We propose a novel Hybrid CNN architecture to address the problem of sketch recognition. • S-Net extracts shape features, which is invariant for sketch rotation and transformation. • Hybrid CNN achieves state-of-the-art on sketch classification and sketch-based image retrieval tasks. • The classification accuracy is 84.42% and 82.74% on TU-Berlin and SketchX datasets, respectively. • The MAP of SBIR is 57.4% and 28.74% on SketchX and Flickr15K-Large datasets, respectively. With the popularity of touch-screen devices, it is becoming increasingly important to understand users' free-hand sketches in computer vision and human-computer interaction. Most of existing sketch recognition methods employ the similar strategies used in image recognition, relying on appearance information represented by hand-crafted features or deep features from convolutional neural networks. We believe that sketch recognition can benefit from learning both appearance and shape representation. In this paper, we propose a novel architecture, named Hybrid CNN, which is composed of A-Net and S-Net. They describe appearance information and shape information, respectively. Hybrid CNN is then comprehensively evaluated in the sketch classification and retrieval tasks on different datasets, including TU-Berlin, Sketchy and Flickr15k. Experimental results demonstrate that the Hybrid CNN achieves competitive accuracy compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• We propose a unified algorithm to compress the CNN for IAHCCR. • We achieve 12.4 × compression and 1.7 × acceleration with only 0.17% accuracy loss. • The experiments are conducted on IAHCC-UCAS2016, ICDAR-2013, and MNIST. • The state-of-the-art performance is achieved on IAHCC-UCAS2016. Since the convolutional neural network (CNN) has brought great breakthroughs in the field of computer vision, it recently has been introduced to the in-air handwritten Chinese character recognition (IAHCCR) to achieve better recognition performance. However, the CNN is typically over-parameterized and contains lots of redundant filters or parameters. This leads the CNN to suffer from huge computation cost and considerable storage usage, limiting its deployments to resource-constrained devices like mobile phones and intelligent TVs. In this paper, we propose a unified algorithm to effectively compress the CNN for IAHCCR with little accuracy loss. Specifically, we first utilize the channel pruning strategy to simplify the network structure, and then adopt the network quantization technique to represent parameters with lower precision. We conduct experiments on the in-air handwriting dataset IAHCC-UCAS2016, where the baseline CNN achieves the state-of-the-art accuracy of 95.33% with 15.5 MB of storage. After the compression, we achieve 12.4 × storage saving and 1.7 × theoretical acceleration with only 0.17% accuracy loss. Moreover, evaluations on other benchmark datasets including ICDAR-2013 and MNIST further demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

• We propose a novel method termed SFCM to fuse CNNs features of different layers. • Our SFCM can be implemented by direct connection and residual connection. • Our SFCM can be used in many existing frameworks to achieve better performance. • We evaluate the SFCM on many different challenging computer vision tasks. Different layers of deep convolutional neural networks(CNNs) can encode different-level information. High-layer features always contain more semantic information, and low-layer features contain more detail information. However, low-layer features suffer from the background clutter and semantic ambiguity. During visual recognition, the feature combination of the low-layer and high-level features plays an important role in context modulation. If directly combining the high-layer and low-layer features, the background clutter and semantic ambiguity may be caused due to the introduction of detailed information. In this paper, we propose a general network architecture to concatenate CNN features of different layers in a simple and effective way, called Selective Feature Connection Mechanism (SFCM). Low-level features are selectively linked to high-level features with a feature selector which is generated by high-level features. The proposed connection mechanism can effectively overcome the above-mentioned drawbacks. We demonstrate the effectiveness, superiority, and universal applicability of this method on multiple challenging computer vision tasks, including image classification, scene text detection, and image-to-image translation. [ABSTRACT FROM AUTHOR]

Highlights • We propose the use of pooling and deconvolution operations in CNNs in order to greatly increase their receptive fields. • We highlight characteristics specific to the stereo matching problem and how they can relate to the way that CNNs operate. • We use a simple feature space transformation that allow us to better evaluate the quality of the features extracted. • We present several easy to train models capable of accurate stereo matching, even without spatial regularization. Abstract Computational stereo is one of the classical problems in computer vision. Numerous algorithms and solutions have been reported in recent years focusing on developing methods for computing similarity, aggregating it to obtain spatial support and finally optimizing an energy function to find the final disparity. In this paper, we focus on the feature extraction component of stereo matching architecture and we show standard CNNs operation can be used to improve the quality of the features used to find point correspondences. Furthermore, we use a simple space aggregation that hugely simplifies the correlation learning problem, allowing us to better evaluate the quality of the features extracted. Our results on benchmark data are compelling and show promising potential even without refining the solution. [ABSTRACT FROM AUTHOR]

Highlights • The paper is theoretically well founded and motivated by recent research on the physiological aspects of visual recognition. • The proposed idea is simple yet effective. It provides a practical way of training CNN for cross-domain 3D shape retrieval. • High versatility, a single framework for all the three investigated tasks. • A new selected-view matching strategy based on-line learning. • The experiments are sufficient with significantly improved results. Graphical abstract Abstract Recent vision-brain physiological experiments [65] have demonstrated that contours, and in particular, contour junctions present in the 2D images are very informative for revealing the 3D structure of the object. Inspired by this observation, we take 2D sketches (or 2D views of 3D sketches) and edge maps of 2D views of 3D models as a unified domain to train the Convolutional Neural Network (CNN). The CNN features are then used for 3D object representation. We show that the CNN can successfully learn the object structure from different types of clues. The performance of the proposed method demonstrates that the semantic gap between the 2D/3D sketches and the 3D models can be bridged without any cross-domain similarity learning. Experiments show that our approach significantly outperforms the state-of-the-art 2D/3D sketch-based 3D retrieval methods. [ABSTRACT FROM AUTHOR]

Highlights • It is for the first time the pure binarized convolutional layers are incorporated into Gabor CNNs. • HGCNs can reduce the size of required storage space of any given convolutional filters by a factor of 32. • HGCNs can achieve the comparable performance to the full-precision model. Abstract Despite great effectiveness of very wide and deep convolutional neural networks (DCNNs) in various computer vision tasks, the significant cost in terms of storage requirement of such networks impedes the deployment on computationally limited devices. Therefore, the network of resource and accuracy trade-offs always have been paid much attention in the popular DCNNs. A new deep model, Hybrid Gabor Convolutional Networks (HGCNs), is proposed in this paper. Incorporating binarized feature map and binarized filters into DCNNs in alternating way, HGCNs alleviate the accuracy loss while reduce the memory storage by 32. Also, HGCNs can be easily implemented and compatible with any other popular deep learning architecture by manipulating the basic structure elements of the DCNNs (i. e. , the convolution operator) based on the binarization filter. Most importantly, HGCNs achieve a comparable performance while largely reducing the storage memory compared with state-of-the-art networks such as Resnet. The source code will be public soon. [ABSTRACT FROM AUTHOR]

Depth information has been discovered to improve the performance of computer vision tasks, such as semantic segmentation and object recognition. However, careful acquisition of depth data needs highly developed depth sensors which are expensive. As a classic computer vision task, depth estimation from a single image has obtained promising results based on supervised learning methods. In this paper, we investigate the extension of color images with corresponding deep-regressed depth images in boosting the performance of semantic segmentation. Furthermore, the usage of combining color channels with the estimated depth or the ground truth depth channel is compared. Specifically, there are two stages in our work. Firstly, we adopt the framework of convolutional neural networks (CNN) for the depth estimation by combing the global depth network and the depth gradient network. After refining based on these two networks, the depth image map can be estimated in a deep-regressed manner. Secondly, after augmenting the color images with the predicted depth images, fully convolutional networks (FCN) are further used to implement the pixel-level semantic labeling. In the experiments, we employ two popular RGBD datasets, i.e., SUNRGBD and NYUDv2, for 37 and 40-class semantic segmentation, respectively. By comparing with the ground truth depth images, experimental results demonstrate that the networks trained on the estimated depth images can achieve comparable performance on facilitating the accuracy of semantic segmentation task. [ABSTRACT FROM AUTHOR]

Extracting high level information from digital images and videos is a hard problem frequently faced by the computer vision and machine learning communities. Modern surveillance systems can monitor people, cars or objects by using computer vision methods. The objective of this work is to propose a method for identifying soft-biometrics, in the form of clothing and gender, from images containing people, as a previous step for further identifying people themselves. We propose a solution to this classification problem using a Convolutional Neural Network, working as an all-in-one feature extractor and classifier. This method allows the development of a high-level end-to-end clothing/gender classifier. Experiments were done comparing the CNN with hand-designed classifiers. Also, two different operating modes of CNN are proposed and compared each other. The results obtained were very promising, showing that is possible to extract soft-biometrics attributes using an end-to-end CNN classifier. The proposed method achieved a good generalization capability, classifying the three different attributes with good accuracy. This suggests the possibility to search images using soft biometrics as search terms. [ABSTRACT FROM AUTHOR]