Search

Date fruit (Phoenix dactylifera L.) is known for its diversity in terms of genotypes that differ in shape, color, texture, taste, and other physical and sensory attributes, in addition to biochemical compounds. This richness generates complex, long, and tedious work at the scale of farm as well as in handling, processing, and storage facilities to prepare standardized, and homogenous final products that deal with consumer preferences and needs. To alleviate this, an increased importance and particular trend were observed recently for the use of artificial intelligence (AI) techniques and computer vision technology since they are considered as accurate, objective, non-destructive, fast, and efficient emerging technologies. In this context, this manuscript provides comprehensive insights into key applications of AI techniques and computer vision technology, using different classification models and algorithms developed on the basis (for example) of neural network classifiers, such as convolutional and artificial neural network, on harvest and postharvest assessment of date fruit. Thus, the importance of date fruit dataset used in the AI domain was discussed with link to all accessible datasets in the field of date palm. In addition, available and relevant AI tools and approaches applied to the postharvest of date fruit from maturity distinguishing until storage through other key handling operations, sorting, grading, etc., were detailed in the five sections of this paper. In terms of conclusion, this review showed that these smart technologies can play a substantial role and offer new opportunities to improve the whole value chain of date palm sector. • Harvesting and post-harvest handling impact quality and storability of dates. • These operations are tedious and subjective leading to quantitative/qualitative losses. • Artificial intelligence (AI) techniques are emergent technologies that can reduce these losses. • AI techniques are more accurate, fast, non-destructive and objective. [ABSTRACT FROM AUTHOR]

Nutrition plays a vital role in maintaining human health. Traditional methods used for assessing food composition & nutritional content often require destructive sample preparation, which can be time-consuming and costly. Therefore, computer vision-based approaches have emerged as promising alternatives that enable rapid and non-destructive analysis of various nutritional parameters in foods. In this review, we summarized computer vision applications in meat processing, grains, fruits and vegetables, and seafood. We reviewed recent advancements in computer vision and deep learning-based algorithms employed for food recognition and nutrient estimation. Various existing food recognition and nutrient estimation datasets are also reviewed. Conventional methods offer some limitations, while vision-based technologies provide quick and non-destructive analysis of food composition & nutritional content. Computer vision and deep neural network architectures provide remarkable accuracy for food nutrient measurement. In conclusion, deep learning-based models are paving the way for a promising future in nutritional and health optimization research. In the future, vision-based technologies are expected to transform food classification and detection by enabling more rapid, affordable, and accurate nutritional analyses. Therefore, computer vision is developing into a useful tool for fast and precise evaluation of food nutrients without enabling samples to be damaged. [Display omitted] • Integrating nutrition analysis with computer vision for non-destructive food quality assessment are reviewed. • Transformer-based and CNN-based hybrid approaches are the accurate nutrient recognition methods in the food industry. • Vision-based technologies have revolutionized food analysis by facilitating faster, cost-effective, and precise methods. [ABSTRACT FROM AUTHOR]

• Conventional quality analysis of emulsions proved subjective and time-consuming. • Machine learning outperformed deep learning predicting in-process emulsion quality. • Droplet characteristics are better predictors compared to pixel intensity. • Random Forest model presented 99.8% accuracy. • The automated approach demonstrated promising potential in reducing over processing. Determining the uniformity and consistency of droplet size in the dispersed phase is key to emulsion stability. Conventional methods, which include manual microscopic evaluation and laser diffraction, have presented many challenges to achieve an accurate evaluation of droplet dispersion in pharmaceutical emulsions. Artificial intelligence techniques have demonstrated potential in overcoming the subjectivity and time-consumption associated with the manual approaches in industry. A new automated machine learning approach is presented in this study to predict in-process emulsion quality from micrographs. Droplet characteristics are extracted from emulsion micrographs using a histogram-based image segmentation technique. Machine learning classification models are developed, with a selected set of droplet characteristics as predictors, via Random Forest, Multinomial Logistic Regression and Vanilla Neural Network to classify the micrographs into four categories. The hyper-parameters of the models are tuned using 10-fold cross validation. A pixel-based Convolutional Neural Network model is also investigated. Random Forest presented the best accuracy of 99.78% compared to the deep learning models, which presented a bias towards the high frequency classes. The automated machine learning approach demonstrated promising potential for inline emulsion quality evaluation. [ABSTRACT FROM AUTHOR]

• Cyber-physical system architecture created to link truck loads to bridge responses. • Measured truck loads linked to bridge responses create input-output data for SHM. • Highway truck loads tracked using deep learning applied to traffic camera feeds. • Bridge monitoring triggered by detecting trucks in real-time using CNN algorithms. Bridges are critical components of highways ensuring traffic can efficiently travel over obstructions such as bodies of water, valleys, and other roads. Ensuring bridges are in sound structural condition is essential for safe and efficient highway operations. Structural health monitoring (SHM) systems designed to measure bridge responses have been developed to quantitatively track the health of bridges. More recently, SHM systems have also begun to integrate measurement of vehicular loads that create the responses measured. However, precise correlation of traffic loads to bridge responses remains a costly and technically difficult strategy. To address existing technical limitations, a cyber-physical system (CPS) framework is proposed to track truck loads in a highway corridor, to trigger SHM systems to record bridge responses, and to automate the linking of bridge response data with truck weights collected by weigh-in-motion (WIM) stations installed along the corridor but not collocated with the bridges. To link truck weights to bridge responses, computer vision methods based on convolutional neural networks (CNN) are used to automate the detection and reidentification of trucks using traffic cameras. The single-stage CNN object detector YOLO is trained using a customized dataset to identify trucks from camera images at each instrumentation site; high precision is obtained with the YOLO detector exceeding 95% average precision (AP) for an intersection over union (IOU) threshold of 0.75. To reidentify the same truck at different locations in the corridor, this study adopts a CNN-based encoder trained via a triplet network and a mutual nearest neighbor strategy using feature vectors extracted from images at each measurement location. The proposed reidentification method is implemented in the CPS cloud environment and obtains a F1-score of 0.97. The study also explores the triggering of bridge monitoring systems based on visual detection of trucks by a traffic camera installed upstream to the bridges. The triggering strategy proves to be highly efficient with 99% of the triggered data collection cycles capturing truck events at each bridge. To validate, the CPS architecture is implemented on a 20-mile highway corridor that has a WIM station already installed; four traffic cameras and two bridge SHM systems are installed along the corridor and integrated with a CPS architecture hosted on the cloud. In total, over 10,000 trucks are observed at all measurement locations over one year allowing peak bridge responses to be correlated to both measured truck weights and to one another. [ABSTRACT FROM AUTHOR]

• We propose a novel Hybrid CNN architecture to address the problem of sketch recognition. • S-Net extracts shape features, which is invariant for sketch rotation and transformation. • Hybrid CNN achieves state-of-the-art on sketch classification and sketch-based image retrieval tasks. • The classification accuracy is 84.42% and 82.74% on TU-Berlin and SketchX datasets, respectively. • The MAP of SBIR is 57.4% and 28.74% on SketchX and Flickr15K-Large datasets, respectively. With the popularity of touch-screen devices, it is becoming increasingly important to understand users' free-hand sketches in computer vision and human-computer interaction. Most of existing sketch recognition methods employ the similar strategies used in image recognition, relying on appearance information represented by hand-crafted features or deep features from convolutional neural networks. We believe that sketch recognition can benefit from learning both appearance and shape representation. In this paper, we propose a novel architecture, named Hybrid CNN, which is composed of A-Net and S-Net. They describe appearance information and shape information, respectively. Hybrid CNN is then comprehensively evaluated in the sketch classification and retrieval tasks on different datasets, including TU-Berlin, Sketchy and Flickr15k. Experimental results demonstrate that the Hybrid CNN achieves competitive accuracy compared with the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Salient object detection can be applied as a critical preprocessing step in many computer vision tasks. Recent studies of salient object detection mainly employed convolutional neural networks (CNNs) for mining high-level semantic properties. However, the existing methods can still be improved to find precise semantic information in different scenarios. In particular, in the two main methods employed for salient object detection, the patchwise detection models might ignore spatial structures among regions and the fully convolution-based models mainly consider semantic features in a global manner. In this paper, we proposed a salient object detection framework by embedding topological features into a deep neural network for extracting semantics. We segment the input image and compute weight for each region with low-level features. The weighted segmentation result is called a topological map and it provides an additional channel for the CNN to emphasize the structural integrity and locality during the extraction of semantic features. We also utilize the topological map for saliency refinement based on a conditional random field at the end of our model. Experimental results on six benchmark datasets demonstrated that our proposed framework achieves competitive performance compared to other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

• We propose a unified algorithm to compress the CNN for IAHCCR. • We achieve 12.4 × compression and 1.7 × acceleration with only 0.17% accuracy loss. • The experiments are conducted on IAHCC-UCAS2016, ICDAR-2013, and MNIST. • The state-of-the-art performance is achieved on IAHCC-UCAS2016. Since the convolutional neural network (CNN) has brought great breakthroughs in the field of computer vision, it recently has been introduced to the in-air handwritten Chinese character recognition (IAHCCR) to achieve better recognition performance. However, the CNN is typically over-parameterized and contains lots of redundant filters or parameters. This leads the CNN to suffer from huge computation cost and considerable storage usage, limiting its deployments to resource-constrained devices like mobile phones and intelligent TVs. In this paper, we propose a unified algorithm to effectively compress the CNN for IAHCCR with little accuracy loss. Specifically, we first utilize the channel pruning strategy to simplify the network structure, and then adopt the network quantization technique to represent parameters with lower precision. We conduct experiments on the in-air handwriting dataset IAHCC-UCAS2016, where the baseline CNN achieves the state-of-the-art accuracy of 95.33% with 15.5 MB of storage. After the compression, we achieve 12.4 × storage saving and 1.7 × theoretical acceleration with only 0.17% accuracy loss. Moreover, evaluations on other benchmark datasets including ICDAR-2013 and MNIST further demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

• We propose a novel method termed SFCM to fuse CNNs features of different layers. • Our SFCM can be implemented by direct connection and residual connection. • Our SFCM can be used in many existing frameworks to achieve better performance. • We evaluate the SFCM on many different challenging computer vision tasks. Different layers of deep convolutional neural networks(CNNs) can encode different-level information. High-layer features always contain more semantic information, and low-layer features contain more detail information. However, low-layer features suffer from the background clutter and semantic ambiguity. During visual recognition, the feature combination of the low-layer and high-level features plays an important role in context modulation. If directly combining the high-layer and low-layer features, the background clutter and semantic ambiguity may be caused due to the introduction of detailed information. In this paper, we propose a general network architecture to concatenate CNN features of different layers in a simple and effective way, called Selective Feature Connection Mechanism (SFCM). Low-level features are selectively linked to high-level features with a feature selector which is generated by high-level features. The proposed connection mechanism can effectively overcome the above-mentioned drawbacks. We demonstrate the effectiveness, superiority, and universal applicability of this method on multiple challenging computer vision tasks, including image classification, scene text detection, and image-to-image translation. [ABSTRACT FROM AUTHOR]

Alzheimer's disease, the most common form of dementia is a neurodegenerative brain order that has currently no cure for it. Hence, early diagnosis of such disease using computer-aided systems is a subject of great importance and extensive research amongst researchers. Nowadays, deep learning or particularly convolutional neural network (CNN) is getting more attention due to its state-of-the-art performances in variety of computer vision tasks such as visual object classification, detection and segmentation. Several recent studies, that have used brain MRI scans and deep learning have shown promising results for diagnosis of Alzheimer's disease. However, most common issue with deep learning architectures such as CNN is that they require large amount of data for training. In this paper, a mathematical model P F S E C TL based on transfer learning is used in which a CNN architecture, VGG-16 trained on ImageNet dataset is used as a feature extractor for the classification task. Experimentation is performed on data collected from Alzheimer's Disease Neuroimaging Initiative (ADNI) database. The accuracy of the 3-way classification using the described method is 95.73% for the validation set. [ABSTRACT FROM AUTHOR]

In recent years, convolutional neural networks (CNNs) have achieved state-of-the-art results for many computer vision tasks. However, the traditional CNNs are computational-intensive and memory-intensive, hence they are unsuitable for the application in mobile edge computing scenarios with limited computing resources and low power consumption. The depthwise separable CNNs can significantly reduce the number of model parameters and improve the calculation speed, so it is naturally suitable for mobile edge computing applications. In this paper, we propose a Field Programmable Gate Array (FPGA)-based depthwise separable CNN accelerator with all the layers working concurrently in a pipelined fashion to improve the system throughput and performance. To implement the accelerator, we present a custom computing engine architecture to handle the dataflow between adjacent layers by using double-buffering-based memory channels. Besides, in fully connected layers, data titling technique is adopted to divide matrix multiplication from large dimension into small matrix. Finally, our proposed accelerator for depthwise separable CNN has been implemented and evaluated on Intel Arria 10 FPGA. The results of experiment indicate that the proposed depthwise separable CNN accelerator has a performance of 98.9 GOP/s and achieve up to 17.6× speed up and 29.4× low power than CPU and GPU implementations respectively. [ABSTRACT FROM AUTHOR]

Highlights • A deep-based model for solving facial expression recognition is proposed. • Propose a Salient Expression Region Descriptor (SERD) to locate expression-sensitive regions and refine deep features. • Construct an encoder-decoder framework to disentangle multiple variations (e.g. ages, races, poses, etc.) from features. • Experiments on six famous expression databases verify the effectiveness of the proposed model. Abstract Facial Expression Recognition (FER) has long been a challenging task in the field of computer vision. In this paper, we present a novel model, named Deep Attentive Multi-path Convolutional Neural Network (DAM-CNN), for FER. Different from most existing models, DAM-CNN can automatically locate expression-related regions in an expressional image and yield a robust image representation for FER. The proposed model contains two novel modules: an attention-based Salient Expressional Region Descriptor (SERD) and the Multi-Path Variation-Suppressing Network (MPVS-Net). SERD can adaptively estimate the importance of different image regions for FER task, while MPVS-Net disentangles expressional information from irrelevant variations. By jointly combining SERD and MPVS-Net, DAM-CNN is able to highlight expression-relevant features and generate a variation-robust representation for expression classification. Extensive experimental results on both constrained datasets (CK+, JAFFE, TFEID) and unconstrained datasets (SFEW, FER2013, BAUM-2i) demonstrate the effectiveness of our DAM-CNN model. [ABSTRACT FROM AUTHOR]

Deep Convolutional Neural Network (CNN) with a unique structure for combining the feature extraction and classification stages, has been considered to be a state-of-the-art computer vision technique for classification tasks. This study presents a novel and accurate method for discriminating healthy date fruit (cv. Shahani), from defective ones. Furthermore, owing to the use of deep CNN, this method is able to predict the ripening stage of the healthy dates. The proposed CNN model was constructed from VGG-16 architecture which was followed by max-pooling, dropout, batch normalization, and dense layers. This model was trained and tested on an image dataset containing four classes, namely Khalal, Rutab, Tamar, and defective date. This dataset was collected by a smartphone under uncontrolled conditions with respect to illumination and camera parameters such as focus and camera stabilization. The CNN model was able to achieve an overall classification accuracy of 96.98%. The experimental results on the suggested model demonstrated that the CNN model outperforms the traditional classification methods that rely on feature engineering for discrimination of date fruit images. [ABSTRACT FROM AUTHOR]

In aquaculture, information on fish appetite is of great significance for guiding feeding and production practices. However, most fish appetite assessment methods are inefficient and subjective. To solve these problems, in this study, an automatic method for grading fish feeding intensity based on a convolutional neural network (CNN) and machine vision is proposed to evaluate fish appetite. The specific implementation process was as follows. First, images were collected during the feeding process, and a dataset was constructed and extended using rotation, scale, and translation (RST) augmentation techniques and noise-invariant data expansion. Then, a CNN was trained on the training dataset, and the fish appetite levels were graded using the trained CNN model. Finally, the performance of the method was evaluated and compared with other quantitative and qualitative feeding intensity assessment methods. The results show that the grading accuracy reached 90%; thus, the model can be used to detect and evaluate fish appetite to guide production practices. • Fish feeding intensity was evaluated using a convolutional neural network and machine vision; • Data expansion was achieved through rotation, scale, and translation (RST) and noise-invariant data expansion; • The grading accuracy of fish feeding intensity reached 90%; • The proposed method can reduce labor costs and lay a foundation to guide production practices. [ABSTRACT FROM AUTHOR]

Highlights • We propose the use of pooling and deconvolution operations in CNNs in order to greatly increase their receptive fields. • We highlight characteristics specific to the stereo matching problem and how they can relate to the way that CNNs operate. • We use a simple feature space transformation that allow us to better evaluate the quality of the features extracted. • We present several easy to train models capable of accurate stereo matching, even without spatial regularization. Abstract Computational stereo is one of the classical problems in computer vision. Numerous algorithms and solutions have been reported in recent years focusing on developing methods for computing similarity, aggregating it to obtain spatial support and finally optimizing an energy function to find the final disparity. In this paper, we focus on the feature extraction component of stereo matching architecture and we show standard CNNs operation can be used to improve the quality of the features used to find point correspondences. Furthermore, we use a simple space aggregation that hugely simplifies the correlation learning problem, allowing us to better evaluate the quality of the features extracted. Our results on benchmark data are compelling and show promising potential even without refining the solution. [ABSTRACT FROM AUTHOR]

Abstract This paper proposes an object detection model named Feature Enriching Object Detection Framework with Weak Segmentation Loss (FEOD) based on convolutional neural networks, which is an improvement of You Only Look Once (YOLO). To overcome the shortcoming of insensitivity to small objects, a novel feature enriching module is proposed to augment the semantic information of the feature in the shallow layer within a typical deep detector. Meanwhile, Focal Loss is also introduced to our model to further improve the algorithm performance. To obtain features more suitable for object detection, a more powerful feature extractor – DetNet is used as the backbone. An mAP of 83.8 in VOC2007 test and an mAP of 82.1 in VOC2012 test are achieved with a FPS of 11.3 with a NVIDIA GTX1080Ti GPU. For a lower resolution version, we achieve an mAP of 81.9 and 80.9 respectively in VOC2007 and VOC2012 tests with FPS of 17. Comparisons with other object detection algorithms have shown that our method works well and achieves a trade-off of accuracy and speed. [ABSTRACT FROM AUTHOR]

Highlights • The paper is theoretically well founded and motivated by recent research on the physiological aspects of visual recognition. • The proposed idea is simple yet effective. It provides a practical way of training CNN for cross-domain 3D shape retrieval. • High versatility, a single framework for all the three investigated tasks. • A new selected-view matching strategy based on-line learning. • The experiments are sufficient with significantly improved results. Graphical abstract Abstract Recent vision-brain physiological experiments [65] have demonstrated that contours, and in particular, contour junctions present in the 2D images are very informative for revealing the 3D structure of the object. Inspired by this observation, we take 2D sketches (or 2D views of 3D sketches) and edge maps of 2D views of 3D models as a unified domain to train the Convolutional Neural Network (CNN). The CNN features are then used for 3D object representation. We show that the CNN can successfully learn the object structure from different types of clues. The performance of the proposed method demonstrates that the semantic gap between the 2D/3D sketches and the 3D models can be bridged without any cross-domain similarity learning. Experiments show that our approach significantly outperforms the state-of-the-art 2D/3D sketch-based 3D retrieval methods. [ABSTRACT FROM AUTHOR]

Highlights • It is for the first time the pure binarized convolutional layers are incorporated into Gabor CNNs. • HGCNs can reduce the size of required storage space of any given convolutional filters by a factor of 32. • HGCNs can achieve the comparable performance to the full-precision model. Abstract Despite great effectiveness of very wide and deep convolutional neural networks (DCNNs) in various computer vision tasks, the significant cost in terms of storage requirement of such networks impedes the deployment on computationally limited devices. Therefore, the network of resource and accuracy trade-offs always have been paid much attention in the popular DCNNs. A new deep model, Hybrid Gabor Convolutional Networks (HGCNs), is proposed in this paper. Incorporating binarized feature map and binarized filters into DCNNs in alternating way, HGCNs alleviate the accuracy loss while reduce the memory storage by 32. Also, HGCNs can be easily implemented and compatible with any other popular deep learning architecture by manipulating the basic structure elements of the DCNNs (i. e. , the convolution operator) based on the binarization filter. Most importantly, HGCNs achieve a comparable performance while largely reducing the storage memory compared with state-of-the-art networks such as Resnet. The source code will be public soon. [ABSTRACT FROM AUTHOR]

Abstract Deep convolutional neural networks (DCNNs) have been used to achieve state-of-the-art performance on many computer vision tasks (e.g., object recognition, object detection, semantic segmentation) thanks to a large repository of annotated image data. Large labeled datasets for other sensor modalities, e.g., multispectral imagery (MSI), are not available due to the large cost and manpower required. In this paper, we adapt state-of-the-art DCNN frameworks in computer vision for semantic segmentation for MSI imagery. To overcome label scarcity for MSI data, we substitute real MSI for generated synthetic MSI in order to initialize a DCNN framework. We evaluate our network initialization scheme on the new RIT-18 dataset that we present in this paper. This dataset contains very-high resolution MSI collected by an unmanned aircraft system. The models initialized with synthetic imagery were less prone to over-fitting and provide a state-of-the-art baseline for future work. [ABSTRACT FROM AUTHOR]

Convolutional neural network (CNN) has demonstrated its superior ability to achieve amazing accuracy in computer vision field. Nevertheless, for practical domain-specific image recognition tasks, it still remains difficult to obtain massive high-quality labeled datasets due to the strong requirements for extensive, tedious manual processing. Inspired by the well-known observation that human brain can accurately recognize objects without relying on massive congeneric examples, we propose a novel deep variance network (DVN) to further enhance the generalization ability of CNN in this paper, which could still produce higher recognition accuracy even with unbalanced training datasets than original CNN. The key idea of our DVN is built upon the intrinsic exploitation of inter-class homogeneity and intra-class heterogeneity. Towards such goal, we make the first attempt to incorporate a hierarchical Bayesian model into the powerful CNN framework, which can transfer the joint feature distribution from certain object’s complete training dataset to other object’s incomplete training dataset in an iterative way. In each training cycle, the CNN-resulted features are clustered into discrimination-related subspaces to guide the learning and adaptive adjustment of homogeneity and heterogeneity over unbalanced training datasets. In practice, we furnish several state-of-the-art deep networks with our proposed DVN, and conduct extensive experiments and comprehensive evaluations over CIFAR-10, MNIST, and SVHN benchmarks. The experiments have shown that, most of the furnished deep networks can benefit from our DVN, wherein they gain at most 6.9% accuracy improvement over CIFAR-10 benchmark, 52.83% error reduction over MNIST benchmark, and an improvement of 6.2% over SVHN datasets. [ABSTRACT FROM AUTHOR]

Pedestrian detection and tracking have become an important field in the computer vision research area. This growing interest, started in the last decades, might be explained by the multitude of potential applications that could use the results of this research field, e.g. robotics, entertainment, surveillance, care for the elderly and disabled, and content-based indexing. In this survey paper, vision-based pedestrian detection systems are analysed based on their field of application, acquisition technology, computer vision techniques and classification strategies. Three main application fields have been individuated and discussed: video surveillance, human-machine interaction and analysis. Due to the large variety of acquisition technologies, this paper discusses both the differences between 2D and 3D vision systems, and indoor and outdoor systems. The authors reserved a dedicated section for the analysis of the Deep Learning methodologies, including the Convolutional Neural Networks in pedestrian detection and tracking, considering their recent exploding adoption for such a kind systems. Finally, focusing on the classification point of view, different Machine Learning techniques have been analysed, basing the discussion on the classification performances on different benchmark datasets. The reported results highlight the importance of testing pedestrian detection systems on different datasets to evaluate the robustness of the computed groups of features used as input to classifiers. [ABSTRACT FROM AUTHOR]

Depth information has been discovered to improve the performance of computer vision tasks, such as semantic segmentation and object recognition. However, careful acquisition of depth data needs highly developed depth sensors which are expensive. As a classic computer vision task, depth estimation from a single image has obtained promising results based on supervised learning methods. In this paper, we investigate the extension of color images with corresponding deep-regressed depth images in boosting the performance of semantic segmentation. Furthermore, the usage of combining color channels with the estimated depth or the ground truth depth channel is compared. Specifically, there are two stages in our work. Firstly, we adopt the framework of convolutional neural networks (CNN) for the depth estimation by combing the global depth network and the depth gradient network. After refining based on these two networks, the depth image map can be estimated in a deep-regressed manner. Secondly, after augmenting the color images with the predicted depth images, fully convolutional networks (FCN) are further used to implement the pixel-level semantic labeling. In the experiments, we employ two popular RGBD datasets, i.e., SUNRGBD and NYUDv2, for 37 and 40-class semantic segmentation, respectively. By comparing with the ground truth depth images, experimental results demonstrate that the networks trained on the estimated depth images can achieve comparable performance on facilitating the accuracy of semantic segmentation task. [ABSTRACT FROM AUTHOR]

In this paper, we propose a semi-supervised facial landmark detection algorithm (SEMI) based on convolutional neural network (CNN), which can detect facial components and landmarks simultaneously. Unlike previous coarse-to-fine algorithms, our model does not need extra input such as initial landmark prediction. It also solves the occlusion problem of large area by detecting the visible facial components while existing face detectors failed to detect faces. Semi-supervised learning algorithm is also an effective data augmentation method. In our experiment, each image has two types of ground truth, one is bounding-box related (classification and coordinates) and the other is landmark coordinates inside the bounding-box. The supervised data have both two types of ground truth while the semi-supervised data only have the bounding-box. Our model was trained by the merge of two parts of data. Extensive evaluations on Helen, LFPW and 300-W show that our algorithm is able to complete the landmark task and performs better than many state-of-the-art facial landmark detecting algorithms. [ABSTRACT FROM AUTHOR]

Recent years have witnessed the great success of convolutional neural network (CNN) based models in the field of computer vision. CNN is able to learn hierarchically abstracted features from images in an end-to-end training manner. However, most of the existing CNN models only learn features through a feedforward structure and no feedback information from top to bottom layers is exploited to enable the networks to refine themselves. In this paper, we propose a Learning with Rethinking algorithm. By adding a feedback layer and producing the emphasis vector, the model is able to recurrently boost the performance based on previous prediction. Particularly, it can be employed to boost any pre-trained models. This algorithm is tested on four object classification benchmark datasets: CIFAR-100, CIFAR-10, MNIST-background-image and ILSVRC-2012 dataset, and the results have demonstrated the advantage of training CNN models with the proposed feedback mechanism. [ABSTRACT FROM AUTHOR]

In the last few years, deep learning has led to very good performance on a variety of problems, such as visual recognition, speech recognition and natural language processing. Among different types of deep neural networks, convolutional neural networks have been most extensively studied. Leveraging on the rapid growth in the amount of the annotated data and the great improvements in the strengths of graphics processor units, the research on convolutional neural networks has been emerged swiftly and achieved state-of-the-art results on various tasks. In this paper, we provide a broad survey of the recent advances in convolutional neural networks. We detailize the improvements of CNN on different aspects, including layer design, activation function, loss function, regularization, optimization and fast computation. Besides, we also introduce various applications of convolutional neural networks in computer vision, speech and natural language processing. [ABSTRACT FROM AUTHOR]

In this work, we propose a novel latent fingerprint enhancement method based on FingerNet inspired by recent development of Convolutional Neural Network (CNN). Although CNN is achieving superior performance in many computer vision tasks from low-level image processing to high-level semantic understanding, limited attention has been paid in fingerprint community. The proposed FingerNet has three major parts: one common convolution part shared by two different deconvolution parts, which are the enhancement branch and the orientation branch. The convolution part is to extract fingerprint features particularly for enhancement purpose. The enhancement deconvolution branch is employed to remove structured noise and enhance fingerprints as its task. The orientation deconvolution branch performs the task of guiding enhancement through a multi-task learning strategy. The network is trained in the manner of pixels-to-pixels and end-to-end learning, that can directly enhance latent fingerprint as the output. We also study some implementation details such as single-task learning, multi-task learning, and the residual learning. Experimental results of the FingerNet system on latent fingerprint dataset NIST SD27 demonstrate effectiveness and robustness of the proposed method. [ABSTRACT FROM AUTHOR]

Image super-resolution (SR) has attracted great attention due to its wide practical applications. The objective of SR is to reconstruct high-resolution images from low-resolution ones. By virtue of the great success in computer vision fields achieved by deep learning approach, especially the convolutional neural networks (CNNs), it is a good way to tackle the SR problem using CNNs. In this paper, a 7-layer dilated convolutional neural network (DCNN) with skip-connections is proposed to recover the high-resolution image from the interpolated low-resolution image. Dilated convolutions allow us to arbitrarily control the field-of-view (FOV) of the network. To the best of our knowledge, this is the first study to apply dilated convolution and skip-connections for image super-resolution. We explore different FOV of the network by adjusting the dilation rate and different combination of contextual information using skip-connections to achieve a trade-off between performance and speed. We also design a cascade model (CDCNN) to address the different magnification factors problems. Comparisons with state-of-the-art methods, experimental results demonstrate that the proposed model achieves a notable improvement in terms of both quantitative and qualitative measurements. [ABSTRACT FROM AUTHOR]

Like other problems in computer vision, offline handwritten Chinese character recognition (HCCR) has achieved impressive results using convolutional neural network (CNN)-based methods. However, larger and deeper networks are needed to deliver state-of-the-art results in this domain. Such networks intuitively appear to incur high computational cost, and require the storage of a large number of parameters, which render them unfeasible for deployment in portable devices. To solve this problem, we propose a Global Supervised Low-rank Expansion (GSLRE) method and an Adaptive Drop-weight (ADW) technique to solve the problems of speed and storage capacity. We design a nine-layer CNN for HCCR consisting of 3755 classes, and devise an algorithm that can reduce the network’s computational cost by nine times and compress the network to 1/18 of the original size of the baseline model, with only a 0.21% drop in accuracy. In tests, the proposed algorithm can still surpass the best single-network performance reported thus far in the literature while requiring only 2.3MB for storage. Furthermore, when integrated with our effective forward implementation, the recognition of an offline character image takes only 9.7 ms on a CPU. Compared with the state-of-the-art CNN model for HCCR, our approach is approximately 30 times faster, yet 10 times more cost efficient. [ABSTRACT FROM AUTHOR]

Multi-label image classification is a challenging problem in computer vision. Motivated by the recent development in image classification performance using Deep Neural Networks, in this work, we propose a flexible deep Convolutional Neural Network (CNN) framework, called Local-Global-CNN (LGC), to improve multi-label image classification performance. LGC consists of firstly a local level multi-label classifier which takes object segment hypotheses as inputs to a local CNN. The output results of these local hypotheses are aggregated together with max-pooling and then re-weighted to consider the label co-occurrence or interdependencies information by using a graphical model in the label space. LGC also utilizes a global CNN that is trained by multi-label images to directly predict the multiple labels from the input. The predictions of local and global level classifiers are finally fused together to obtain MAP estimation of the final multi-label prediction. The above LGC framework could benefit from a pre-train process with a large-scale single-label image dataset, e.g., ImageNet. Experimental results have shown that the proposed framework could achieve promising performance on Pascal VOC2007 and VOC2012 multi-label image dataset. [ABSTRACT FROM AUTHOR]

Regularization is an essential technique discussed in an attempt to solve the overfitting problem in deep convolutional neural networks (CNNs). In this paper, we proposed a novel companion objective function as a regularization strategy to boost the classification performance in deep CNNs. Three aspects of this companion objective function are studied. Firstly, we proposed two kinds of auxiliary supervision for convolutional filters and non-linear activations respectively in the companion objective function. Both of them enhanced the performance by aleviating the overfitting problem and auxiliary supervision for non-linear activations provided more efficiency. Secondly, regularization of auxiliary supervision in the pre-training phrase is discussed. With the assistance of auxiliary supervision, CNNs could obtain a more favorable initialization for end-to-end supervised fine-tuning. Finally, this companion objective function is verified to be compatible with other regularization strategies such as dropout and data augmentation. Experimental results on benchmark datasets (CIFAR-10 and CIFAR-100) demonstrated advantages of our proposed companion objective function as a regularization approach. [ABSTRACT FROM AUTHOR]

Action recognition (AR) is one of the most important tasks in video analysis and computer vision. Recently a large number of related methods have been proposed. While most of these methods are investigated on AR datasets collected from the visible spectrum, the AR problem under infrared scenarios still has not attracted much attention. There is even few public infrared datasets available for supporting the fundamental evaluation requirements of this research. To this issue, this work aims to emphasize the importance of the infrared AR problem in applications and arouse researchers' attention on this task. Specifically, we construct a new Inf rared A ction R ecognition ( InfAR ) dataset captured at different times, including in summer and winter, and explore how discriminable actions in our InfAR dataset are with the state-of-the-art pipelines based on low-level features and deep convolutional neural network (CNN), respectively. Our results reveal: (1) In all, dense trajectory feature can achieve the best performance while the appearance features, e.g., HOG, have relatively poorer performance; (2) the encoding method of vector of locally aggregated descriptors is evidently better than that of the widely-used Fisher Vector; (3) the late fusion facilitates a better performance than early fusion; (4) action videos captured in winter is more discriminable than in summer; (5) compared to appearance information, the motion information is more essential for infrared action recognition and utilizing this information through deep CNN can improve greatly the performance. The best performance achieved on our dataset is 76.66% (Average Precision), leaving a reasonable space for further exploring the insights underlying such type of infrared AR problem and accordingly designing proper techniques to further promote the performance on this specifically constructed InfAR dataset. [ABSTRACT FROM AUTHOR]

Scene text detection and scene segmentation are meaningful tasks in the computer vision field. Could the semantic scene segmentation assist scene text detection in any degree? For example, can we expect the probability of a region being text is low if its surrounding segment, i.e., its context, is labeled as sky? In this paper, we have a positive answer by constructing a scene context-based text detection model. In this model, we use texton features and a fully-connected conditional random field (CRF) to estimate pixel-level scene class׳s probability to be considered as image׳s context feature. Meanwhile, maximally stable extremal regions (MSERs) are extracted, integrated and extended as image patches of character candidates. Then, each image patch is fed to a simple two-layer convolutional neural network (CNN) to automatically extract its character feature. The averaged context feature of the corresponding patch is considered as the patch׳s context feature. The character feature and context feature are fused as the input into a support vector machine for text/non-text determination. Finally, as a post-processing, neighboring text regions are grouped hierarchically. The performance evaluation on ICDAR2013 and SVT databases, as well as a preliminary evaluation on a patch-level database, proves that the scene context can improve the performance of scene text detection. Moreover, the comparative study with state-of-the-art methods shows the top-level performance of our method. [ABSTRACT FROM AUTHOR]

Flow pattern identification is an important topic in multiphase flow research. To overcome the subjectivity of manual identification, intelligent identification of flow patterns has attracted much attention in recent years. Both traditional machine learning methods and deep learning methods have been utilized in this field. However, traditional machine learning methods lack accuracy, and existing deep learning methods mostly rely on artificial feature extraction or complex preprocessing. In this paper, we propose a new method with high accuracy and low preprocessing dependency to solve these issues. We modify ResNet, which has proven high performance in computer vision, to fit the data collected by the wire-mesh sensor system (WMS). Due to its outstanding feature extraction ability, the new model can reach high accuracy with simple normalization as the preprocessing step. Additionally, the model can directly process data at various scales without retraining or rebuilding, which gives it high usability and economic value. The experimental results show that the accuracy of this method can reach 99.58% on our dataset. • A wire-mesh sensor system is applied to flow pattern recognition for the first time. • The method reaches a high accuracy without complex preprocessing. • Neural network models are compatible with data scales. • The influence of the data scale on the recognition effect is analyzed. [ABSTRACT FROM AUTHOR]

In this paper we present our solution to the 300 Faces in the Wild Facial Landmark Localization Challenge. We demonstrate how to achieve very competitive localization performance with a simple deep learning based system. Human study is conducted to show that the accuracy of our system has been very close to human performance. We discuss how this finding would affect our future direction to improve our system. [ABSTRACT FROM AUTHOR]

Extracting high level information from digital images and videos is a hard problem frequently faced by the computer vision and machine learning communities. Modern surveillance systems can monitor people, cars or objects by using computer vision methods. The objective of this work is to propose a method for identifying soft-biometrics, in the form of clothing and gender, from images containing people, as a previous step for further identifying people themselves. We propose a solution to this classification problem using a Convolutional Neural Network, working as an all-in-one feature extractor and classifier. This method allows the development of a high-level end-to-end clothing/gender classifier. Experiments were done comparing the CNN with hand-designed classifiers. Also, two different operating modes of CNN are proposed and compared each other. The results obtained were very promising, showing that is possible to extract soft-biometrics attributes using an end-to-end CNN classifier. The proposed method achieved a good generalization capability, classifying the three different attributes with good accuracy. This suggests the possibility to search images using soft biometrics as search terms. [ABSTRACT FROM AUTHOR]

• Classification of roughness to unmeasured surfaces is firstly proposed. • No need to artificially design any indicators. • The method used in this article is robust to the light source environment. • This method is combined with machine vision system. At present, machine vision roughness detection mostly needs to design roughness related indexes based on images, and the index design has human intervention and is heavily dependent on the light source environment. To solve this problem, the paper classifies the surface roughness based on the deep convolutional neural network method, which can realize the roughness detection without index design. The most important thing is that the detection method has good light source robustness under different light source environments. The study adopts an end-to-end image analysis method, by means of image enhancement pre-processing of a small number of source images, after multi-layer convolution and pooling operations, as well as comprehensive processing of fully connected and classification layers, the convolutional kernel can automatically extract the features of the image, and finally the surface roughness obtained by vertical disc cutter milling can be classified and predicted. In addition, based on the experimental light source environment with two different luminance of night and day, by comparing with the technically mature ResNet50 and DenseNet121 convolutional neural network models, the deep convolutional neural network Xception model not only has high roughness classification accuracy, but also has more light source environment robustness. This method makes the online measurement of visual roughness possible. [ABSTRACT FROM AUTHOR]

[Display omitted] • A method combing NIRS and a computer vision system were proposed in this study. • The method (a sensor fusion strategy) is used for grading Keemun black tea. • A CNN model was constructed to extract NIRS features. • The accuracy of the proposed strategy was higher than that of a single-sensor strategy. • The sensor fusion strategy nearly attained 100% in grading black tea samples. Keemun black tea is classified into 7 grades according to the difference in its quality. The appearance and flavour are crucial indicators of its quality. This research demonstrates a rapid grading method of jointly using near-infrared reflectance spectroscopy (NIRS) and computer vision systems (CVS) to evaluate the flavour and appearance quality of tea. A Bruker MPA Fourier Transform near-infrared spectrometer was used to record the spectrum of samples. A computer vision system was used to capture the image of tea leaves in an unobstructed manner. 80 tea samples for each grade were analyzed. The performance of four NIRS feature extraction methods (principal component analysis, local linear embedding, isometric feature mapping, and convolutional neural network (CNN)) was compared in this study. Histograms of six geometric features (leaf width, leaf length, leaf area, leaf perimeter, aspect ratio, and rectangularity) of different tea samples were used to describe their appearance. A feature-level fusion strategy was used to combine softmax and artificial neural networks (ANN) to classify NIRS and CVS features. The results indicated that for an individual NIRS signal, CNN achieved the highest classification accuracy with the softmax classification model. The histograms of the combined shape features indicated that when the softmax classification model was used, the classification accuracy was also higher than ANN. The fusion of NIRS and CVS features proved to be the optimal combination; the accuracy of calibration, validation and testing sets increased from 99.29%, 96.67% and 98.57% (when the optimal features from a single-sensor were used) to 100.00%, 99.29% and 100.00% (when features from multiple-sensors were used). This study revealed that the combination of NIRS and CVS features can be a useful strategy for classifying black tea samples of different grades. [ABSTRACT FROM AUTHOR]

• Recent studies show the potential of deep learning for binary classification of skin lesions. • This research proposes an advanced deep convolution neural network model for multi-class classification of skin lesions. • The proposed model is designed with several layers, and multiple filter sizes, but fewer filters and parameters. • It is evaluated by the experimental results on the ISIC-19 database in terms of sensitivity, specificity, and other metrics. • It outperforms state-of-the-art algorithms, exhibiting 0.991 AUROC. As an analytic tool in medicine, deep learning has gained great attention and opened new ways for disease diagnosis. Recent studies validate the effectiveness of deep learning algorithms for binary classification of skin lesions (i.e., melanomas and nevi classes) with dermoscopic images. Nonetheless, those binary classification methods cannot be applied to the general clinical situation of skin cancer screening in which multi-class classification must be taken into account. The main objective of this research is to develop, implement, and calibrate an advanced deep learning model in the context of automated multi-class classification of skin lesions. The proposed Deep Convolutional Neural Network (DCNN) model is carefully designed with several layers, and multiple filter sizes, but fewer filters and parameters to improve efficacy and performance. Dermoscopic images are acquired from the International Skin Imaging Collaboration databases (ISIC-17, ISIC-18, and ISIC-19) for experiments. The experimental results of the proposed DCNN approach are presented in terms of precision, sensitivity, specificity, and other metrics. Specifically, it attains 94 % precision, 93 % sensitivity, and 91 % specificity in ISIC-17. It is demonstrated by the experimental results that this proposed DCNN approach outperforms state-of-the-art algorithms, exhibiting 0.964 area under the receiver operating characteristics (AUROC) in ISIC-17 for the classification of skin lesions and can be used to assist dermatologists in classifying skin lesions. As a result, this proposed approach provides a novel and feasible way for automating and expediting the skin lesion classification task as well as saving effort, time, and human life. [ABSTRACT FROM AUTHOR]

• We propose a novel neural network architecture for unconstrained text recognition. • Our proposed architecture uses only the highly efficient convolutional primitives • We achieve state-of-the-art results on seven public benchmark datasets. • Our proposed model has won the ICFHR2018 Competition on Automated Text Recognition. Unconstrained text recognition is an important computer vision task, featuring a wide variety of different sub-tasks, each with its own set of challenges. One of the biggest promises of deep neural networks has been the convergence and automation of feature extractors from input raw signals, allowing for the highest possible performance with minimum required domain knowledge. To this end, we propose a data-efficient, end-to-end neural network model for generic, unconstrained text recognition. In our proposed architecture we strive for simplicity and efficiency without sacrificing recognition accuracy. Our proposed architecture is a fully convolutional network without any recurrent connections trained with the CTC loss function. Thus it operates on arbitrary input sizes and produces strings of arbitrary length in a very efficient and parallelizable manner. We show the generality and superiority of our proposed text recognition architecture by achieving state-of-the-art results on seven public benchmark datasets, covering a wide spectrum of text recognition tasks, namely: Handwriting Recognition, CAPTCHA recognition, OCR, License Plate Recognition, and Scene Text Recognition. Our proposed architecture has won the ICFHR2018 Competition on Automated Text Recognition on a READ Dataset. [ABSTRACT FROM AUTHOR]

Piping and Instrumentation Diagrams (P&IDs) are the most commonly used engineering drawings to describe components and their relationships, and they are one of the most important inputs for data analysis in Nuclear Power Plants (NPP). In traditional analysis, the information related to the components is extracted manually from the P&IDs. This usually takes large amounts of effort and is error-prone. With the rapid development in the area of computer vision and deep learning, automatically detecting components and their relationships becomes possible. In this paper, we aim to use the latest neural network models to automatically extract information on components and their identifications, from the P&IDs in NPPs. We use a Faster Regional Convolutional Neural Networks (Faster RCNN) architecture called ResNet-50, to detect the components in the P&IDs. Compared to common object detection, object detection for P&IDs poses unique challenges to these methods. For example, the P&IDs symbols are much smaller than the background, and detecting such small objects remains a challenging task for modern neural networks. To address these challenges, we 1) propose several techniques for data augmentation that effectively solve the problem of training data shortage, and 2) propose a feature grouping strategy for detecting components with distinct features. Besides, we introduce a SegLink model for text detection, which can automatically extract components' identifications from P&IDs. We also develop a method for building a data structure to reflect the relationships between components (e.g., to which pipe a component is connected, or what are the downstream or upstream components of one specific component) based on the extracted information. This data structure can be further used for plant safety analysis, and operation and maintenance cost optimization. Sensitivity analysis and comparison with other Convolutional Neural Networks (CNNs) are performed. The results of these analyses are also discussed in this paper. This analysis framework has been tested on the P&IDs from a commercial NPP. The Average Precision for components, which is used to measure the performance of the proposed method, is about 98%. The success rates of component-text mapping and component-pipe mapping are 270/275 and 319/319, respectively. It is worth noting that this framework is generic and can also be applied to P&IDs of non-nuclear industries. [ABSTRACT FROM AUTHOR]

• We propose a unified model for recognizing human actions in static images. It explicitly investigates the body structure information as well as integrates the body structure exploration and action classification tasks into a unified model. Moreover, we design a twostep learning technique, where keypoint estimation provides intermediate supervision for learning human action representations. • We design two body structure cues, SBPs and LAD, to fully explore the structure information of human bodies from the local and global perspectives. • In order to construct body parts with different scales in unconstrained images, we propose a technique to use human keypoint heatmaps to generate scale adaptive SBPs, which extract fine-grained local human features. Moreover, we propose a technique to automatically determine the most discriminative body part of each action category for identifying the ongoing action. In order to extract global hightlevel body structure features, we propose the LAD to model the spatial angle relationship of pairs of human limbs. The LAD is more robust and achieves better performance compared with the distance based skeleton descriptor. • We evaluate our model on two challenging image-based action datasets, and the results show that our method achieves the state-of-the-art performance. Although Convolutional Neural Networks (CNNs) have made substantial improvements in many computer vision tasks, there remains room for improvements in image-based action recognition due to the limited capability to exploit the body structure information.In this work, we propose a unified deep model to explicitly explore body structure information and fuse multiple body structure cues for robust action recognition in images.In order to fully explore the body structure information, we design the Body Structure Exploration sub-network.It generates two novel body structure cues, Structural Body Parts and Limb Angle Descriptor, which capture structure information of human bodies from the global and local perspectives respectively. And then, we design the Action Classification sub-network to fuse the predictions from multiple body structure cues to obtain precise results. Moreover, we integrate the two sub-networks into a unified model by sharing the bottom convolutional layers, which improves the computational efficiency in both training and testing stages. We comprehensively evaluate our network on the challenging image-based human action datasets, Pascal VOC 2012 Action and Stanford40. Our approach achieves 93.5% and 93.8% mAP respectively, which outperforms all recent approaches in this field. [ABSTRACT FROM AUTHOR]

Enclosed spaces are common in built structures but pose a challenge to many forms of manual or robotic surveying and maintenance tasks. Part of this challenge is to train robot systems to understand their environment without human intervention. This paper presents a method to automatically classify features within a closed void using deep learning. Specifically, the paper considers a robot placed under floorboards for the purpose of autonomously surveying the underfloor void. The robot uses images captured using an RGB camera to identify regions such as floorboards, joists, air vents and pipework. The paper first presents a standard mask regions convolutional neural network approach, which gives modest performance. The method is then enhanced using a two-stage transfer learning approach with an existing dataset for interior scenes. The conclusion from this work is that, even with limited training data, it is possible to automatically detect many common features of such areas. • A method for semantic segmentation in closed spaces • Database of underfloor images are captured and segmented using a mask R-CNN. • Two-stage transfer learning and other optimisations presented to maximise accuracy • Case-study application to the detection of floorboards, joists, vents, pipes, etc for automated insulation spraying • Mapping of segmented data labels to 3D point cloud data [ABSTRACT FROM AUTHOR]

• An expert and intelligent system is developed for detonator code recognition. • Multi-scale concatenation of convolutions accelerates speed in inference. • Progressive self-attention boosts performance in recognition. • Release a publicly available annotated detonator image dataset. • An end-to-end recognition accuracy of 99.18% is achieved. Detonators are hazardous and must be strictly controlled to strengthen public security since they contain explosive. To deal with the stressful management of detonators by manually recording detonator codes, an expert and intelligent system for detonator code recognition is an alternative management scheme for automatically recording detonator codes. How to achieve accurate and reliable recognition performance is also a challenging problem in such an intelligent system. In this study, we develop an intelligent vision-based system based on deep learning for identifying detonator codes. Specifically, we design a detonator image acquisition subsystem to construct the detonator code image dataset, and an intelligent image processing subsystem named as automatic detonator code recognition network (ADCR-Net). The ADCR-Net is a pipeline cascading a deep localization network and a deep recognition network. The multi-scale concatenation block and the features integration module are proposed for the localization network of the ADCR-Net to improve the features expressions. The multi-level progressive self-attention block is put forward for the recognition network of the ADCR-Net to help with focusing on multi-level activation maps. Also, a multi-label loss function for the recognition network is designed to balance the significance of the classifiers of the intelligent system during training. Experiments on our released dataset demonstrate the effectiveness and efficiency of the proposed expert and intelligent system, which achieves 99.18% accuracy in end-to-end recognition. Our work provides an effective resolution to the expert and intelligent system for automatic detonator code recognition. Also, we provide an alternative framework to deal with the tradeoff between inference time and recognition accuracy in deep learning which is becoming a popular method in many expert and intelligent systems. Furthermore, the released dataset is available for distribution to the related researchers to develop their expert systems in the field of industrial optical character recognition. [ABSTRACT FROM AUTHOR]

Kansei evaluation is crucial to the process of Kansei engineering. However, traditional methods are subjective and random. In order to eliminate the differences of individual evaluation criteria in product Kansei attributes evaluation, and further improve the evaluation efficiency, a novel automatic evaluation and labeling architecture for product Kansei attributes was proposed in this paper based on Convolutional Neural Networks (CNNs). The architecture consists of two modules: (1) Target detection module (Faster R-CNN was taken as an example), (2) Fine-Grained classification module (DFL-CNN was taken as an example). A case study was provided to validate the proposed architecture. The proposed architecture transformed design evaluation tasks into the recognition and classification tasks. The experiments achieved 98.837%, 96.899%, 86.047%, and 81.008% accuracy in the binary, triple, and two five-classification tasks, respectively. Our results proved the feasibility of using computer vision to mimic human vision for the automatic evaluation of Kansei attributes. [ABSTRACT FROM AUTHOR]

Manual processing of Ground Penetrating Radar (GPR) images is a very time-intensive task. The authors proposed a novel computer vision-based method for automatic detection of rebars in complex GPR images in highly deteriorated concrete bridge decks. The proposed detection model consists of a fine-tuned Histogram of Oriented Gradients feature descriptor, a Multi-Layer Perceptron for classification, and a post processing algorithm for eliminating false detections and labeling rebar in Region of Interest. State-of-art results are obtained on testing the method on real bridge deck GPR data and comparing the results with RADAN software. Overall accuracy of 89.4% is obtained on URIGPRv1.0 dataset, which is introduced in this paper. The proposed method is 54.35% more accurate comparing to the results obtained by RADAN software. The proposed classifier outperformed accuracy of a 3-layer convolutional neural network by 11.9%. • Proposing a novel detection chain yields a new state-of-the-art result in rebar detection in GPR B-scan images • Optimizing HOG/LBPH feature computation and MLP model parameters for maximizing F-Measure in classification task • Developing a post-processing algorithm for minimizing false RoIs, removing non-top layer RoI, and extracting coordinates • Introducing a GPR dataset with 8,000 records, named URIGPRv1.0, cropped from real concrete bridge deck GPR B-scan images. • A computer program, named URICAB, is developed for automating interpretation of GPR data [ABSTRACT FROM AUTHOR]

With the development of artificial intelligence and deep learning, image sentiment analysis has become a hotspot in computer vision and attracts more attention. Most of the existing methods focus on identifying the emotions by studying complex models or robust features from the whole image, which neglects the influence of object semantics on image sentiment analysis. In this paper, we propose a novel object semantics sentiment correlation model (OSSCM), which is based on Bayesian network, to guide the image sentiment classification. OSSCM is constructed by exploring the relationships between image emotions and the object semantics combination in the images, which can fully consider the effect of object semantics for image emotions. Then, a convolutional neural networks (CNN) based visual sentiment analysis model is proposed to analyze image sentiment from visual aspect. Finally, three fusion strategies are proposed to realize OSSCM enhanced image sentiment classification. Experiments on public emotion datasets FI and Flickr_LDL demonstrate that our proposed image sentiment classification method can achieve good performance on image emotion analysis, and outperform state of the art methods. [ABSTRACT FROM AUTHOR]

• Deep neural network is used to detect fuel assembly defects. • Faster R-CNN with inception-v2 works best in the detection task of fuel assembly defects. • The proposed detection method achieves 0.98 TPR against 0.1 FPR. Fuel assembly may be damaged during production, transportation and operation, which affects the safety of the nuclear power plant directly. Manual visual inspection is often used to identify the defect of fuel assemblies. This method is a time-consuming, tedious and subjective process. The accurate detection is highly depended on personnel experience and technician's attention. Although some traditional defect detection algorithms based on computer vision have been used, their performances with respect to fuel assembly detection are not good as expected. In this work, based on the state-of-the-art deep neural network, the Faster R-CNN is proposed to detect the scratch of nuclear fuel assembly. The proposed framework achieves 0.98 TPR against 0.1 FPR that is significantly higher than most of current appearance detection method. [ABSTRACT FROM AUTHOR]

• A deep learning method based on CNN and LSTM was used to recognise pig aggression. • This study directly process video episodes rather than individual frames. • CNN features were input into LSTM framework to extract spatial-temporal features. • The proposed method could recognise aggressive episodes with an accuracy of 97.2%. • A frame skipping approach can improve the accuracy to 98.4% and halve running time. Aggression is considered as a major animal welfare problem in commercial pig farming. The aim of this study is to develop a deep learning method based on convolutional neural network (CNN) and long short-term memory (LSTM) to recognise aggressive episodes of pigs. Compared to previous studies of pig behaviours based on deep learning, this study directly process video episodes rather than individual frames. In the experiment, nursery pigs (8/pen) were mixed for 3 days and then 8 h of video was recorded in each day. From these videos, 600 aggressive 2 s-episodes were manually selected and then augmented into 2400 episodes by using horizontal, vertical and diagonal mirroring. From the videos, 2400 non-aggressive 2 s-episodes were also manually selected. 80% of the data were randomly allocated as training set and the remaining 20% as validation set. Firstly, the CNN architecture VGG-16 was used to extract spatial features. These features were then input into LSTM framework to further extract temporal features. Through fully connected layer, the prediction function Softmax was finally used to determine if the current episode is aggression or non-aggression. Using the proposed method, aggressive episodes could be recognised with an accuracy of 97.2%. This result indicates that this method can be used to recognise aggressive episodes of pigs. Additionally, this paper further investigates the validity of this method under the conditions of skipping frames and reducing the episode length. The results show that a frame skipping approach whereby 30 fps is reduced into 15 fps within each 2 s-episode can improve the accuracy into 98.4% and halve the total running time. [ABSTRACT FROM AUTHOR]

The process of identifying and bringing to the fore people's unsafe behaviour is a core function of implementing a behaviour-based safety (BBS) program in construction. This can be a labour-intensive and challenging process but is needed to enable people to reflect and learn about how their unsafe actions can jeopardise not only their safety but that of their co-workers. With advances being made in computer vision, the capability exists to automatically capture and identify unsafe behaviour and hazards in real-time from two-dimensional (2D) digital images/videos. The corollary developments in computer vision have stimulated a wealth of research in construction to examine its potential application to practice. Hindering the application of computer vision in construction has been its inability to accurately, and generalise the detection of objects. To address this shortcoming, developments in deep learning have provided computer vision with the ability to improve the accuracy, reliability and ability to generalise object detection and therefore its usage in construction. In this paper we review the developments of computer vision studies that have been used to identify unsafe behaviour from 2D images that arises on construction sites. Then, in light of advances made with deep learning, we examine and discuss its integration with computer vision to support BBS. We also suggest that future computer-vision research should aim to support BBS by being able to: (1) observe and record unsafe behaviour; (2) understand why people act unsafe behaviour; (3) learn from unsafe behaviour; and (4) predict unsafe behaviour. [ABSTRACT FROM AUTHOR]

• The first large labeled methane leak video dataset (GasVid) was collected. • The first deep-learning-based methane detection method (GasNet) was developed. • The first detection probability curve of automatic detection system was generated. • Our automated technology simplifies the leak detection survey with high accuracy. • The deep learning model was optimized to achieve highest detection accuracy. In a climate-constrained world, it is crucial to reduce natural gas methane emissions, which can potentially offset the climate benefits of replacing coal with gas. Optical gas imaging (OGI) is a widely-used method to detect methane leaks, but is labor-intensive and cannot provide leak detection results without operators' judgment. In this paper, we develop a computer vision approach for OGI-based leak detection using convolutional neural networks (CNN) trained on methane leak images to enable automatic detection. First, we collect ∼1 M frames of labeled videos of methane leaks from different leaking equipment, covering a wide range of leak sizes (5.3–2051.6 g CH 4 /h) and imaging distances (4.6–15.6 m). Second, we examine different background subtraction methods to extract the methane plume in the foreground. Third, we then test three CNN model variants, collectively called GasNet, to detect plumes in videos. We assess the ability of GasNet to perform leak detection by comparing it to a baseline method that uses an optical-flow based change detection algorithm. We explore the sensitivity of results to the CNN structure, with a moderate-complexity variant performing best across distances. The generated detection probability curves show that the detection accuracy (fraction of leak and non-leak images correctly identified by the algorithm) can reach as high as 99%, the overall detection accuracy can exceed 95% across all leak sizes and imaging distances. Binary detection accuracy exceeds 97% for large leaks (∼710 g CH 4 /h) imaged closely (∼5–7 m). The GasNet-based computer vision approach could be deployed in OGI surveys for automatic vigilance of methane leak detection with high accuracy in the real world. [ABSTRACT FROM AUTHOR]

The similarity of gray characteristics and shapes of the defects, stems, and calyxes in apple images is a difficult challenge for automatic identification and detection of apple defects in the machine vision field. This paper attempts towards finding an automatic and efficient way to identify apple defects. An automatic detection method is proposed for apple defects based on laser-induced light backscattering imaging and convolutional neural network (CNN) algorithm. Laser backscattering spectroscopic images of apples are obtained using semiconductor laser. We take preprocessing steps to get the finest image dataset for CNN. An AlexNet model with an 11-layer structure is established and trained to identify apple defects. We analyze how well the model does with the recognition effects of apple defects. The proposed CNN model for the detection of apple defects achieves a higher recognition rate of 92.5%, and the accuracy is better than conventional machine learning algorithms. The method based on laser backscattering imaging analysis and CNN theory provides an idea and theoretical basis for efficient, non-destructive, and online detection of fruits quality. [ABSTRACT FROM AUTHOR]