Search

– Due to the ongoing miniaturization of electronic products and the use of miniature printed circuit boards (PCBs), existing AI-based defect detection methods have exhibited poor performance in detecting tiny PCB defects. This issue can potentially compromise safety, degrade manufacturing quality, and increase production costs. To tackle this problem, we propose two novel techniques for PCB defect detection, namely Selective Feature Attention (SF attention) and Pixel Shuffle Pyramid (PSPyramid). SF attention identifies important features from a pyramid feature map to fuse the semantic and spatial information, while PSPyramid effectively fuses semantic features to detect various types of defects on PCBs, especially tiny defects. Moreover, a customized training strategy, specifically for PCB defect detection, is devised. To evaluate the performance of our proposed algorithms, extensive experiments have been conducted on two well-known PCB datasets containing tiny defects: the DeepPCB and TDD datasets. Our proposed non-referential method achieves performance comparable to existing referential methods on the DeepPCB dataset, making it more feasible for industrial applications. Compared to state-of-the-art methods, our method reduces the error by 16%, in terms of AP50, on the TDD dataset. The experimental results demonstrate the effectiveness of our proposed method in improving the quality assurance process for PCBs in the electronics industry.

The main challenge of continuous sign language translation (CSLT) lies in the extraction of both discriminative spatial features and temporal features. In this paper, a spatial-temporal feature extraction network (STFE-Net) is proposed for CSLT, which optimally fuses spatial and temporal features, extracted by the spatial feature extraction network (SFE-Net) and the temporal feature extraction network (TFE-Net), respectively. SFE-Net performs pose estimation for the presenters in sign-language videos. Based on COCO-WholeBody, 133 key points are abbreviated to 53 key points, according to the characteristics of the sign language. High-resolution pose estimation is performed on the hands, along with the whole-body pose estimation, to obtain finer-grained hand features. The spatial features extracted by SFE-Net and the sign language words are then fed to TFE-Net, which is based on Transformer with relative position encoding. In this paper, a dataset for Chinese continuous sign language was created and used for evaluation. STFE-Net achieves Bilingual Evaluation Understudy (BLEU-1, BLEU-2, BLEU-3, BLEU-4) scores of 77.59, 75.62, 74.25, 72.14, respectively. Furthermore, our proposed STFE-Net was also evaluated on two public datasets, RWTH-Phoenix-Weather 2014T and CLS. The BLEU-1, BLEU-2, BLEU-3 and BLEU-4 scores achieved by our method on the former dataset are 48.22, 33.59, 26.41 and 22.45, respectively, and the corresponding scores are 61.54, 58.76, 57.93 and 57.52, respectively, on the latter dataset. Experiment results show that our model can achieve promising performance. If any reader needs the code or dataset, please email lunfee@whut.edu.cn.

Video quality enhancement methods are of great significance in reducing the artifacts of decoded videos in the High Efficiency Video Coding (HEVC). However, existing methods mainly focus on improving the quality of natural sequences, not for screen content sequences that have drawn more attention than ever due to the demands of remote desktops and online meetings. Different from the natural sequences encoded by HEVC, the screen content sequences are encoded by Screen Content Coding (SCC), an extension tool of HEVC. Therefore, we propose a Mode Information guided CNN (MICNN) to further improve the quality of screen content sequences at the decoder side. To exploit the characteristics of the screen content sequences, we extract the mode information from the bitstream as the input of MICNN. Furthermore, due to the limited number of screen content sequences, we establish a large-scale dataset to train and validate our MICNN. Experimental results show that our proposed MICNN can achieve 3.41% BD-rate saving on average. In addition, our MICNN method consumes acceptable computational time compared with the other video quality enhancement methods.

Object detection can locate objects in an image using bounding boxes, which can facilitate classification and image understanding, resulting in a wide range of applications. Knowing how to mine useful features from images and detect objects of different scales have become the focus for object-detection research. In this paper, considering the importance of foreground features in the process of object detection, a foreground feature extraction module, based on deformable convolution, is proposed, and the attention mechanism is integrated to suppress the interference from the background. To learn effective features, considering that different layers in a convolutional neural network have different contributions, we propose methods to learn the weights for feature fusion. Experiments on the VOC datasets and COCO datasets show that the proposed algorithm can effectively improve the object detection accuracy, which is 12.1% higher than Faster R-CNN, 1.5% higher than RefineDet, and 2.3% higher than the Hierarchical Shot Detector (HSD).

Abstract Face hallucination is a super‐resolution technique specially designed to reconstruct high‐resolution faces from low‐resolution faces. Most state‐of‐the‐art algorithms leverage position‐patch prior knowledge of human faces to better super‐resolve face images. However, most of them assume the training face dataset is sufficiently large, well cropped or aligned. This paper, proposes a novel example‐based face hallucination method, based on cluster consistent dictionary learning with the assumption that human faces have similar facial structures. In this method, the paired face image patches are firstly labelled as face areas including eyes, nose, mouth and other parts, as well as non‐face areas without requiring the training face images cropped and aligned. Then, the training patches are clustered according their labels and textures. The cluster consistent dictionary is learned to represent the low‐resolution patches and the high‐resolution patches. Finally, the high‐resolution patches of the input low‐resolution face image can be efficiently generated by using the adjusted anchored neighbourhood regression. As utilizing the labelled facial parts prior knowledge, the proposed method represents more details in the reconstruction. Experimental results demonstrate that the authors' algorithm outperforms many state‐of‐the‐art techniques for face hallucination under different datasets.

Abstract Studying representation learning and generative modelling has been at the core of the 3D learning domain. By leveraging the generative adversarial networks and convolutional neural networks for point‐cloud representations, we propose a novel framework, which can directly generate 3D objects represented by point clouds. The novelties of the proposed method are threefold. First, the generative adversarial networks are applied to 3D object generation in the point‐cloud space, where the model learns object representation from point clouds independently. In this work, we propose a 3D spatial transformer network, and integrate it into a generation model, whose ability for extracting and reconstructing features for 3D objects can be improved. Second, a point‐wise approach is developed to reduce the computational complexity of the proposed network. Third, an evaluation system is proposed to measure the performance of our model by employing various categories and methods, and the error, considered as the difference between synthesized objects and raw objects are quantitatively compared, is less than 2.8%. Extensive experiments on benchmark dataset show that this method has a strong ability to generate 3D objects in the point‐cloud space, and the synthesized objects have slight differences with man‐made 3D objects.

Identifying tiny objects with extremely low resolution is generally considered a very challenging task even for human vision, due to limited information presented inside the object areas. There have been very limited attempts in recent years to deal with low-resolution recognition. The existing solutions rely on either generating super-resolution images or learning multi-scale features. However, their performance improvement becomes very limited, especially when the resolution becomes very low. In this paper, we propose a Representation Learning Generative Adversarial Network (RL-GAN) to generate super image representation that is optimized for recognition. Our solution deals with the classical vision task of object recognition in the distance. We evaluate our idea on the challenging task of low-resolution object recognition. Comparison of experimental results conducted on public and our newly created WIDER-SHIP datasets demonstrate the effectiveness of our RL-GAN, which improves the classification results significantly, with 10-15% gain on average, compared with benchmark solutions.

How to design an effective membership probability is an important component for Gaussian mixture model (GMM) of point set registration. In order to improve the robustness of point set registration, in this paper, a new representation is proposed for membership probability of Gaussian mixture model, by utilizing two types of feature descriptor, i.e. shape context or fast point feature histograms. Moreover, for each point of the model point set, a dynamic programming (DP) algorithm is developed to search for the optimal candidate points from the target point set. Compared to the state-of-the-art approaches, the proposed approach is more robust to deformation, outlier, occlusion, and rotation. Experimental results on several widely used 2D and 3D data demonstrate the effectiveness and feasibility of the proposed algorithm.

Almost all successful nodule detectors rely heavily on a fixed set of anchor boxes. In this paper, inspired by the success of the keypoint estimation method in natural image detection, we propose an anchor-free framework for accurate pulmonary nodule detection. We first present a novel representation for detecting nodules, in terms of their 3D center locations, which reduces the number of hyper-parameters and the corresponding computation related to anchors, thus making the nodule detection pipeline much simpler. Then, an effective two-stream network is introduced to reduce the false positive nodule candidates, by aggregating information from the image stream and motion-history stream. Experiments show that the proposed approach achieves a sensitivity of 96.1%, with 8 false positives per scan, and a CPM score of 90.6%, on the publicly available LUNA16 dataset, which outperforms other state-of-the-art methods. By testing on the SPIE-AAPM dataset with models pre-trained on the LUNA16, our proposed method yields 92.8% sensitivity with 8 false positives per scan. This demonstrates the effectiveness and generalization ability of our method.

Visual tracking is a challenging task due to unconstrained appearance variations and dynamic surrounding backgrounds, which basically arise from the complex motion of the target object. Therefore, the information and the correlation between the target motion and its resulting appearance should be considered comprehensively to achieve robust tracking performance. In this paper, we propose a deep neural network for visual tracking, namely the Motion-Appearance Dual (MADual) network, which employs a dual-branch architecture, by using deep two-dimensional (2D) and deep three-dimensional (3D) convolutions to integrate the local and global information of the target object's motion and appearance synchronously. For each frame of a tracking video, 2D convolutional kernels of the deep 2D branch slide over the frame to extract its global spatial-appearance features. Meanwhile, 3D convolutional kernels of the deep 3D branch are used to collaboratively extract the appearance and the associated motion features of the visual target from successive frames. By sliding the 3D convolutional kernels along a video sequence, the model is able to learn the temporal features from previous frames, and therefore, generate the local patch-based motion patterns of the target. Sliding the 2D kernels on a frame and the 3D kernels on a frame cube synchronously enables a better hierarchical motion-appearance integration, and boosts the performance for the visual tracking task. To further improve the tracking precision, an extra ridge-regression model is trained for the tracking process, based not only on the bounding box given in the first frame, but also on its synchro-frame-cube using our proposed Inverse Temporal Training method (ITT). Extensive experiments on popular benchmark datasets, OTB2013, OTB50, OTB2015, UAV123, TC128, VOT2015 and VOT2016, demonstrate that the proposed MADual tracker performs favorably against many state-of-the-art methods.

Due to some unfavorable factors, how to accurately register point sets is still a challenging task. In this paper, an effective point set registration approach is proposed based on a hybrid structure constrain. In the proposed method, a composite weight coefficient is determined based on the amplitudes of the vector and the corresponding projection of neighbor points. Given the composite weight coefficient, a local structure constraint is constructed as a linear combination of the vectors of neighbor points. A Gaussian mixture model is established by utilizing the local structure constraint and a global structure constraint based on the motion coherence theory. In addition, an expectation-maximization algorithm is derived to solve the unknown variables in the proposed model. For the constraint terms, an update strategy is utilized to obtain the approximately optimal weight coefficients. Compared to the state-of-the-art approaches, the proposed model is more robust due to the use of multiple effective constraints. Experimental results on some widely used data sets demonstrate the effectiveness of the proposed model.

Non-Rigid Structure from Motion (NRSfM) is the task of reconstructing the 3D point set of a non-rigid object from an ensemble of images with 2D correspondences, which has been a long-lasting challenging research topic. Compared to the state-of-the-art methods for NRSfM, the Procrustean Markov Process (PMP) model has obtained a relatively good performance. However, the estimation error and the convergence time of the PMP model will increase simultaneously when noise is present. To address this problem, in this paper, a coherent constraint is constructed to suppress the noise in the initialization step of the PMP algorithm. Moreover, an Accelerated Expectation Maximization (AEM) algorithm is devised to optimize the PMP estimation model. Experimental results on several widely used sequences demonstrate that our proposed algorithm achieves state-of-the-art performance, as well as its effectiveness and feasibility.

Abstract Background To adopt a healthy lifestyle is considered an essential component of nursing education. Self-rated health is a subjective assessment of health status and is consistent with objective health status. Previous studies have shown an association between self-rated health and engagement in a healthy lifestyle. Nursing students need to feel good about their subjective health status and to be able to adopt health improvements in their lifestyle before attempting to disseminate health messages to clients. The aims of this study were to compare the difference in self-rated health and health promotion lifestyle profile between senior and junior nursing students, describe correlations between self-rated health and health promotion lifestyle profile, and identify the predictors of self-rated health. Methods A cross-sectional descriptive survey was adopted. The study sample consisted of 314 junior and senior year nursing students from a tertiary institution. The self-reported questionnaire consisted of a single-item question to examine their self-rated health. The Health Promoting Lifestyle Profile-II: Chinese version short form (HPLP-IICR) was used to investigate the health-promoting lifestyles of the students. Descriptive statistics, Mann-Whitney U test, Chi-square test, Fisher’s exact test, Spearman’s correlation, and ordinal logistic regression were used to analyze the data. Results The median scores for self-rated health were 3 (Mean 3.26, IQR 3–4) and 3 (Mean 3.19, IQR 3–4) out of 5 for Year 2 and Year 5 students, respectively, with no significant difference between the two groups. The two groups of students showed no significant differences in overall score and in most subscales of the HPLP-IICR. An ordinal logistic regression showed that those students with higher health management score (OR: 1.12, 95% CI: 1.04–1.21) and who had experienced no family conflicts in the recent month than having family conflict (OR: 1.64, 95% CI: 1.01–2.66) were more likely to have higher self-rated health. Conclusion Nursing education and clinical practice can undoubtedly increase the health knowledge of students, but may not lead to changes in actual health-promoting behaviours. Students with a higher health management score and no family conflicts are more likely to give a positive appraisal of their health status.

Marine hydrological elements are of vital importance in marine surveys. The evolution of these elements can have a profound effect on the relationship between human activities and marine hydrology. Therefore, the detection and explanation of the evolution laws of marine hydrological elements are urgently needed. In this paper, a novel method, named Evolution Trend Recognition (ETR), is proposed to recognize the trend of ocean fronts, being the most important information in the ocean dynamic process. Therefore, in this paper, we focus on the task of ocean-front trend classification. A novel classification algorithm is first proposed for recognizing the ocean-front trend, in terms of the ocean-front scale and strength. Then, the GoogLeNet Inception network is trained to classify the ocean-front trend, i.e., enhancing or attenuating. The ocean-front trend is classified using the deep neural network, as well as a physics-informed classification algorithm. The two classification results are combined to make the final decision on the trend classification. Furthermore, two novel databases were created for this research, and their generation method is described, to foster research in this direction. These two databases are called the Ocean-Front Tracking Dataset (OFTraD) and the Ocean-Front Trend Dataset (OFTreD). Moreover, experiment results show that our proposed method on OFTreD achieves a higher classification accuracy, which is 97.5%, than state-of-the-art networks. This demonstrates that the proposed ETR algorithm is highly promising for trend classification.

In recent years, non-rigid structure from motion (NRSFM) has become one of the hottest issues in computer vision due to its wide applications. In practice, the number of available high-quality images may be limited in many cases. Under such a condition, the performances may not be satisfactory when existing NRSFM algorithms are applied directly to estimate the 3D coordinates of a small-size image sequence. In this paper, a sub-sequence-based integrated algorithm is proposed to deal with the NRSFM problem with small sequence sizes. In the proposed method, sub-sequences are first extracted from the original sequence. In order to obtain diversified estimations, multiple weaker estimators are constructed by applying the extracted sub-sequences to a recent NRSFM algorithm with a rotation-invariant kernel (RIK). Compared to other first-order statistics, the trimmed mean is a relatively robust statistic. Considering the fact that the estimations of some weaker estimators may have large errors, the trimmed means of the outputs for all the weaker estimators are computed to determine the final estimated 3D shapes. Compared to some existing methods, the proposed algorithm can achieve a higher estimation accuracy, and has better robustness. Experimental results on several widely used image sequences demonstrate the effectiveness and feasibility of the proposed algorithm.

The one-sample-per-person problem has become an active research topic for face recognition in recent years because of its challenges and significance for real-world applications. However, achieving relatively higher recognition accuracy is still a difficult problem due to, usually, too few training samples being available and variations of illumination and expression. To alleviate the negative effects caused by these unfavorable factors, in this paper we propose a more accurate spectral feature image-based 2DLDA (two-dimensional linear discriminant analysis) ensemble algorithm for face recognition, with one sample image per person. In our algorithm, multi-resolution spectral feature images are constructed to represent the face images; this can greatly enlarge the training set. The proposed method is inspired by our finding that, among these spectral feature images, features extracted from some orientations and scales using 2DLDA are not sensitive to variations of illumination and expression. In order to maintain the positive characteristics of these filters and to make correct category assignments, the strategy of classifier committee learning (CCL) is designed to combine the results obtained from different spectral feature images. Using the above strategies, the negative effects caused by those unfavorable factors can be alleviated efficiently in face recognition. Experimental results on the standard databases demonstrate the feasibility and efficiency of the proposed method.