Search

In industrial production, defects are treated differently based on their severity, so it is necessary to classify defects. However, in actual production, the classification accuracy is often insufficient due to the presence of few easily confused defects, which requires conservative treatment of all defects in production practice, resulting in significant human and economic costs. To solve this problem, this paper proposes a method for interactive separating easily confused defects. This method separates few easily confused defects from other defects, ensuring that the classification results of the remaining majority of defects can be directly used. This method selects easily confused defects from the training data as one or more new defect categories, called virtual defects, so that the trained network can distinguish between virtual defects and other defects. This paper designs a visual interface to assist users in interactively selecting easily confused defects to construct virtual categories. CMOS defect data from actual industrial sites are adopted for effectiveness verification, and the results show that the proposed method can quickly classify few confusing defects and ensure that the classification accuracy of remaining defects meets the requirements of industrial applications. [ABSTRACT FROM AUTHOR]

The large amount of medical information carried in the electronic medical record can help doctors better understand the situation of patients and assist doctors in clinical diagnosis. As the two core tasks of Chinese electronic medical record (EMR) information extraction, named entity recognition and entity relationship extraction have become the main research directions. Its main goal is to identify the medical entities in the EMR text and extract the medical relationships between the entities. This paper systematically expounds the research status of Chinese electronic medical record, points out the important role of named entity recognition and entity relationship extraction in Chinese electronic medical record information extraction, then introduces the latest research results of named entity recognition and relationship extraction algorithm for Chinese electronic medical record information extraction, and analyzes the advantages and disadvantages of each model in each stage. In addition, the current problems of Chinese EMR are discussed, and the future research trend is prospected. [ABSTRACT FROM AUTHOR]

The development of medical imaging technology has generated a massive amount of medical image data, which reflects the internal structural features of the human body. Medical image segmentation technology can improve the efficiency of medical diagnosis, making it an important assistive tool for modern medical diagnosis. However, noise or artifacts that are inevitably present in the imaging process bring great challenges to the segmentation work. In existing segmentation models, single-frame medical image semantic segmentation models do not consider the relationship between image frames, while video semantic segmentation models utilize temporal information but have some limitations in edge extraction. To address these issues, this paper proposes a U-Net-based temporal semantic segmentation model modified by optical flow. This model can extract optical flow information between consecutive frames and perform feature extraction and weight allocation on the current frame and optical flow for correction. Experiments show that the model obtains optimal results on three evaluation metrics, namely Dice similarity, pixel accuracy and cross-merge ratio, on different types of datasets, namely Drosophila electron micrographs, combined healthy abdominal organ segmentation and coronary angiogram, which validate the effectiveness and generalization of the proposed model. [ABSTRACT FROM AUTHOR]

With the development of artificial intelligence, digital images have been widely used in various fields. However, due to the appearance of image editing software, a large number of images have been tampered with maliciously, which seriously affects the authenticity of image content. Different from the general object detection, the study of image tampering detection needs to pay more attention to the tamper information of the image itself, which is often manifested in a weak form. Therefore, the detection model needs to focus on learning more abundant tamper features. This paper proposes a dual-stream Faster R-CNN model that combines gradient edge information and attention mechanism, and the model can realize detection and location of regions with different tampering types. One of the two streams is the color stream, which uses the attention mechanism to extract the surface features of the image, such as brightness contrast, visual difference of tampering with the boundary, etc. The second of the two streams is a gradient stream. A Gradient high-pass filter is used to enhance the anomaly edge features between the real area and the tampered area, making it easier for the model to find faint tampered traces in the tampered image. Finally, the features of color stream and gradient stream are fused by means of compact bilinear pooling. Due to the relatively small size of publicly available image tampering data sets, the Pascal VOC 2012 is used to create an image tampering detection data set which containing 10 010 images for model pre-training. The experimental results on COVER, Columbia, and CASIA data sets show that the model proposed in this paper improves the detection accuracy by 7.1% to 9.6% compared to the latest models, and exhibits higher robustness under JPEG compression and image blur attacks. [ABSTRACT FROM AUTHOR]

Precipitation nowcasting refers to the high-resolution prediction of precipitation in the short term, which is an important but difficult task. In the context of deep learning, it is viewed as a radar echo map-based spatiotemporal sequence prediction problem. Precipitation prediction is a complex self-supervised task. Since the motion always changes significantly in both spatial and temporal dimensions, it is difficult for ordinary models to cope with complex nonlinear spatiotemporal transformations, resulting in blurred predictions. Therefore, how to further improve the model prediction performance and reduce ambiguity is a key focus of research in this field. Currently, the research on precipitation nowcasting is still in the early stage, and there is a lack of systematic classification and discussion about the existing research work. Therefore, it is necessary to conduct a comprehensive investigation in this field. This paper comprehensively summarizes and analyzes the relevant knowledge in the field of precipitation nowcasting from different dimensions, and gives future research directions. The specific contents are as follows: (1) The significance of precipitation nowcasting, and the advantages and disadvantages of traditional forecasting models are clarified. (2) The mathematical definition of the nowcasting prob-lem is given. (3) Common predictive models are comprehensively summarized, analyzed. (4) Several open source radar datasets in different countries and regions are introduced, and download links are given. (5) The metrics used for prediction quality assessment are briefly introduced. (6) The different loss functions used in different models is discussed. (7) The research direction of precipitation nowcasting in the future is pointed out. [ABSTRACT FROM AUTHOR]

Air pollution seriously affects human health and social sustainable development. However, the multi-variable air quality data obtained by sensors often have missing values, which brings difficulties to data analysis and processing. Currently, many analysis methods for changes in a certain air component only rely on time data and spatial data of this attribute, ignoring the influence of other air components on the trend of this attribute in the same time interval. In addition, it is difficult to achieve ideal results in filling discrete missing data. This paper proposes a Time Attention Model (TAM) based on deep learning, which uses attention mechanism to focus on the correlation between different timestamps and the correlation between different feature time series, and combines short-term historical data to fill missing values in multi-variable air quality data. The proposed model is evaluated using air quality data from Beijing, and the experimental results show that TAM has advantages over ten other baseline models. [ABSTRACT FROM AUTHOR]

Because of its advantages of no ionizing radiation, Cardiac Cine Magnetic Resonance Imaging (CMRI) has become the main method in medical diagnosis, and the accurate classification and recognition of left ventricular, right ventricular and left myocardium is an important step before the heart surgery. However, manual segmentation of cardiac structure time-consuming and error-prone, so automatic segmentation of both ventricular and myocardium is crucial. Firstly, U-Net++ is selected as the basic network framework. Secondly, in order to improve the feature reuse rate and solve the overfitting issue caused by the increase of network depth, a dense residual module is proposed in the encoding part of U-Net++, so that more features can be learned during the network down-sampling process. In addition, in the decoding part, in order to make the network segmentation results more in line with the target organ between the physical characteristics, multiple convolution kernels are used to expand the receptive field and a long distance dependent module is used share the global context information to make the network in the decoding process as much as possible to get the relationship information between the target organs and make the segmentation result more accurate. Finally, considering the consistency and uniqueness between the biventricle and the left myocardium, the post-treatment operation of obtaining the maximum connectivity domain and filling the small holes is added. The experimental data used in this paper is the ACDC Cardiac Segmentation Challenge data set, which includes SHORT axial MRI images of 150 volunteers at the end of systolic and diastolic phases. In this paper, the test set of this data set is verified, and the experimental results are obtained by online submission. Experimental results show that, compared with other methods, the proposed method can effectively segment the target organ. Specially, the Dice coefficient at the end of diastolic period reaches 0.96, 0.94 and 0.89 respectively in the left ventricle, right ventricle and left myocardium, and the segmentation precision at the end of systolic period reaches 0.87, 0.86 and 0.89 respectively [ABSTRACT FROM AUTHOR]

The curvilinear motion of solar photosphere bright points, which is approximate rotation, is of great significance for studying how the energy from solar convection zone is transmitted to the corona. The existing algorithms only detect the global curvilinear motion of photosphere bright points. This paper proposes a multi-scale and multi-modal deep learning method to detect the global and local curvilinear motion of photosphere bright points. This mothod constructs a multi-scale network model based on the bidirectional long short-term memory network (Bi-LSTM) to extract multi-scale time sequence features of the trajectories of photosphere bright points. EfficientNet-B0 is adopted to extract the spatial features of the trajectories. The temporal features and spatial features are fused into multi-modal features to detect various curvilinear motions of photosphere bright points. The experiment results show that the accuracy of this method is 85.08%, which is 6.12% higher than that of the single-scale method and 3.1% higher than that of the multi-scale and single-mode method. This method can also be applied to the motion type detection requirements in other fields. [ABSTRACT FROM AUTHOR]

Recent years, with the widespread use of video surveillance technology, video anomaly detection, which can intelligently analyze massive videos and quickly discover the abnormalities, has received wide attention. This paper aims to give a comprehensive survey on deep learning based video anomaly detection methods. Firstly, a brief introduction of video anomaly detection is given, including the basic concepts, basic tasks, modeling process, learning paradigms as well as the evaluation perspectives. Secondly, the video anomaly detection methods are classified into four categories: reconstruction-based, prediction-based, classification-based, and regression-based. Their basic modeling ideas, typical algorithms, advantages, and disadvantages are discussed in detail. On this basis, the commonly used single-scene video anomaly detection public datasets and evaluation indicators are introduced, and the performance of representative anomaly detection algorithms is compared and analyzed. Finally, summary is conducted, and the future development directions related to datasets, algorithm and evaluation criteria of video anomaly detection are proposed. [ABSTRACT FROM AUTHOR]

Generating images from text is a challenge task in machine learning community. Although significant success has been achieved so far, problems such as unstable network training and disappearing gradients still exist. In response to the above shortcomings, based on the stacked generative confrontation network model (StackGAN), this paper proposes a text-to-image generation method that combines spectral normalization and perceptual loss function. Firstly, the network model applies spectral normalization to the discriminator, restricts the gradient of each layer of the network to a fixed range, slows down the convergence speed of the discriminator, and hence improves the stability of network training. Secondly, the perceptual loss function is added to the generator network to enhance the consistency between the text content and the generated image. The network model uses Inception scores to evaluate the quality of the generated images. The experimental results show that, compared with the original StackGAN, the network model has better stability and generates clearer images. [ABSTRACT FROM AUTHOR]

CD56 is a nerve cell adhesion molecule that can be used in the diagnosis and study on a variety of tumor cells. CD56 is one of the latest tumor molecular markers, and the research on CD56 images in the field of digital medical image processing has just started. With the development of deep learning technologies such as semantic segmentation, more and more researchers are applying semantic segmentation technology to medical image processing in order to assist doctors in diagnosis. In CD56 images, the proportion of the number of pixel points of background, negative cells and positive cells is very unbalanced, which is roughly 70∶10∶1 and affects the segmentation effect of semantic segmentation technology on CD56 images. In this paper, the loss function of the semantic segmentation model is improved by adding loss weight to different class and adding adaptive weight to each pixel, so that the model can pay more attention to cells, especially positive cells. At the same time, the clustering method is used to refine the annotations of CD56 images before model training, which further improves the segmentation accuracy of the model. The experimental results on CD56 image dataset show that, the refinement of the image annotations and the improvement of the loss functions in the relevant semantic segmentation models can improve the segmentation accuracy of the CD56 images. [ABSTRACT FROM AUTHOR]

Alzheimers disease (AD) is an irreversible neuro degenerative brain disease and the most common dementia in the elderly. Manual classification of Alzheimer's magnetic resonance image (MRI) has problems delay classification and time-consuming classification. As the aging population becomes more and more serious accurately and quickly classify patients with AD has important research significance. This paper combines convolutional neural network (CNN) technology with MRI technology and designs a 3D-ResNet model for AD classification which achieves 98. 39% accuracy 96.74% sensitivity and 99. 99% specificity on the validation set and achieves 97. 43% accuracy 94. 92% sensitivity and 99. 99% specificity on the test set. The classification time of each patient is 0. 23 s. In addition，for the problem that the pathogenesis of AD is not yet clear，this paper uses Class Activation Mapping (CAM) technology to visualize the AD-related brain regions. [ABSTRACT FROM AUTHOR]

In recent years, many sensor models based on LoRa devices have verified the long-distance sensing potential of LoRa devices, but the use of feature-blurred LoRa wireless signals to identify human activities still requires further research. This paper analyzes the propagation law of LoRa signals affected by human activities, and proposes a LoRa signal processing method to extract signal change features. Subsequently, data are collected to create two LoRa datasets that record human activities, and the proposed method is tested through advanced deep learning models. The accuracy of recognizing activity types, activity roles in a room, activity roles, and activity rooms in four rooms reaches over 90%. Compared to the method of using convolutional recurrent neural networks for direct training, it is also more time-saving and spatial resource-saving. [ABSTRACT FROM AUTHOR]

The deployment environment of distributed photovoltaic power plants is relatively complicated, and many kinds of faults inevitably occur during the actual operation. In order to solve the above problem, this paper proposes a fault diagnosis model of distributed photovoltaic power stations based on deep residual network. It analyzes and processes the sequence data of equipment operation, and achieves rapid and accurate judgment of fault categories. This model applies a one-dimensional convolution kernel to perceive the characteristics of time series data. Then, it uses a multi-level convolution structure to increase the diagnostic ability. Finally, the residual network is utilized to solve the problem of gradient disappearance caused by the increase of model depth, and accelerate the training of the deep model. The experimental results based on the power station test data show that the residual network model achieves higher fault diagnosis accuracy than several st ate-of-the-art intelligen t models. The application of this model can not only greatly reduce the investment in fault inspection of photovoltaic power plants, but also improve the efficiency of fault diagnosis of photovoltaic power plants. [ABSTRACT FROM AUTHOR]

Behavior det ection is a research hot spot in the field of video understanding and computer vision, which attracts the attention of scholars at home and abroad. It has been widely used in many fields such as intelligent surveillance and human-computer interaction. With the development of technology, deep learning has made a great breakthrough in image classification. The application of the recognition methods based on deep learning to human behavior detection has become a hotspot. Therefore, the paper firstly introduces several datasets commonly used in behavior detection, and the research status of deep learning in the field of behavior detection in recent years. Then, the basic process of behavior detection methods and recognition methods based on deep learning are analyzed. Finally, the future development trend and possible shortcomings are analyzed from the aspects of method performance and application prospect. [ABSTRACT FROM AUTHOR]

Sketch retrieval (SBIR) is an extension of content-based image retrieval (CBIR), which is a flexible and convenient way to retrieve target images. How to minimize the difference between the sketch domain and the image domain is crucial to SBIR. The traditional methods extract the manual fea-tu res to achieve the approxima te conversion bet ween the sketch field and the image field, so as to reduce the domain difference. However, these methods cannot effectively fit the content of the two domains, resulting in low retrieval accuracy. Deep learning methods break through the limitations of traditional methods, which extract high-dimensional features from a large amount of data and have been proved to effectively solve the cross-domain modeling problems. This paper focuses on deep learning-based sketch retrieval methods, and covers several aspects such as the deep feature extraction model, public dataset, coarse-grained and fine-grained retrieval based on deep learning, deep hashing technology, category generalization, etc. Related works are reviewed and commented on. Then, a comparative experiment is conducted. For one hand, three existing public SBIR datasets such as Sketchy, TU-Berlin and QuickDraw are used for suitability evaluation. For the other hand, three latest SBIR deep learning models such as GRLZS model, SEM-PCYC model and SAKE model are selected for performance analysis and comparison. Finally, current challenges and future research trends of SBIR are summarized. [ABSTRACT FROM AUTHOR]

Ocean mesoscale eddy is an important ocean mesoscale phenomenon that plays an important role in ocean circulation, mat erial and energy transport, and has an important impact on the safety of ship navigation and hydroacoustic communication. Efficient and accurate detection and identification of ocean mesoscale eddies are of great research value for both physical ocean cognition and ocean exploitation. Traditional eddy detection and identification methods rely on a single threshold designed by experts experiences. With the rise of deep learning, the current machine learning methods show certain advantages in the accuracy and automation of eddy detection and identification. This paper summarizes and comparatively analyzes the existing machine learning-based detection and identification methods to provide a systematic knowledge and reference basis for the development of ocean mesoscale eddy detection and identification research. [ABSTRACT FROM AUTHOR]

Semantic segmentation algorithms can classify images at the pixel level, and are widely used in fields such as unmanned driving, medical image processing, and industrial automation, and have important research value. The research of semantic segmentation algorithms focuses on three aspects: improving the accuracy of segmentation, reducing the amount of parameter s and increasing the speed of inference. The lightweight semantic segmentation algorithm ENet uses a multi-layer convolutional codec and a large number of dilated convolutions to avoid excessive downsampling and use of spatial information. Although it retains some spatial information integrity and large receptive field, the codec is bloat-ed, the transmission of spatial information is poor, and the sensory field overflows and causes gr id effect. Aiming at the above problems, this paper tailors the EN et algorithm structure, uses the attention mechanism and the pyramid dilated convolution to design spatial information transmission module, optimizes the algorithm structure, improve s the algorithm receptive field, and completely transmits the spatial information transmission. The experimental results on public datasets Cityscapes and BDD100K show that the new module can improve the performance of the original algorithm with a smaller amount o f parameters and calculations, which proves the redundancy of t he original algorithm and the effectiveness of the designed module. [ABSTRACT FROM AUTHOR]

Single image super-resolution (SISR) refers to the recovery of a high-resolution image from a single low-resolution image. With deep learning used in the field of image super-resolution, deep networks can independently learn the mapping relationship between low-resolution and high-resolution training images, showing better reconstruction performance than the traditional methods. Therefore, deep learning has become dominant in super-resolution. This paper focuses on the exploration of the existing deep network model of super-resolution in terms of reconstruction mode, network structure, and loss function. By comparing the similarities and differences between different models, the advantages and disadvantages of different model building methods and the applicable application scenarios are analyzed. Meanwhile, the reconstruction results of different network models on the benchmark test datasets are compared and the potential directions are concluded. [ABSTRACT FROM AUTHOR]

Crowd counting and crowd density estimation are the important branch in crowd analysis， and are also the important information that surveillance always concerns. Although some important progress has been made in this field in recent decades, there are still some challenging problems. This paper reviews the current researches and development trends of the crowd counting and crowd density estimation methods based on computer vision. Firstly，the development background and application direction of the crowd counting and crowd density estimation technology are introduced. Secondly, the important methods proposed in recent years are summarized, which can be divided into two types from the machine learning point of view: the shallow learning based methods and the deep learning based methods. In the other hand, they can be divided into two types from learning model standpoint： the direct method (i. e.， the detection based method) and the indirect method (i. e.，the pixel-based, texture-based and corner point based methods). This paper introduces the shallow learning based methods in the last twenty years in detail, and makes a brief summary of the deep learning based methods in recent years. Then, a brief introduction is made on the performance evaluation techniques of crowd counting and crowd density estimation methods, and several data sets are provided for testing and evaluating these methods. Finally, the technical challenges in the field are summarized and the future research directions are prospected. [ABSTRACT FROM AUTHOR]

At present, the fatigue early warning algorithm mostly adopts real-time monitoring and alarming, which has great security risks in the high-speed driving environment. In view of the temporal correlation of human fatigue state, this paper proposes an early warning algorithm based on spatiotemporal feature extraction of facial motion. Firstly, a convolutional neural network with spatial transformation structure is constructed to identify the face region and detect and mark the facial feature points. Secondly, a spatiotemporal feature extraction network is established, and the real-time acquired facial image feature sequence is used to predict and output the future image sequence. Finally, in the outputted image sequence, the comprehensive states of eyes and mouth are used to deter mine whether a fatigue warning is issued or not. Experimental results show that, under the condition that the image is acquired at 15 frames per second and the 30 frames in the future 2 seconds are predicted, the proposed algorithm can achieve the accuracy of more than 90 % when issuing a fatigue warning 26 frames (about 1. 5 seconds) in advance, and the accuracy of 97 % when issuing a fatigue warning 15 frames (1 second) in advance. Under the average speed of 100 km/h in China's expressways, it is equivalent to an early warning of 40 meters in advance, which can further reduce the occurrence of traffic accidents. [ABSTRACT FROM AUTHOR]

The traditional Unmanned Aerial Vehicle (UAV) human-machine interaction requires specialized equipment and professional training, and convenient and innovative ways of interaction are often more popular. In this paper, with ordinary cameras, we study the UAV gesture control system based on computer vision and deep learning. The system first uses the fast tracking algorithm to extract the operator's region in the video sequence, greatly reducing the pressure of subsequent video processing while removing the influence of complex background and camera drift. Secondly, according to the time information of the actions, the optical flow features are encoded in different colors and superimposed on a picture, and the video is converted into a color texture map that contains both temporal features and spatial features. Finally, colored texture images are well learned and classified by a deep Convolutional Neural Network (CNN) and UAV controlling commands are generated according to the classified results. The proposed system estimates actions within 1.6s every 0.4s and uses CNN to classify pictures so as to achieve real-time human-computer interaction. The system has a recognition accuracy of over 93% within 60 meters. In indoor and outdoor environments, the operator can conveniently control the UAV by imitating command actions. [ABSTRACT FROM AUTHOR]