Search

Objectives: The quality of network structure is the key to obtain excellent classification performance. Most of existing deep models are designed by the trial-and-error method. When the same model processes different hyperspectral images (HSIs), different network hyperparameters need to be set artificially to adapt to the data characteristics of different images, greatly increasing the difficulty of network design. Aiming at the problem of deep learning model design in hyperspectral image classification, this paper proposes an automatic network structure search method for HSI classification. Methods: In the proposed method, the differentiable structure search technology is first used to search the network structure on the source HSI data sets, then the deep network model is constructed by stacking cells, and finally the classification performance is evaluated on the target HSIs. It should be pointed out that, the proposed method only conducts one search on the source HSI data sets, and the obtained deep model can be applied to other target HSI classification tasks, which can effectively improve the utilization rate of the model. To improve the generalization ability and classification accuracy of the model, multi-source and multi-resolution HSIs are employed to construct the precollected data set, and the partial channel connection operation is introduced to further improve the search efficiency. Results: Three HSIs inculding Houston, Chikusei and Pavia Center are selected as source data. Four HSIs including University of Pavia, Indian Pines, Salinas and Houston 2018 are selected as the target HSIs. The source data is used to determine the network model and the target HSIs are used to evaluate the classification performance. Experimental results show that the proposed method can automatically search out a universal deep network model suitable for HSI classification task, and this model can achieve better classification performance than the conventional deep learning models. The overall classification accuracies of University of Pavia, Indian Pines, Salinas and Houston 2018 are 98.15%, 98.74%, 97.30% and 74.47%, respectively. Conclusions: Compared with other deep models, the model in this paper is automatically determined by differentiable architecture search algorithm according to the data characteristics of HSIs, effectively improving the classification accuracy of target HSIs while avoiding complex network design. Extensive experiments verify the applicability and effectiveness of the proposed method in HSI classification. [ABSTRACT FROM AUTHOR]

Since the objects in remote sensing images have the characteristics of arbitrary direction, dense distribution and large scale differences, object detection in remote sensing images has become a challenging problem. Aiming at this problem, this paper systematically reviewed the work related to rotating object detection in deep learning remote sensing images in recent three years. Firstly, this paper introduced the representation method and characteristics of rotated bounding box. Then, it analyzed the existing methods of rotating object detection in remote sensing images from four aspects: the feature extraction network, the generation of rotating anchor and candidate bounding box, the label allocation and sampling strategy, and the loss function. Next, it introduced the commonly used remote sensing image datasets for rotated object detection, and compared the performance of different algorithms. Finally, this paper prospected rotating object detection in remote sensing images. [ABSTRACT FROM AUTHOR]

Although high-quality and high-resolution depth maps can significantly improve the performance of computer vision tasks in various natural scenes, the limitations of depth camera hardware make the depth maps captured by consumer-level depth cameras have problems such as low resolution, poor quality, invalid holes, etc. Depth super-resolution reconstruction (DSR) is a kind of technology that can effectively improve the resolution and quality of depth maps, and DSR has become a research hot-spot in the field of computer vision. Firstly, this paper introduced the definition of DSR and the research progress of DSR algorithm at home and abroad in recent years, and then mainly stated and analyzed deep learning DSR reconstruction algorithms. Next, it introduced the depth image quality evaluation criteria. Finally, this paper prospected the application fields of DSR and the challenges as well as the opportunities in the future. [ABSTRACT FROM AUTHOR]

Aiming at the problem that the traditional deep learning fish recognition method has a low accuracy rate and the parameters cannot be determined adaptively during the model training process. This paper proposes an improved ShuffleNet fish identification method based on Particle Swarm Optimization( PSO) and Transfer Learning(TL). The research takes 20 species of fish as the object, uses the particle swarm algorithm to take the model's loss function as the fitness function, optimizes the two hyperparameters of batch size and learning rate, and uses the transfer learning method for training to construct TL-PSO-ShuffleNet model. The research shows that compared with the models of AlexNet, MobileNet, and ShuffleNet, the recognition accuracy rate is increased by 57. 89%, 30. 43% and 23. 28%, respectively. The fish identification method proposed in this paper has the characteristics of high accuracy and self-adaptive parameter setting, which provides a reference for the research on automatic fish identification. [ABSTRACT FROM AUTHOR]

The function of protein is crucial for understanding the mechanisms of cellular and biological activities, as well as for studying the mechanisms of diseases. Traditional experimental and sequence alignment methods are insufficient to support large-scale protein functional annotation when in the face of the rapid growth of sequence databases. For this situation, this paper proposed EGnet（Evolutionary Graph Network） model, which utilized the protein pre-training language model ESM2 and One-hot encoding to obtain the protein sequence encoding. The model integrated the coevolutionary information between residues, including PI（Paired Interaction） and SPI（Strong Paired Interaction）, through sequence self-attention and physical calculations. Subsequently, the two types of coevolutionary information and the sequence encoding used in inputs for a multi-layered cascaded graph convolutional network to learn the node features of the sequence encoding and achieve end-to-end protein function prediction. Compared with earlier methods, EGnet achieved better performance on the EC（Enzyme Commission） category labels in the ENZYME database, which reached 0.89 in the F-score and 0.91 in the AUPR. The results indicate that EGnet can achieve good performance by using only a single sequence to predict protein function, providing a rapid and effective method for protein function annotation. [ABSTRACT FROM AUTHOR]

Existing lip-reading algorithms for Chinese sentences are inadequate at feature extraction and visual ambiguity resolution, which leads to low accuracy. Aiming at this problem, this paper proposed a lip-reading algorithm for Chinese sentences based on temporal convolutional network and 3D convolutional neural network（3DT-CHLipNet）.Firstly, it used a Temporal Convolutional Network to extract long-term features from lip dynamics sequences, which has a much larger receptive field than the Long Short Term Memory Network. Secondly, in order to minimize the visual ambiguity in Chinese lipreading, it adopted a Transformer model with the self-attention mechanism to capture the context information and improve the accuracy of sentence prediction. Finally, it introduced a temporal masking data augmentation strategy in the data preprocessing to further reduce the error rate of the algorithm. Comparison experiments on CMLR, the largest open-source Sequence-to-Sequence Chinese Mandarin Lip Reading dataset, show that the improvement in accuracy over representative lip reading algorithms for Chinese sentences ranges from 2.17%to 23.99%. [ABSTRACT FROM AUTHOR]

In order to solve the disadvantage that the ViT cannot change the input patch size and the input patches are all single-scale information, this paper proposed an image classification network based on Transformer called MultiFormer. MultiFormer embeds small patches with different scales of input at each stage into large patches with rich semantic information through the AWS（Attention With Scale） module; and captures local and global attention alternately through the GLA-P（Global-Local Attention With Patch） module, preserving both fine-grained and coarse-grained features during embedding. This paper designed MultiFormer-Tiny,-Small and-Base networks of three different sizes to achieve 81.1%, 82.2% and 83.2% Top-1 accuracy respectively in ImageNet image classification experiments, the latter two models improve by 3.1% and 3.4% compared to the same volume of convolutional neural networks ResNet-50 and ResNet-101; MultiFormer-Base offers a 2.1% improvement with far fewer parameters and computational effort than the ViT-Base/16 model, and without the need for extensive data pre-training. [ABSTRACT FROM AUTHOR]

Action recognition (AR) is a hot research area in computer vision field, and has an extensive application prospect for security monitoring, autopilot, production safety etc. Firstly, this paper analysed the connotation and denotation of AR and put forward the technical challenges; Secondly, the paper analysed and compared the working principles of AR from three aspects: time feature extraction, efficient optimization and long-term feature capture; Thirdly, in order to select suitable AR models for different application scenarios, this paper compared the performance characterization of 43 benchmark AR methods in recent ten years based on UCF101, HMDB51, Something-Something and Kinetics400 data sets. Finally, this paper pointed out the future development direction of AR field, and the research results can provide theoretical reference and technical support for video feature extraction and visual content understanding. [ABSTRACT FROM AUTHOR]

Expert recommendation is one of the research hotspots in the online Q&A community. However, the majority of the currently used algorithms concentrate on matching user's static interest and question information, ignoring the efficient capture of user' s dynamic interest representation information, which results in a lack of recommendation accuracy. In order to address the aforementioned issues, this paper proposed an expert recommendation algorithm for the Q&A community that incorporated multi-head self-attention. Firstly, this algorithm built out question encoders of convolutional neural networks and attention mechanisms to process target question and user' s historically answered questions, in order to extract the corresponding question representations. Secondly, it treated the sequence of user's historically answered questions as a time series and used the multi-head self-attention mechanism to learn the dynamic interest representations hidden in the sequence, and combined with the user' s static interest representations to obtain the user' s comprehensive interest representations. Finally, it calculated the similarity between the target question representation and the comprehensive user representation to generate recommendation results. Based on the real data from the Zhihu Q&A community, the experimental results show that the performance of this algorithm is significantly better than the current popular expert recommendation algorithm. [ABSTRACT FROM AUTHOR]

The image rain removal algorithm analyzes and processes rain images to remove rain streaks and restore clean background scenes, which helps to improve the recognition accuracy of computer vision task, so that it becomes a hot spot in current research. In order to systematically understand the current research status and development trends in this field. Firstly, this paper introduced some typical synthetic rain models. Secondly, it analyzed typical image rain removal algorithm models and methods from two aspects, such as model-driven and data-driven. Then it compared the algorithm evaluation metrics and public rain datasets respectively. Finally, this paper prospected the future development trend of single image rain removal algorithms. [ABSTRACT FROM AUTHOR]

Traditional manually designed feature descriptors are prone to cause a large number of mismatches in the face of multi temporal remote sensing images with large geometric and radiation differences and obvious changes in ground features. This paper developed a convolutional neural network matching algorithm with double screening and proposed double constraints to extract the deep semantic feature map of the image through CNN, it selected the key points that satisfied both the priority maximum principle and the precise extreme value principle from the feature map, and extracted 512-dimension descriptors from the corresponding positions. In the feature matching stage, the strategy of combining the rough to fine reverse matching constraints and RANSAC constraints ensured that the false matching points were eliminated effectively, and retained a certain number of correct matching points, which improved the matching accuracy. The experimental results show that the number of matches is moderate, the correct matching rate is improved compared with other algorithms selected in the experiment, and the image registration error is reduced by about 2 pixels. The running time is obviously improved, matching effect is good, with certain adaptability and robustness. [ABSTRACT FROM AUTHOR]

Rain streaks and fog brought by rain can degrade the quality of images taken outdoors. In order to remove the in-fluence of rain and fog on images, this paper proposed an image derain method based on rain and fog separation processing and multi-scale convolutional neural network. Firstly, it extracted the rain lines and image details to the high frequency layer using guided filtering, while separated the fog and background information to the low frequency layer. Then it constructed a multi-scale convolutional neural network to remove the rain lines in the high frequency layer, and incorporated multiple dense connection modules into the network to improve the accuracy of feature extraction. Secondly, it constructed a lightweight defogging network with multi-layer feature fusion to remove the fog in the low frequency layer, and used the parameter integration structure to avoid estimation of multiple atmospheric scattering model parameters resulting in sub-optimal solutions. Finally, it combined the processed high and low frequency results to restore a clear image. Tested on several synthetic rain and fog datasets as well as on real natural scene images, the qualitative and quantitative results show that the proposed method removes rain and fog while retaining color information well, and improves the image structure similarity by 0. 02 to 0. 08 and the image peak signal-to-noise ratio by 0. 2 to 3. 5 dB compared to recent algorithms. [ABSTRACT FROM AUTHOR]

Abstract: Accurate identification of cotton top bud is important for cotton topping operation to detect cotton terminal bud accurately in field, a recognition method using Faster R-CNN (Faster Region Convolutional Neural Networks, Faster R-CNN) by integrating dynamic mechanism was proposed to solve the recognition difficulties caused by the small size of cotton terminal bud when it is topped in densely planted fields. The RegNetX-6.4GF model was used as the backbone network to improve its image feature extractor capabilities. Due to number of proposals under a higher IOU(Intersection Over Union, IOU) as well as the matching degree between anchor and the target shape affect the performance of the detector, the method proposed in this paper changed the original anchor generation mechanism by combining FPN (Feature Pyramid Network, FPN) and Guided Anchoring in RPN (Region Proposal Network, RPN), which will cause the distribution of the proposals generated by the RPN of the algorithm under different IOUs dynamically change during the training process. To adapt the dynamic change of proposals distribution, we integrated Dynamic Region Convolutional Neural Networks ( Dynamic R-CNN) in Faster R-CNN, which can dynamically adjust the value of IOU to obtain high quality proposals. And the GROIE (Generic ROI Extractor, GROIE) mechanism was inducted to extract ROI (Region of Interest, ROI) to improve the feature fusion capability. In this paper, 4 819 images of gossypium hirsutum population which contain seven leaf types were taken from the top of cotton plant at distance of 30-50 cm (medium distance) and 50-100 cm (long distance) under uniform light, oblique strong light, direct strong light, and shadows. Those images were processed as MS COCO 2017 format dataset and assigned to the training set, validation set, and test set, which contained 2 815, 704, and 1 300 pictures respectively. The experimental results demonstrated that FPS (Frames Per Second, FPS) of proposed model was up to 10.3 frames/s and the Mean Average Precision (MAP) of bud identification reached to 98.1% which was 3.2 percentage points higher than original Faster R-CNN model. The validation set were used to compare performance of mainstream recognition algorithm and proposed method. Results showed that the improved Faster R-CNN's MAP was 7.3% higher than original Faster R-CNN, which was also higher than RetinaNet, Cascade R-CNN (Cascade Region Convolutional Neural Networks, Cascade R-CNN) and RepPoints by 78.9%, 10.1% and 8.3% when IOU was set to 0.5. The improved Faster R-CNN proposed in this paper meets the accuracy and real-time requirements of cotton topping operation. [ABSTRACT FROM AUTHOR]

This paper presented an image dehazing method based on deep learning to solve the problem that the remote sensing images had reduced image sharpness due to haze. Firstly, this algorithm distorted the original atmospheric scattering model to get an end-to-end defrosting model. Then, it unified several unknown parameters to one parameter and estimated the unknown parameters by using multiscale convolution neural network. Finally, it brought the parameter estimates into the defrosting model to get a haze-free image. For no reference image dataset, this paper used the existing dataset to preliminary train the network, then added the self-built dataset to secondary train the network. The experimental results show that, compared with the related defrosting algorithms, the proposed algorithm improves the visual effect and objective index to some extent, and effectively improves the clarity of remote sensing images in haze weather conditions. [ABSTRACT FROM AUTHOR]

This paper constructed a new TerraSAR-X dataset named GDUT-Nansha and then proposed a new lightweight semantic segmentation algorithm using synthetic aperture radar images（ SAR）. It is difficult to apply traditional deep learning models to SAR datasets with small volume due to the tremendous amount of parameters. A revision scheme is, therefore, put forward to deal with such problem. This paper accomplished an improved lightweight convolutional neural network model named revised weighted loss ENet（ RWL-ENet） for SAR images. The current study introduced a weighted loss function to solve the problem of imbalance of training datasets. Compared with other classical convolution neural network models, the efficiency and robustness of the new dataset were validated. Meanwhile, RWL-ENet model attained 0. 884,0. 804, and 0. 645 in terms of three quantitative metrics, including pixel accuracy（ PA）, mean pixel accuracy（ m PA）, and mean intersection over union（ m IoU）. In addition, the parameters of this new proposed model are much less than other classic network models. [ABSTRACT FROM AUTHOR]

Aiming at the traditional model parallel methods for accelerating convolution neural network (CNN) tend to have low utilization, this paper proposed PipeCNN, which accelerated CNN with software pipeline. Firstly, this paper studied the forward propagation and back propagation, and then explored data correlation during training. Secondly, it parallelized CNN with the support of software pipeline, and then analyzed two feasible gradient updating methods in PipeCNN. Finally, it used circular queue to realize communication between two layers and then proposed a task allocation algorithm to divide CNN into working parts. Experiments show that the method can obtain good speedup and utilization while ensuring the accuracy of the model. It shows that software pipeline can effectively solve the problem of low utilization in model parallel, and accelerate the training of CNN. [ABSTRACT FROM AUTHOR]

Objectives: Mars is the main target object for deep space exploration. Mars rovers, or roving probes, are important tools for surface exploration and scientific research on Mars. For the growing amount of remote sensing data collected by Mars rovers, there is an urgent need for a method that can intelligently detect novel targets of scientific value from the massive amount of images, reduce the time cost of detection planning, and provide information for subsequent scientific analysis. The traditional novel detection methods mostly include distance-based metrics and image-based reconstruction methods, distance-based metrics calculate novel scores pixel by pixel without considering spatial contextual information, and image-based re‐ construction methods focus on reconstruction of typical landscape backgrounds, and novelty is manifested by image reconstruction errors, which is not effective in extracting small novel targets such as boreholes and dust removal points. Methods: To address the above problems of traditional novel detection methods in Mars rover novel target detection, this paper proposes an improved Mars rover multispectral image depth novel target detection method, uses full convolutional self-coding neural network to extract deep features for typical landscape reconstruction, and joints Mahalanobis distance for novel target and typical landscape background separation, fully exploits the spatial and spectral dimensional features to improve the accuracy of Mars rover novel target detection results. Results: The experiments use the multispectral image dataset of Curiosity rover released by NASA ( National Aeronautics and Space Administration ), and the proposed convolution auto-encoder combined Mahalanobis distance method(CAE-M) is compared with Reed-Xiaoli detector, principal component analysis, convolution auto-encoder convolution, and generative adversarial networks on the surface of Gale crater. The results show that CAE-M outperforms previous detection methods in terms of detection accuracy and visual interpretation, and has a balanced and stable performance in different classes of novel target detection. Conclusion: The proposed CAE-M method comprehensively utilizes spatial and spectral information of multispectral images, which can help Mars rover exploration missions to quickly and intelligently screen and sort novel data with scientific value in massive data, save the time and cost of route planning, improve scientific returns. [ABSTRACT FROM AUTHOR]

Objectives: The scale of buildings and their distribution is key indicators to measure the economic and social development of a region. Therefore, it is significant to study the extraction of buildings based on remote sensing images. Existing neural network methods still have shortcomings in the completeness of building extraction and the accuracy of building edges. To solve the above problems, this paper proposes a multi-level feature fusion network (MFFNet) based on high-resolution images. Methods: Firstly, we use edge detection operators to improve the ability of the network to recognize the boundaries of buildings. Secondly, we use a multi-path convolution fusion module to extract building features from multiple dimensions, and introduce a large receptive field convolution module to break through feature extraction. The process is limited by the size of the receptive field. After fusing the extracted features, the convolutional attention module is used to compress them, and the global features are further mined by pyramid pooling, so as to achieve high-precision extraction of buildings. Results: The current mainstream UNet, pyramid scene parsing network (PSPNet),multi attending path neural network (MAPNet) and multiscale-feature fusion deep neural networks with dilated convolution (MDNNet) are used as the comparison methods, and we use Wuhan University Aerial Image Dataset, Satellite Dataset II (East Asia) and Inria Aerial Image Dataset as experimental data for testing. Compared with the other four methods, MFFNet improves intersection over union, precision, recall, F1-score and mean average precision by 1.53%, 2.65%, 2.41%, 3.32% and 1.19% on average, achieves a better effect. Conclusions: MFFNet not only accurately captures the detail features of buildings, but also strengthens the extraction and utilization of global features. It has better ex‐ traction effect on large buildings and buildings in complex environment. [ABSTRACT FROM AUTHOR]

Traditional image inpainting algorithms cannot accurately capture high-level semantics when regions involve complex non-repetitive structures, such as faces. In the past three years, the method based on deep learning has been applied to image inpainting, and the structural similarity of the repair results had increased by more than 10% compared with the traditional methods. This paper firstly expounded the research and development process of facial completion technologies, mainly introduced the face repair algorithm based on deep learning, which was divided into unsupervised and supervised categories. In each category, it focused on the various facial completion algorithm ideas that had emerged in recent years. Then it summarized the six mainstream types of image datasets. Two evaluation indexes were summarized in this paper. Finally, it discussed the future research direction of facial completion technique. [ABSTRACT FROM AUTHOR]

Due to the low randomness of character distribution and high randomness of word combination in word-based DGA domain names, the traditional detection methods are difficult to identify them. The method based on deep learning such as LSTM can capture domain's sequence features, but it lacks the word combination features and has low detection accuracy. In order to solve above problems, this paper proposed a new DGA domain name detection method based on fusion embedding layer. In domain embedding stage, it carried out a fusion vector representation based on characters and word segmentation technology . Then, it constructed a hybrid deep learning model based on CNN and BiGRU. The experimental results show that the proposed model has increased by 3 . 1 % and 4. 3% respectively in accuracy compared with the models based on CNN or LSTM alone. When dealing with the matsnu, suppobox and ngioweb DGAs, the proposed model is more effective. [ABSTRACT FROM AUTHOR]

When traditional convolutional neural networks are used in mineral species identification, due to their large parameters and the limitations of fixed input image resolution, sufficient computing resources and image preprocessing are required, which are difficult to deploy in actual exploration. For this reason, based on the depth separable convolution,combined with the attention mechanism, this paper constructs a mineral intelligent recognition model through dense connection, and this model can train multi-resolution mineral images. Ex perimental results show that the model's memory usage is only 20 Mb, the verification accuracy and the test accuracy are both higher than 90 %, the classification effect is better than the classic convolutional neural network, and it shows excellent ability to distinguish positive and negative samples. These results show that the proposed model h as fewer parameters and low memory footprint which is suitable for portable devices. This model can successfully identify mineral images with different resolutions, which has good generalization and potential application. [ABSTRACT FROM AUTHOR]

Objectives: CAE (convolutional autoencoder) combines the autoencoder neural network structure and the convolutional network structure. This paper processes the gravity data based on the deep learning, and reconstructs the gravity contour image from the noisy gravity data with the CAE. The autoencoder structure contains the equal dimensional input and output data, which goes for the gravity data processing. The convolutional network structure is widely used in image recognition recent years, which can learn and recognize the specific objects in an image. Methods: To create the training set, we generate 1000 hexahedrons with the random triaxial length parameters and the random density to simulate the natural gravity source bodies. The 2D gravity data set for these bodies is fast computed with the gravity forward formula and the noisy gravity data set is generated by adding Gaussian noise to the computed data. Meanwhile the accuracy of the length parameters and density is limited in order to improve the representativeness of the training data set. Hence the 2D gravity data and the noisy data will be output and input data of the CAE. We design 5 layers CAE. The input and output layers both are 26×26. 32 and 64 feature maps are generated with 3×3 convolution kernels in the inner layers. The training is executed using RMSProp(root mean square propagation) optimizer. Results: To test the generalization of the trained CAE, the testing set with 500 samples is generated in the same way with the training set. The relative error histogram on the testing set shows that the reconstructing error is less than 5% and most of the error is around 0. To test the recognition of the gravity features in the 2D gravity, we test the CAE with simulated gravity data which contain 2 and more gravity anomalies. The results show that the CAE can recognize all gravity anomalies in an noisy gravity image and reconstruct them and the output image shows fairly fine reconstruction from the gravity with 10% noise. The measured gravity data of Kauring testing ground are used to test the generalization performance of the CAE. The reconstructed image is compared with the traditional FFT(fast Fourier transform) filtering and db wavelet filtering results. To increase the reconstruction difficulty, the Kauring data is added with 10% Gaussian noise. The results show that the trained CAE performs significantly less error than the FFT filter and the wavelet filter. Conclusions: In terms of reconstructing performance, the CAE can avoid over processing that may eliminate the real gravity anomaly, which is often seen in filtering especially for FFT filtering. The CAE reconstructs all of the 3 gravity anomalies of Kauring while the FFT filtering reconstructs 2. And the over-smoothing artifact is less than filtering methods. The CAE can process 2D gravity data and show lower error and generate less artifacts than filtering methods in our tests with testing set, simulated gravity data and measuring data. Beyond that, this article reveals that the deep learning can learn gravity image. With the convolution structure in a neural network, the network can learn single gravity anomaly samples and extensively recognize all gravity anomalies in a gravity image, which signifies that the gravity can integrate with the artificial intelligence technology. [ABSTRACT FROM AUTHOR]

At present, super-resolution method based on CNN can obtain excellent results in the evaluation metric of peak signal- to-noise ratio and the structural similarity index, but the visual perceptual quality of super- resolution image is poor, and the details of the facial features are lost. In order to solve these problem, this paper designed a new deep neural network to predict the super-resolution wavelet coefficients to get clear super-resolution face images . Firstly, it used the prior knowledge of face images to manually give the facial features more attention. Then it introduced linear low-rank convolution in the network, and finally used the idea of long-distance dependence to supplement the details of the super-resolution images. The experimental results show that the method in this paper has achieved competitive results in the evaluation metric of peak signal-to-noise ratio and structural similarity index, and the visual perceptual quality of its super-resolution face images are excellent. [ABSTRACT FROM AUTHOR]

To solve the problems that the objects with different semantic classes are identified as co-salient objects in current co-saliency detection methods, this paper proposed a CNN and semantic correction-based co-saliency detection method (CSCCD). The proposed method first adopted the guided super pixel filter to process the super pixels obtained by SLIC and the saliency results generated by DSS, which showed clear object boundaries. Then it utilized Mask R-CNN to extract semantic features. It proposed the definitions of image semantic feature and semantic consistency. It also defined the image group semantic correction to solve the problem that detected the objects with different pose belonging to a semantic class as different semantic classes. With the concept, this paper defined image group semantic correlation class, solving semantic correlation problem of multiple images. It generated the final co-saliency detection results by fusing the saliency detection regions with the image group semantic consistent regions. The experimental results on public benchmark datasets show that this algorithm can effectively highlight the whole and outline of the object, and its comprehensive performance in objective quantification is obviously improved. [ABSTRACT FROM AUTHOR]

Aiming at the problem of less research and low detection rate of malware detection in the current container environment, this paper proposed a static detection method of malware in container based on LS TM-CNN. The purpose of the method is to detect malware before it runs, to block its attack behavior from the source, and to reduce the performance loss brought by the detection process to the operation of container. The method obtained the software to be tested in the container without agent, and extracted API( application programming interface) call sequences as program behavior data, used word2vec model to vectorize API call sequences, and extracted the semantic information and multi-dimensional local features based on LSTM and CNN respectively to detect malware. This paper implemented the method in container environment and tested on an open sourced dataset VirusShare, the results show that the method can achieve a detection rate of 99. 76% and the false alarm rate is less than 1%, which is superior to other similar methods. [ABSTRACT FROM AUTHOR]

Aiming at the problem of poor diagnosis ability and generalization ability of the trained model caused by serious imbalance of bearing fault data, this paper proposed a method of generative adversarial networks based on Wasserstein distance to balance dataset. Firstly, it trained a small number of fault samples for adversarial training. Then when the network reached the Nash equilibrium, it added the generated fault samples to the original small number of fault samples to balance the dataset. This paper proposed a diagnostic model based on global average pooled convolutional neural network. The balanced data set was input into the diagnostic model for training. The model was adaptively extracted layer by layer to achieve accurate classification diagnosis of faults. The experimental results show that the proposed diagnostic method is superior to other algorithms and models, and has strong generalization ability and robustness. [ABSTRACT FROM AUTHOR]

The automatic detection of mesoscale ocean eddies is extremely essential to monitor their dynamic changes. Therefore, effective detection of ocean eddies is vital to improve understanding of ocean dynamics. The traditional methods of detecting eddies using remote sensing data are usually based on physical parameters, geometric features, manual features or expert knowledge. In recent years, the deep learning technology has been improved by many experts. In this paper, the deep learning method is used to detect the ocean eddies from the sea surface height (SSH) data. Firstly, a multi-eddy detection model based on deep convolution neural network is proposed aiming to resolve eddy detection challenge on SSH data photoed by satellite. The model can accurately extract the feature information of eddies and fit the relationship between semantic information and sea level anomaly. Secondly, a new dataset, i.e., SCSE-Eddy, is used to train the proposed model and evaluate the performance of eddy detection method based on artificial intelligence (AI). This dataset is composed of the daily satellite remote sensing SSH data covering the South China Sea and its eastern sea area over the past 15 years. Experimental results show that, compared with the existing methods, the model proposed in this paper achieves the best performance and distinguishes close eddies well. [ABSTRACT FROM AUTHOR]

CNN (convolutional neural network) is widely used in image classification tasks. Most existing CNN-based classification models are trained as flat N-way classifiers. However, the difference among different image categories limits the capacity of the classifier. To solve the problem, this paper proposed CT-CNN (confusion tree convolutional neural network) model which combined the CT with CNN. CT-CNN first established a CT to identify the confused categories. Then it embedded the hierarchical structure of CT into CNN model, which leaded the CNN training procedure to pay more attention on strongly confused categories. Experiments on public datasets prove that CT-CNN can overcome the limitation of uneven distribution of classification difficulty between categories of large-scale datasets and achieve better performance on complex large-scale image classification tasks. [ABSTRACT FROM AUTHOR]

Because convolutional neural networks( CNN) mostly focus on global feature learning and ignore local feature information containing more details, it is difficult to improve the accuracy of indoor scene recognition. To solve this problem, this paper proposed an indoor scene recognition method based on improved global-local attention network ( GLANet). Firstly, it used GIAN et to capture the global and local features of the scene image to increase the detail information of the image features. Then, it introduced a non-local attention module into the local network to further preserve the detail features of the image through the convolution of attention diagram and feature graph. Finally, it used various features of different stages of the network for classification. Through training and verification on the MIT lndoor67 and the SUN397 dataset, the recognition accuracy of the proposed method was increased by l. 98% and 3. 07% respectively compared with the LGN method. Experimental results show that the algorithm can effectively capture global semantic information and fine spatial details, and significantly improves the recognition accuracy. [ABSTRACT FROM AUTHOR]

In order to reduce the impact of pack and obfuscation on malware classification and improve the accuracy, this paper proposed a malware classification method based on convolutional neural network and multi-feature fusion. The classifier was based on convolutional neural network and it took grayscale image of malware and the mixed sequence with AP! function call and opcode as features. The classifier had three components :image component, sequence component and fusion component. After training, the classifier could detect malware categories. The experimental results show that this method has higher classification accuracy and macro-F1 than some existing methods such as HYDRA and Orthrus. This method can classify malware more accurately and reduce the impact of packing and obfuscation. [ABSTRACT FROM AUTHOR]

Aiming at the knowledge overload in the process of online learning and the problems of data sparsity and cold start in the traditional recommendation algorithm, this paper proposed an improved learning resource recommendation algorithm based on MLP. The algorithm used the advantage of MLP in nonlinear data processing, converted the prediction mode of vector multiplication of learner characteristics and learning resource characteristics into the input mode of MLP, and improved the DN-CBR neural network recommendation model. In order to verify the effectiveness of the model, this paper built a dataset with the online learning platform data of iCourse as the sample. Compared with DN-CBR model, the improved model improves the normalized cumulative loss gain and hit rate by 1. 2 % and 3 % respectively on the dataset, which effectively improves the recommendation performance of the model. [ABSTRACT FROM AUTHOR]

Vehicle detection and identification in urban roads is of great significance for improving traffic safety and developing intelligent transportation. The traditional detection method relies on the features of manual extraction, which has been difficult to apply the complex and variable traffic scenarios, and has the defects of low recognition accuracy and high time complexity. The deep learning model can automatically extract effective features, and the generalization ability is strong, but it is difficult to classify similar vehicles more closely. To this end, this paper proposed a small vehicle target detection algorithm based on residual network. The algorithm changed the connection form of the traditional convolutional neural network to a residual connection mode based on local connection and weight sharing. At the same time, this paper changed the number of network structure control parameters, fused the features of different levels of the picture, applied the pooling layer of the region of interest to normalize the front layer features, and finally obtained the confidence and correction parameters of the target frame through the classification layer and the regression layer. Experiments show that the improved model can enhance the learning ability of the network under the premise of ensuring time efficiency, improve the average accuracy value, and obtain good detection results on the detection of similar small vehicles. [ABSTRACT FROM AUTHOR]

Aiming at the problems of time-consuming and high error rate in handwriting test scores statistics a handwriting test paper recognition method based on improved convolution neural network algorithm was proposed. In order to simplify the classification of the score» the score column of the handwritten test paper was divided to obtain ten types of numbers from 0 to 9. In order to improve the efficiency of handwritten score recognition, a classification and recognition algorithm of convolution neural network and Bayesian was proposed. The constructed convolution neural network model was used to extract the characteristics of handwritten digits. The PCA algorithm was used to reduce the dimensionality of the features.The hayes classifier was used to distinguish ten kinds of numbers from 0 to 9. the auuracy and efficiency of the algorithm were verified in the MNIST database.The paper score summation model was established and automatic summation is performed after recognition. The experimental results show that for the recognition uf the handwritten scores of 1 188 lest papers in 3 courses♦ the algorithm in this paper has a recognition rate of 98. 23% compared with other algorithms, the average recognition time per test paper is 7. 5 s, and verified its practicality. [ABSTRACT FROM AUTHOR]

A variety of Chinese characters is an important Chinese cultural symbol. Its design and operation are a hard work requiring a lot of professional knowledge. Therefore, for this work, this paper proposed a new method of Chinese fonts style transfer based on generative adversarial networks. In the experiment, it used a generative model based on the residual network structure and performed the adversarial training between the generative model and the discriminative model under the constraint of mean square error. At last, the trained generative model could be used to implement one-to-one and many-to-many style transfer between different Chinese fonts. Experiments show that,compared with the usual l1regularization method used before, the method in this paper has better performance in fonts detail generation, simplifies Chinese fonts modeling, improves the fidelity of generated images, and has better flexibility and versatility. [ABSTRACT FROM AUTHOR]

Most of the current fake face detection technologies use deep learning to identify the feature differences between real videos and fake videos, and have achieved good results on uncompressed videos. But the detection performance on compressed video will be severely reduced. Therefore, this paper proposed a fake face video detection method based on improved triplet loss to tackle the above problem. Firstly, it used an artifact map generated by the artifact map generator to deepen the feature difference between the fake face and the real face. Secondly, it used improved triplet loss to solve the problem of hard samples that were difficult to detect correctly. Finally, it chose a deep learning network that was more suitable for fake face detection to extract convolutional features. Experiments on the FaceForensics++ dataset show that proposed method is better than other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In order to ensure the normal operation of the equipment and to predict the remaining life of the bearing, this paper proposed a life prediction model based on the combination of two-dimensional convolutional neural network and an improved WaveNet. To overcome the gradient vanishing problem of the unoptimized recurrent networks in the process of prediction training, the WaveNet time-series networks structure was introduced into the model. Aiming at the situation that the original WaveNet structure was not suitable for rolling bearing vibration data, the improved WaveNet structure combined with two-dimensional convolutional neural networks was applied to the life prediction of rolling bearing. The model extracted the features of one-dimensional vibration sequence using two-dimensional convolutional networks, and then input the features to the WaveNet to predict the remaining life of the rolling bearing. Compared with the deep recurrent networks, the combined model has higher computational efficiency and more accurate results. Compared with the CNN-WaveNet-0 model, the improved model has more accurate prediction results. Compared with the deep long short-term memory networks model, the root mean square error of the prediction results of this model is reduced by 11.04%, and the scoring function of the prediction results is reduced by 11.34% . [ABSTRACT FROM AUTHOR]

In the scene of document verification，the standard deep learning face recognition method has low accuracy and poor real-time performance on embedded devices. To solve these problems，this paper proposes a modified efficient convolutional neural network（CNN）called Lightnet and adopts the transfer learning method. Lightnet is an efficient CNN module composed of depthwise separable convolution，linear bottleneck structure and attention module. After introducing the loss function AM-Softmax with additive angle margin， the network model can effectively solve the problems of redundancy parameter and vast calculation for standard CNN in the foundation of ensuring the high accuracy of face recognition. The transfer learning method can enhance the scene-identity face matching performance by freezing all the convolution layer weights of the pre-trained model and fine-tuning training in the self-made scene-identity face matching dataset. The experimental results show that the designed efficient scene-identity face matching algorithm has achieved good results in terms of verification accuracy，parameters and verification speed，and has good robustness in life scenarios. [ABSTRACT FROM AUTHOR]

Due to the small amount of microscopic features image data of traditional Chinese medicinal materials powder, unbalanced distribution of sample classes and small difference between classes, it is difficult to achieve a satisfying classification effect through traditional image classification methods. To solve the above problems,this paper proposed an improved method of deep convolution neural network based on dynamic ReLU and attention mechanism model. Firstly, it used Xception as the basic network, which had an obvious effect on small sample data classification. Secondly, it replaced the static ReLU activation function in the network with the improved dynamic ReLU function, so that each sample had its own unique ReLU parameters. Finally, it embedded the improved SE module in the network to enable the network to learn the importance of each feature channel automatically. The proposed method can make the network pay more attention to the detailed information in the image, and can solve the problem of unbalanced distribution of sample classes and small differences between classes. The experimental results show that the image classification accuracy of 56 kinds of traditional Chinese medicinal materials powder vessel is increased by about 1.5% to 93.8%, which demonstrates that the proposed method is advantageous over other image classification methods. [ABSTRACT FROM AUTHOR]

Video action recognition is a very challenging topic in the field of computer vision. The main task is to use the deep information by intelligent video analysis technology such as deep learning to recognize the human behavior. To further improve the performance of the two main frameworks, this paper proposed GSTIN for spatiotemporal feature fusion. GSTIN designed a spatiotemporal feature fusion module InBST, which could make network obtain the interactive temporal and spatial information. Based on the spatiotemporal feature fusion module InBST, GSTIN constructed a multi branch GSTIN suitable for action recognition. GSTIN was tested two classic video action recognition datasets UCFl0l and HMDB51. Compared with the action recognition networks, experimental results show that GSTIN has better recognition performance. [ABSTRACT FROM AUTHOR]

At present, the application of convolutional neural network in remote sensing image classification requires a lot of computation and occupies a lot of memory. This paper proposed a knowledge distillation method based on pruning network to classify remote sensing images. Based on the model pruning theory, it introduced attention mechanism into the network structure to strengthen the extraction of important features, and carried out model pruning on the network. Then it introduced knowledge distillation technology to carry out transfer learning to compensate the loss of classification accuracy after pruning. In order to prove the advance and reliability of the method, it carried out a comparison experiment with similar algorithms on NWPU-RESISC45 remote sensing satellite data set. The experimental results show that the proposed method not only has better performance in classification accuracy, but also has more advantages in model size. [ABSTRACT FROM AUTHOR]

As a kind of mechanical standard parts, rolling bearing is widely used in all kinds of rotating machinery. Its health condition is very important for the normal operation of equipments. Mastering its remaining useful life(RUL) can better ensure the security and efficiency of production activities . Aiming at the common problems of current machine RUL prediction methods based on deep learning: a) the prediction performance largely depends on manual feature des ign, b) the models cannot fully extract the useful features from the data, c) the multi-sensor data is not explicitly considered in the learning process . This paper proposed a new equipment RUL prediction network: parallel a set of integrated network comprised of CNN network with SE mechanism (CNN-SE-Net) and Bi-LSTM network ( CNN-SE-Bi-LSTM ). It directly used data from different sensors as inputs to the prediction network, and extracted spatial features with the improved CNN-SE-Net, extracted the temporal features with BiLSTM to establish several independent branches of RUL prediction model which could automatically learn high-level representation from input data. It obtained the final RUL prediction model by paralleling the features learned from the branches with the fully connection layer. The effectiveness of the proposed network was verified by the data of accelerated degradation test of rolling bearing, and comparing experiments with some existing improved algorithms. The results show that, face with the original multi-sensor data, the algorithm can adaptively provide accurate RUL prediction result, and the prediction performance is better than some existing prediction methods. [ABSTRACT FROM AUTHOR]

Many biomedical named entity recognition ( Bio-NER) works focus on extracting flat entities, ignore nested entities and discontinuous entities. In addition, most biomedical named entities do not follow a unified nomenclature and have many typical field features, but there is only lower efficiency. To this end, this paper proposed a CRF combined boundary assembly method, it effectively utilized the features of biomedical entities. This method consisted of three steps: boundary detecting, boundary assembling and entity discriminating. Firstly, it used a neural network model and a feature-based CRF model to identify the start and end boundaries of entities, and then generated candidate entities through boundary combination. Finally, it screened and classified the candidate entities using a multi-input convolutional neural network model. Experiments show that this method can effectively recognize nested and discontinuous entities in biomedical literature, and achieve a F-score of 81. 89% on the GENIA data set. [ABSTRACT FROM AUTHOR]

Face makeup transfer refers to the task of transferring reference makeup to people’s face while keeping the features of people’s face and showing the style of the reference makeup as much as possible. The current methods of transferring facial makeup do not fully consider the facial differences,it causes the problems such as insufficient extracted facial information. In order to avoid such problems and realize automatic transferring of face makeup,this paper proposed a face makeup transfer algorithm based on deep convolutional neural network. First,the algorithm automatically located the people’s face and features of reference makeup,and extracted the feature information of key parts. After the autonomous training of deep convolutional neural network,it automatically transferred the reference makeup to people’s face through the makeup transfer network and loss function. The simulation result shows that the algorithm takes less time,has more advantages in computing performance,and makeup transfer effect is more natural without changing the details of the original facial features. [ABSTRACT FROM AUTHOR]

Object detection for seafood is a key to intelligence marine ranching, but its real-time and accuracy need to be improved. In this paper we present an YOLOv3-based real-time object detection method for seafood. Firstly, for better real-time performance, we replace the 3×3 standard convolution in YOLOv3 with the depthwise separable convolution technology. This technology resolves standard convolution into depthwise convolution and pointwise convolution. It greatly reduces the amount of parameters and calculations of the model. Thus, we obtain a lightweight network model, named Depthwise Separable Convolution-YOLO (DSC-YOLO); Then, in the data preprocessing stage, we introduce UGAN-based image enhancement methods to relief the color distortion of underwater seafood images. This method uses an unsupervised method to automatically learn the distribution of training data and generate new data. The network consists of generator network and discriminators network. Its main idea is confronting and gaming between networks, on the basis of original data information to generate a new data information. It realizes the mapping process from underwater blurred image to underwater clear image. To be specific, we use CycleGAN to convert images in air to underwater-like images. Thus, lots of image pairs can be obtained and used to train UGAN model. Once UGAN finish training, learned generator of UGAN can transform real underwater images to clear images. Finally, we introduce Mosaic-based data augment methods to enrich image diversity. To be specific, four images are randomly selected from underwater dataset images, and operated with traditional data augmentation (rotation, translation, scaling, roll). Then, the four images are randomly cropped and spliced, obtaining a Mosaic image. The new image has the same dimension as the original images. Combining the three strategies, we obtain a real-time object detection method for seafood. In order to verify the effectiveness of our proposed method, we conduct several experiments. In these experiments, DSC-YOLO, UGAN underwater image enhancement and Mosaic data enhancement were combined to obtain four models: DSC-YOLO, DSC-YOLO+UGAN, DSC-YOLO+Mosaic, DSC-YOLO+UGAN+Mosaic. The four models compare with the YOLOv3 model over the same dataset. The results of experiments show that compared with YOLOv3 model, the model size of the proposed method reduces by 70%, the inference time reduces by 16%, the Recall (R) improves by 2.7%, the mean Average Precision 0.5 (mAP0.5) promotes by 2.4%, and the F1-Score increases by 0.4%. The results indicate that the proposed method has the potential to meet real-time requirements and deploy on mobile devices. [ABSTRACT FROM AUTHOR]

Due to the various species of fish, and the influence of different light and background environments, the recognition accuracy of some traditional fish recognition algorithms bases on color texture or feature point extrac tion is reduced, and good classification results cannot be achieved. To solve the problem, based on the existing Alex Net convolutional neural network, this paper proposes one method to speed up model training by reducing redundant convolutional layers. An item-based soft attention algorithm is applied to the improved AlexNet convolutional neural network model, which consists of four convolutional layers, one item-based soft attention layer, and two fully-con nected layers. Meanwhile, a fish recognition model is established using transfer learning methods. The test results show that the average recognition accuracy of the proposed algorithm achieves 97. 43%, which is 4.08% higher than the original AlexNet model, and the average recognition rate of some fishes achieves 99.31% and time consumption of fish recognition is reduced by 35%. In conclusion, compared with state-of-the-art fish recognition algorithms. the fish recognition algorithm proposed in this study achieve higher accuracy, lower model complexity, and stronger robustness. [ABSTRACT FROM AUTHOR]

This paper proposed a multiscale feature fusion approach for malicious HTTP request detection.Firstly,it modeled the HTTP request in both word-level and character-level.Secondly,it extracted the high level sematic information in HTTP request by using a specially designed convolutional neural network(CNN).Thirdly,it jointly learnt the multiscale representation for HTTP request with the help of multimodal learning techniques.Finally,it adopted a linear classifier for classification.Experiments on public HTTP CSIC 2010 dataset and WAF dataset show that the proposed method has large improvement on the performance against existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In the recommendation system. the inner product interaction of a single learning matrix de composition or the use of deep neural networks to capture user interaction with the project is not sufficient to effectively learn the potential characteristics of users and projects. In view of this problem. this paper proposes a collaborative filtering algorithm based on deep learning feature representation (DLFeaCF) based on display feedback and implicit feedback. The model first learns the inner product and outer product interaction between the user and the project. Then, based on the inner product, it uses the nonlinear interactive learning ability of the multi-layer perceptron (MLP) to obtain the two aspects from implicit mapping and feature mapping: on the basis of the outer product, CNN learning is used to capture the local features of the user and the project finally, combine the features in the fusion layer and obtain the prediction score, The experiments on the real MovieLens dataset show that the DLFeaCF model can achieve better recommendation performance. [ABSTRACT FROM AUTHOR]

Understanding and accurately predicting the trajectory of pedestrians is of great significance to improve the level of auto ⁃ driving technology and reduce the occurrence of traffic accidents. Aiming at the problems of low prediction accuracy and insufficient utilization of pedestrian interaction information, a pedestrian trajectory prediction model based on self⁃attention mechanism and group behavior characteristics is presented. The model considers the movement information of each pedestrian and its interaction with the surrounding pedestrians. The recurrent neural network and graph convolutional network are used to model the pedestrian’s walking state and the pedestrian’s interaction separately. In the graph convolutional net⁃ work, nodes of the graph represent the movement information of pedestrians, and edges of the graph represent the interaction between pedestrians. Self-attention mechanism is used to calculate pedestrian interaction. In addition, in order to increase the ability of the network to capture the group walk behavior and decrease the prediction error, a peer loss function is proposed. Experiments on public datasets show that the proposed model in this paper improves prediction accuracy significantly. [ABSTRACT FROM AUTHOR]

Aiming at the problems that the existing human motion recognition methods based on deep learning have large parameters and the networks were too deep and heavy, this paper proposed a lightweight two-steam fusion deep neural network model and applied this model to human action recognition. This model combined a shallow multi-scale network with a deep network, and achieved a significant reduction in the amount of model parameters and avoided the problem that network was too deep. Experiments were performed on datasets UCFlOl and HMDB51, achieving 94.0% and 69.4% recognition accuracy in ImageNet pre-training mode, respectively. Experiments show that compared with the existing human motion recognition models based on deep learning, this model greatly reduces the parameter quantity and still has high motion recognition accuracy. [ABSTRACT FROM AUTHOR]

Considering the performance deterioration of OFDM system due to narrowband interference( NBI), this paper employed deep learning into NBI mitigation of OFDM systems and proposed two structures with CNN. These two structures firstly pre-processed the received data in receiver, then adopted CNN to exploit the interference estimation in time domain. Finally it removed the impact of NBI from received signal. The simulation results demonstrate that the two systems can learn the interference effectively and improve system performance. [ABSTRACT FROM AUTHOR]