Search

Since multimodal imaging technology is able to provide multiple perspectives on the lesion, it has become increasingly important in clinical diagnosis and treatment planning. In this paper, a novel medical image fusion using gradient domain-guided filter random walk (GDGFRW) and side window filtering (SWF) in the framelet transform (FT) domain is presented. Firstly, FT is performed on the original multimodal source images to obtain the corresponding approximate and residual representations. Secondly, a novel model—GDGFRW, which combines the superiorities of both gradient domain guided filtering and random walks—is constructed to interpret the approximate sub-bands, while the residual sub-bands are fused by SWF. Finally, the fused approximate sub-bands and residual sub-bands undergo inverse FT to generate the final fused image. To verify its effectiveness, the proposed method was tested on different categories of multimodal medical image fusion issues, in more than 40 pairs of source images. The proposed method outperforms the current representative ones in terms of both subjective visual performance and objective assessment.

In recent years, skeleton-based action recognition, graph convolutional networks, have achieved remarkable performance. In these existing works, the features of all nodes in the neighbor set are aggregated into the updated features of the root node, while these features are located in the same feature channel determined by the same $1\times 1$ convolution filter. This may not be optimal for capturing the features of spatial dimensions among adjacent vertices effectively. Besides, the effect of feature channels that are independent of the current action on the performance of the model is rarely investigated in existing methods. In this paper, we propose cross-channel graph convolutional networks for skeleton-based action recognition. The features fusion mechanism in our network is cross-channel, i.e, the updated feature of the root node is derived from different feature channels. Because different feature channels come from different $1 \times 1$ convolution filters, the cross-channel fusion mechanism significantly improves the ability of the model to capture local features among adjacent vertices. Moreover, by introducing a channel attention mechanism to our model, we suppress the influence of feature channels unrelated to action recognition on model performance, which improves the robustness of the model against the feature channels independent of the current action. Extensive experiments on the two large-scale datasets, NTU-RGB+D and Kinetics-Skeleton, demonstrate that the performance of our model exceeds the current mainstream methods.

The cost of misclassifying a malware program as normal is often higher than that of misclassifying a normal program as malware. Therefore, how to improve the detection accuracy of malware programs is a very important problem. This paper proposes a deep learning malware program detection algorithm based on attention mechanism. Word2Vec model is used to map the Application programming interface (API) into word vectors, and all word vectors of each sample are arranged into a matrix with the same size. On this basis, residual network is used to extract features of samples. The features are input into the attention mechanism to learn the similarity between samples. Then, the features are weighted with the similarity to obtain the new features with better robustness. The new features and the original features are added element by element to obtain the sample features more suitable for classification. Finally, samples are classified by classifier. Experiments show that the classification effect of the proposed method is better than that of the traditional machine learning method.

Accurate road centerline extraction is very important for many vital applications. In the road extraction, the acquisition of labeled data is time-consuming; thus, there is only a small amount of labeled samples in reality. To solve the problem of limited labeled samples, a semi-supervised road centerline extraction is proposed, which incorporates high-level feature selection, Markov random field (MRF), and ridge transversal method. The proposed road extraction approach consists of three steps: multiple features extraction, semi-supervised road area extraction, and road centerlines extraction. To get more abstract and discriminative high-level features, we apply multiple-feature adaptive sparse representation in mid-level features in different views generated by different prototype sets. To obtain an accurate road area result, we combine the feature learning framework with MRF. Then, we integrate Gabor filters and nonmaxima suppression with the ridge transversal method to extract centerlines. It is verified the proposed method achieves comparable performance with the state-of-the-art methods in terms of visual and quantitative aspects.

Cancer remains one of the most threatening diseases, which kills millions of lives every year. As a promising perspective for cancer treatments, anticancer peptides (ACPs) overcome a lot of disadvantages of traditional treatments. However, it is time-consuming and expensive to identify ACPs through conventional experiments. Hence, it is urgent and necessary to develop highly effective approaches to accurately identify ACPs in large amounts of protein sequences. In this work, we proposed a novel and effective method named ME-ACP which employed multi-view neural networks with ensemble model to identify ACPs. Firstly, we employed residue level and peptide level features preliminarily with ensemble models based on lightGBMs. Then, the outputs of lightGBM classifiers were fed into a hybrid deep neural network (HDNN) to identify ACPs. The experiments on independent test datasets demonstrated that ME-ACP achieved competitive performance on common evaluation metrics.

Accurate road extraction from high-resolution aerial imagery has many applications such as urban planning and vehicle navigation system. The common road extraction methods are based on classification algorithm, which needs to design robust handcrafted features for road. However, designing such features is difficult. For the road centerlines extraction problem, the existing algorithms have some limitations, such as spurs, time consuming. To address the above issues to some extent, we introduce the feature learning based on deep learning to extract robust features automatically, and present a method to extract road centerlines based on multiscale Gabor filters and multiple directional non-maximum suppression. The proposed algorithm consists of the following four steps. Firstly, the aerial imagery is classified by a pixel-wise classifier based on convolutional neural network (CNN). Specifically, CNN is used to learn features from raw data automatically, especially the structural features. Then, edge-preserving filtering is conducted on the resulting classification map, with the original imagery serving as the guidance image. It is exploited to preserve the edges and the details of the road. After that, we do some post-processing based on shape features to extract more reliable roads. Finally, multiscale Gabor filters and multiple directional non-maximum suppression are integrated to get a complete and accurate road network. Experimental results show that the proposed method can achieve comparable or higher quantitative results, as well as more satisfactory visual performance.

Recent progress on skeleton-based action recognition has been substantial, benefiting mostly from the explosive development of Graph Convolutional Networks (GCN). However, prevailing GCN-based methods may not effectively capture the global co-occurrence features among joints and the local spatial structure features composed of adjacent bones. They also ignore the effect of channels unrelated to action recognition on model performance. Accordingly, to address these issues, we propose a Global Co-occurrence feature and Local Spatial feature learning model (GCLS) consisting of two branches. The first branch, based on the Vertex Attention Mechanism branch (VAM-branch), captures the global co-occurrence feature of actions effectively, the second, based on the Cross-kernel Feature Fusion branch (CFF-branch), extracts local spatial structure features composed of adjacent bones and restrains the channels unrelated to action recognition. Extensive experiments on two large-scale datasets, NTU-RGB+D and Kinetics, demonstrate that GCLS achieves the best performance when compared to the mainstream approaches.

Outdoor images are often degraded by aerosols suspending in atmosphere in bad weather conditions like haze. To cope with this phenomenon, researchers have proposed many approaches and single image based techniques draw attention mostly. Recently, a fusion-based strategy achieves good results, which derives two enhanced images from single image and blends them to recover haze-free image. However, there are still some deficiencies in the fusion-input images and weight maps, which leads their restoration less natural. In this paper, we propose a multi-scale fusion scheme for single image dehazing. We first use an adaptive color normalization to eliminate a common phenomenon, color distortion, in haze condition. Then two enhanced images, including our newly presented local detail enhanced image, are derived to be blended. Thereafter, five haze-relevant features of dark channel, clarity, saliency, luminance and chromatic are investigated since those can serve as weight maps for fusion. Dark channel, clarity and saliency features are finally selected due to their expression abilities and less interconnection. The fusion is processed with a pyramid strategy layer-by-layer. The multi-scale blended images are combined in a bottom-up manner. At last quantitative experiments demonstrate that our approach is effectiveness and yields better results than other methods.

Standard classification tasks have already achieved good results in computer vision. However, the task of Architectural style classification yet faces many challenges, since the rich inter-class relationships between different styles may disturb the classification accuracy. To better classify buildings, we propose a feature extraction module based on image preprocessed with Deformable Part-based Models (DPM). Specifically, we first use DPM to remove elements that are not related to classification, and capture representative elements of buildings, and then these elements are sent to our feature extraction module. In our feature extraction module, we adopt our improved ensemble projection method to maximize the inter-class distance and minimize the intra-class distance to find the common features in the same style and differences among different styles. Finally, the performances of several classifiers are tested and the best one of SVM classifier is selected to output the ultimate accuracy. Experimental results show that our approach achieves promising performance and is superior to previous methods.

Image classification has been an incredibly active research topic in recent years with widespread applications. Researchers have put forward many remarkable techniques and semi-supervised learning (SSL) is one among them. However, due to not taking the relationship of samples among different classes in consideration, previous approaches cannot often get a clear decision boundary. In this paper, we propose an improved classification model on the basis of SSL. First, we adopt a deformable part-based model to capture a stable global structure and salient objects, and then, we find a better decision boundary by our classification algorithm-based on an improved ensemble projection (IEP). Our IEP exploits the weighted average method. To evaluate the effectiveness of our approach, we do experiments not only with the LandUse-21 (L-21) data set, but also with an architecture style data set. Experimental results show that our approach is capable of achieving the state-of-the-art performance on the two data sets. For each class in L-21 data set, when 50 images are randomly chosen as training images, the multi-class average precision increases to 97.63%. Besides, for the architecture style data set, we achieve the best result with about 80% accuracy and have about a 10% improvement over the previous best work. Although there are a small number of labeled data used to train, we get the satisfactory performance.

Effectively learning the appearance change of a target is the key point of an online tracker. When occlusion and misalignment occur, the tracking results usually contain a great amount of background information, which heavily affects the ability of a tracker to distinguish between targets and backgrounds, eventually leading to tracking failure. To solve this problem, we propose a simple and robust reliable memory model. In particular, an adaptive evaluation strategy (AES) is proposed to assess the reliability of tracking results. AES combines the confidence of the tracker predictions and the similarity distance, which is between the current predicted result and the existing tracking results. Based on the reliable results of AES selection, we designed an active–frozen memory model to store reliable results. Training samples stored in active memory are used to update the tracker, while frozen memory temporarily stores inactive samples. The active–frozen memory model maintains the diversity of samples while satisfying the limitation of storage. We performed comprehensive experiments on five benchmarks: OTB-2013, OTB-2015, UAV123, Temple-color-128, and VOT2016. The experimental results show that our tracker achieves state-of-the-art performance.

This paper presents a method for the reconnection of contour lines from scanned color images of topographical maps based on graphics processing unit (GPU) implementation. The extraction of contour lines, which are shown with brown color on USGS maps, is a difficult process due to aliasing and false colors induced by the scanning process and due to closely spaced and intersecting/overlapping features inherent to the map. First, an effective method is presented for contour line reconnection from scanned topographical maps based on CPU. This method considers both the distance and direction between the two broken points of the contour lines. It gets better performance and has high connection rate, but the time complexity of the algorithm is nonlinear with the increasing size of topographical map. Second, the advantage of the massively parallel computing capability of GPU with the compute unified device architecture is taken to improve the algorithm. Finally, a better performance has been achieved based on the open source computer vision library. The experimental results show that the GPU implementation with loop-based patterns achieves a speedup of 1360× and the identical result compared with the implementation on CPU.

Deep neural networks (DNN) have been widely used for image classification. One major hurdle of deep learning approaches is that large sets of labeled data are necessary, which can be prohibitively costly to obtain, particularly in architectural style classification. Data augmentation can alleviate this labeling effort. In this paper, we use data augmentation to increase the number of architectural style datasets. To extract building elements, the inputs are preprocessed by Deformable Part Model (DPM) first, and then the preprocessed images are sent to the data augmentation to increase the number of images. Next, we design a deep neural network based on GoogLeNet. The proposed network aims to learn robust feature embeddings to improve architectural style classification performance. Finally, architectural style can be classified by the robust feature embeddings. Experimental results show that our approach achieves promising performance and is superior to previous methods.

It is very important to extract accurate road networks from high resolution remote sensing images for various applications, such as transportation database updating. However, existing approaches cannot get satisfactory results. We propose an improved road networks extraction from remote sensing images based on the shear transform, the directional segmentation, the road probability, shape features and a skeletonization algorithm. The proposed method includes the following steps. First, we combine shear transform with directional segmentation to get the initial road regions. Second, road map based on Mahalanobis distance and thresholding is fused with the initial road regions to improve accuracy. Third, road shape features filtering and hole filling are used to extract reliable road segments. Finally, the road centerlines are extracted by an automatic subvoxel precise skeletonization method based on fast marching. Road networks are then generated by post-processing. Experimental results show that the proposed method can extract smooth and correct road centerlines.

In optical character recognition, text strings should be extracted from images first. But only the complete text strings can accurately express the meanings of the words, so the extracted individual characters should be grouped into text strings before recognition. There are lots of text strings in topographic maps, and these texts consist of the characters with multi-colored, multi-sized and multi-oriented, and the existing methods cannot effectively group them. In this paper, a dynamic character grouping method is proposed to group the characters into text strings based on four consistency constraints, which are the color, size, spacing and direction respectively. As we know that the characters in the same word have similar colors, sizes and distances between them, and they are also on some curve lines with a certain bending, but the characters in different words are not. Based on these features of the characters, the background pixels around the characters are expanded to link the characters into text strings. In this method, due to the introduction of the color consistency constraint, the characters with different colors can be grouped well. And this method can deal with curved character strings more accurately by the improved direction consistency constraint. The final experimental results show that this method can group the characters more efficiently, especially for the case in which the beginning or the end characters of words are close to the characters of the other words.

This paper extends our previous DESIQUE [1] algorithm to a local-and-global way (LaG-DESIQUE) to blindly measure image quality without training on human opinion scores. The local DESIQUE extracts block-based log-derivative features and evaluates image quality through measuring the multivariate Gaussian distance between selected natural and test image patches. The global DESIQUE extracts image-based log-derivative features and image quality is estimated based on a two-stage framework, which was trained on a set of regenerated distorted images with their quality scores estimated by MAD [2] algorithm. The overall quality is the weighted average of local and global DESIQUE scores. Test on several image databases demonstrates that LaG-DESIQUE performs competitively well in predicting image quality.

Road information plays an important role in many civilian and military applications. This paper proposes an improved method for road centerlines extraction, which is based on shear transform, directional segmentation, shape features filtering, directional morphological filtering, tensor voting, multivariate adaptive regression splines (MARS) and enhanced broken lines connection. The proposed method consists of five steps. Firstly, directional segmentation based on spectral information and shear transform is used to segment the images for obtaining the initial road map. Shear transform is introduced to overcome the disadvantage of the loss of the road segment information. Secondly, we perform hole filling to remove the holes due to noise in some road regions. Thirdly, reliable road segments are extracted by road shape features and directional morphological filtering. Directional morphological filtering can separate road from the neighboring non-road objects to ensure the independence of each road target candidate. Fourthly, tensor voting and MARS are exploited to extract smooth road centerlines, which overcome the shortcoming that the road centerlines extracted by the thinning algorithm have many spurs. Finally, we propose an enhanced broken lines connection algorithm to generate a complete road network, in which a new measure function is constructed and spectral similarity is introduced.Display Omitted We apply spectral information and shear transform in directional segmentation.Directional morphological filtering is adopted to ensure the independence of road.We propose an enhanced broken lines connection algorithm. Road information plays an important role in many civilian and military applications. Road centerlines extraction from high-resolution remote sensing images can be used to update a transportation database. However, it is difficult to extract a complete road network from high-resolution images, especially when the color of road is close to that of background. This paper proposes an improved method for road centerlines extraction, which is based on shear transform, directional segmentation, shape features filtering, directional morphological filtering, tensor voting, multivariate adaptive regression splines (MARS) and enhanced broken lines connection. The proposed method consists of five steps. Firstly, directional segmentation based on spectral information and shear transform is used to segment the images for obtaining the initial road map. Shear transform is introduced to overcome the disadvantage of the loss of the road segment information. Secondly, we perform hole filling to remove the holes due to noise in some road regions. Thirdly, reliable road segments are extracted by road shape features and directional morphological filtering. Directional morphological filtering can separate road from the neighboring non-road objects to ensure the independence of each road target candidate. Fourthly, tensor voting and MARS are exploited to extract smooth road centerlines, which overcome the shortcoming that the road centerlines extracted by the thinning algorithm have many spurs. Finally, we propose an enhanced broken lines connection algorithm to generate a complete road network, in which a new measure function is constructed and spectral similarity is introduced. We evaluate the performance on the high-resolution aerial and QuickBird satellite images. The results demonstrate that the proposed method is promising.

Digital image technology has been widely used in wildlife monitoring, due to its advantages of non-obligatory, non-contact and non-invasive. However, the primary problem needing to be solved is how to detect and track the wild animals in these images and videos. This paper proposes a face-tracking algorithm for Sichuan golden monkeys by combing SVM and TLD, named as S-TLD. This algorithm is proposed based on the basic framework of TLD, and SVM is used as an alternate classifier to detect the face again, when the object appears in the video for the first time or TLD fails to tracking. Once SVM accomplishes the face detection, TLD completes the following tracking task. Experimental results demonstrate that S-TLD can be applied to the face detection and tracking of Sichuan golden monkeys in the videos, the location and facial information of the monkeys are very helpful for the study of their behavior habits.

Multi-focus image fusion aims to fuse a set of images with different focus objects into one image which contains more information than that of individual source image. Along with the development of multi-scale geometric analysis tools, shearlet has been widely applied into image processing. In order to obtain an image with every object in focus, we propose a novel image fusion method based on shearlet and particle swarm optimization. Shearlet is used to decompose the images. Particle swarm optimization is applied to optimize the weighted factors of fusion. Experimental results show that our method can acquire better fusion quality than other methods.

Image fusion is a technology which can effectively enhance the utilization ratio of image information, the accuracy of target recognition and the interpretation ability of image. However, traditional fusion methods may lead to the information loss and image distortion. Hence a novel remote sensing image fusion method is proposed in this paper. As one of the multi-scale geometric analysis tools, Shearlet has been widely used in image processing. In this paper, Shearlet is used to decompose the image. Genetic Algorithm, a intelligent optimization algorithm, is also applied to image fusion and it aims to optimize the weighted factors in order to improve the quality of fusion. Experimental results prove the superiority and feasibility of this method.

In optical character recognition, text strings are extracted from images so that it can be edited, formatted, indexed, searched, or translated. Characters should be grouped into text strings before recognition, but the existing methods cannot group characters accurately. This paper proposes a new approach to group characters into text strings based on the consistency constraints. According to the features of the characters in the topographic maps, three kinds of consistency constraints are proposed, which are the color, size and direction consistency constraint respectively. In the proposed method, due to the introduction of the color consistency constraint, the characters with different colors can be grouped well; and this method can deal with the curved character strings more accurately by the improved direction consistency constraint. The final experimental results show that this method can group the characters more accurately, and lay a good foundation for text recognition.

Accurate extraction of road networks from high resolution remote sensing images is a problem not satisfactorily solved by existing approaches, especially when the color of road is close to that of background. This paper studies a new road networks extraction from remote sensing images based on the shear transform, the directional segmentation, shape features and a skeletonization algorithm. The proposed method includes the following steps. Firstly, we combine shear transform with directional segmentation to get road regions. Secondly, road shape features filtering are used to extract reliable road segments. Finally, the road centerlines are extracted by a skeletonization algorithm. Road networks are then generated by post-processing. Experimental results show that this method is efficient in road centerlines extraction from remote sensing images.

A method of missing chip detection based on vacuum testing is presented in the paper in view of the requirement of response time in high-speed loading machine. On the basis of analyzing the factors that influence response time in theoretical, the experimental platform of the detection system is set up. Some key factors that affect the response time are analyzed by experiments, and the improvement of experimental platform completed in final. The experimental results show that the improved detection system has faster response time on the basis of guaranteeing the stability, the response time of this system is about 38ms, and it has put into use.