Search

Digital cartoon production requires extensive manual labor to colorize sketches with visually pleasant color composition and color shading. During colorization, the artist usually takes an existing cartoon image as color guidance, particularly when colorizing related characters or an animation sequence. Reference-guided colorization is more intuitive than colorization with other hints, such as color points or scribbles, or text-based hints. Unfortunately, reference-guided colorization is challenging since the style of the colorized image should match the style of the reference image in terms of both global color composition and local color shading. In this paper, we propose a novel learning-based framework which colorizes a sketch based on a color style feature extracted from a reference color image. Our framework contains a color style extractor to extract the color feature from a color image, a colorization network to generate multi-scale output images by combining a sketch and a color feature, and a multi-scale discriminator to improve the reality of the output image. Extensive qualitative and quantitative evaluations show that our method outperforms existing methods, providing both superior visual quality and style reference consistency in the task of reference-based colorization.

Texture plays an important role in cartoon images to represent materials of objects and enrich visual attractiveness. However, manually crafting a cartoon texture is not easy, so amateurs usually directly use cartoon textures downloaded from the Internet. Unfortunately, Internet resources are quite limited and often patented, which restrict the users from generating visually pleasant and personalized cartoon textures. In this paper, we propose a deep learning based method to generate cartoon textures from natural textures. Different from the existing photo cartoonization methods that only aim to generate cartoonic images, the key to our method is to generate cartoon textures that are both cartoonic and regular. To achieve this goal, we propose a regularization module to generate a regular natural texture with similar appearance as the input, and a cartoonization module to cartoffonize the regularized natural texture into a regular cartoon texture. Our method successfully produces cartoonic and regular textures from various natural textures.

Texture exemplar has been widely used in synthesizing 3D movie scenes and appearances of virtual objects. Unfortunately, conventional texture synthesis methods usually only emphasized on generating optimal target textures with arbitrary sizes or diverse effects, and put little attention to automatic texture exemplar extraction. Obtaining texture exemplars is still a labor intensive task, which usually requires carefully cropping and post-processing. In this paper, we present an automatic texture exemplar extraction based on Trimmed Texture Convolutional Neural Network (Trimmed T-CNN). Specifically, our Trimmed T-CNN is filter banks for texture exemplar classification and recognition. Our Trimmed T-CNN is learned with a standard ideal exemplar dataset containing thousands of desired texture exemplars, which were collected and cropped by our invited artists. To efficiently identify the exemplar candidates from an input image, we employ a selective search algorithm to extract the potential texture exemplar patches. We then put all candidates into our Trimmed T-CNN for learning ideal texture exemplars based on our filter banks. Finally, optimal texture exemplars are identified with a scoring and ranking scheme. Our method is evaluated with various kinds of textures and user studies. Comparisons with different feature-based methods and different deep CNN architectures (AlexNet, VGG-M, Deep-TEN and FV-CNN) are also conducted to demonstrate its effectiveness.

We propose an automatic video segmentation method based on an optimized SaliencyCut equipped with information centroid (IC) detection according to level balance principle in physical theory. Unlike the existing methods, the image information of another dimension is provided by the IC to enhance the video segmentation accuracy. Specifically, our IC is implemented based on the information-level balance principle in the image, and denoted as the information pivot by aggregating all the image information to a point. To effectively enhance the saliency value of the target object and suppress the background area, we also combine the color and the coordinate information of the image in calculating the local IC and the global IC in the image. Then saliency maps for all frames in the video are calculated based on the detected IC. By applying IC smoothing to enhance the optimized saliency detection, we can further correct the unsatisfied saliency maps, where sharp variations of colors or motions may exist in complex videos. Finally, we obtain the segmentation results based on IC-based saliency maps and optimized SaliencyCut. Our method is evaluated on the DAVIS dataset, consisting of different kinds of challenging videos. Comparisons with the state-of-the-art methods are also conducted to evaluate our method. Convincing visual results and statistical comparisons demonstrate its advantages and robustness for automatic video segmentation.

Automatic leaf recognition algorithm is widely used in plant taxonomy, horticulture teaching, traditional Chinese medicine research and plant protection, which is one of the research hotspots in information science. Due to the diversity of plant leaves, the variety of leaf forms, and the susceptibility to seasonal and other external factors, there is often a small inter-class variance and a large intra-class variance, which brings great challenges to the task of automatic leaf recognition. To solve this problem, we propose a leaf recognition algorithm base on the attention mechanism and dense connection. Firstly, base on dense connection, DenseNet is applied to realize the cross-layer learning of our model, which effectively improves the generalization ability of the network to the intra-class variance. At the same time, the learning ability of our model to the discriminative features such as the veins and textures of plant leaves is also improved. Secondly, we also employ the attention mechanism to further enhance the ability of our network in learning discriminative features of plant leaves. The experimental results show that our Attention DenseNet achieves a high accuracy of leaf recognition in our plant leaf database, including the challenging cases. Visual and statistical comparisons with state-of-the-art methods also demonstrate its effectiveness.

We present a convolutional neural network (CNN) equipped with a novel and efficient adaptive dual attention module (ADAM) for automated skin lesion segmentation from dermoscopic images, which is an essential yet challenging step for the development of a computer-assisted skin disease diagnosis system. The proposed ADAM has three compelling characteristics. First, we integrate two global context modeling mechanisms into the ADAM, one aiming at capturing the boundary continuity of skin lesion by global average pooling while the other dealing with the shape irregularity by pixel-wise correlation. In this regard, our network, thanks to the proposed ADAM, is capable of extracting more comprehensive and discriminative features for recognizing the boundary of skin lesions. Second, the proposed ADAM supports multi-scale resolution fusion, and hence can capture multi-scale features to further improve the segmentation accuracy. Third, as we harness a spatial information weighting method in the proposed network, our method can reduce a lot of redundancies compared with traditional CNNs. The proposed network is implemented based on a dual encoder architecture, which is able to enlarge the receptive field without greatly increasing the network parameters. In addition, we assign different dilation rates to different ADAMs so that it can adaptively capture distinguishing features according to the size of a lesion. We extensively evaluate the proposed method on both ISBI2017 and ISIC2018 datasets and the experimental results demonstrate that, without using network ensemble schemes, our method is capable of achieving better segmentation performance than state-of-the-art deep learning models, particularly those equipped with attention mechanisms.

Left ventricular (LV) segmentation is essential for the early diagnosis of cardiovascular diseases, which has been reported as the leading cause of death all over the world. However, automated LV segmentation from cardiac magnetic resonance images (CMRI) using the traditional convolutional neural networks (CNNs) is still a challenging task due to the limited labeled CMRI data and low tolerances to irregular scales, shapes and deformations of LV. In this paper, we propose an automated LV segmentation method based on adversarial learning by integrating a multi-stage pose estimation network (MSPN) and a co-discrimination network. Different from existing CNNs, we use a MSPN with multi-scale dilated convolution (MDC) modules to enhance the ranges of receptive field for deep feature extraction. To fully utilize both labeled and unlabeled CMRI data, we propose a novel generative adversarial network (GAN) framework for LV segmentation by combining MSPN with co-discrimination networks. Specifically, the labeled CMRI are first used to initialize our segmentation network (MSPN) and co-discrimination network. Our GAN training includes two different kinds of epochs fed with both labeled and unlabeled CMRI data alternatively, which are different from the traditional CNNs only relied on the limited labeled samples to train the segmentation networks. As both ground truth and unlabeled samples are involved in guiding training, our method not only can converge faster but also obtain a better performance in LV segmentation. Our method is evaluated using MICCAI 2009 and 2017 challenge databases. Experimental results show that our method has obtained promising performance in LV segmentation, which also outperforms the state-of-the-art methods in terms of LV segmentation accuracy from the comparison results.

Automatic kidney and tumor segmentation from CT volumes is essential for clinical diagnosis and surgery planning. However, it is still a very challenging problem as kidney and tumor usually exhibit various scales, irregular shapes and blurred contours. In this paper, we propose a memory efficient automatic kidney and tumor segmentation algorithm based on non-local context guided 3D U-Net. Different from the traditional 3D U-Net, we implement a lightweight 3D U-Net with depthwise separable convolution (DSC), which can not only avoid over fitting but also improve the generalization ability. By encoding long range pixel-wise dependencies in features and recalibrating the weight of channels, we also develop a non-local context guided mechanism to capture global context and fully utilize the long range dependencies during the feature selection. Thanks to the non-local context guidance (NCG), we can successfully complement high-level semantic information with the spatial information simply based on a skip connection between encoder and decoder in the 3D U-Net, and finally realize a more accurate 3D kidney and tumor segmentation network. Our proposed method was validated and evaluated with KiTS dataset, including various 3D kidney and tumor patient cases. Convincing visual and statistical results verified effectiveness of our method. Comparisons with state-of-the-art methods were also conducted to demonstrate its advantages in terms of both efficiency and accuracy.

Polyp segmentation from colonoscopy videos is of great importance for improving the quantitative analysis of colon cancer. However, it remains a challenging task due to (1) the large size and shape variation of polyps, (2) the low contrast between polyps and background, and (3) the inherent real-time requirement of this application, where the segmentation results should be immediately presented to the doctors during the colonoscopy procedures for their prompt decision and action. It is difficult to develop a model with powerful representation capability, yielding satisfactory segmentation results in a real-time manner. We propose a novel and efficient context-aware network, named PolypSeg, in order to comprehensively address these challenges. The proposed PolypSeg consists of two key components: adaptive scale context module (ASCM) and semantic global context module (SGCM). The ASCM aggregates the multi-scale context information and takes advantage of an improved attention mechanism to make the network focus on the target regions and hence improve the feature representation. The SGCM enriches the semantic information and excludes the background noise in the low-level features, which enhances the feature fusion between high-level and low-level features. In addition, we introduce the deep separable convolution into our PolypSeg to replace the traditional convolution operations in order to reduce parameters and computational costs to make the PolypSeg run in a real-time manner. We conducted extensive experiments on a famous public available dataset for polyp segmentation task. Experimental results demonstrate that the proposed PolypSeg achieves much better segmentation results than state-of-the-art methods with a much faster speed.

Accurate retinal vessel segmentation plays a critical role in the diagnosis of many relevant diseases. However, it remains a challenging task due to (1) the great scale variation of retinal vessels, (2) the existence of a large number of capillaries in the vascular network, and (3) the interactions of the retinal vessels and other structures in the images, which easily results in the discontinuities in the segmentation results. In addition, limited training data also often prohibit current deep learning models from being efficiently trained because of the overfitting problem. In this paper, we propose a novel and efficient feature pyramid cascade network for retinal vessel segmentation to comprehensively address these challenges; we call it RVSeg-Net. The main component of the proposed RVSeg-Net is a feature pyramid cascade (FPC) module, which is capable of capturing multi-scale features to tackle scale variations of retinal vessels and aggregating local and global context information to solve the discontinuity problem. In order to overcome the overfitting problem, we further employ octave convolution to replace the traditional vanilla convolution to greatly reduce the parameters by avoiding spatial redundancy information. We conducted extensive experiments on two mainstream retinal vessel datasets (DRIVE and CHASE\(\_\)DB1) to validate the proposed RVSeg-Net. Experimental results demonstrate the effectiveness of the proposed method, outperforming start-of-the-art approaches with much fewer parameters.

Symmetry detection is a method to extract the ideal mid-sagittal plane (MSP) from brain magnetic resonance (MR) images, which can significantly improve the diagnostic accuracy of brain diseases. In this article, we propose an automatic symmetry detection method for brain MR images in 2-D slices based on a 2-channel convolutional neural network (CNN). Different from the existing detection methods that mainly rely on the local image features (gradient, edge, etc.) to determine the MSP, we use a CNN-based model to implement the brain symmetry detection, which does not require any local feature detections and feature matchings. By training to learn a wide variety of benchmarks in the brain images, we can further use a 2-channel CNN to evaluate the similarity between the pairs of brain patches, which are randomly extracted from the whole brain slice based on a Poisson sampling. Finally, a scoring and ranking scheme is used to identify the optimal symmetry axis for each input brain MR slice. Our method was evaluated in 2166 artificial synthesized brain images and 3064 collected in vivo MR images, which included both healthy and pathological cases. The experimental results display that our method achieves excellent performance for symmetry detection. Comparisons with the state-of-the-art methods also demonstrate the effectiveness and advantages for our approach in achieving higher accuracy than the previous competitors.

With the rapid development of deep learning, image semantic segmentation has made great progress and become a hot topic in scene understanding of computer vision. In this paper, we propose an optimized high-resolution net (HRNet) for image semantic segmentation. Unlike traditional networks usually extract feature maps based on a high-to-low encoder, which may easily loss important shape and boundary details especially for the deeper layers with lower resolutions, our optimized HRNet can maintain high resolution features at all times using a relatively shallow and parallel network structure. To improve the ability of our model in better recognizing the objects with various scales and irregular shapes, we introduce a mixed dilated convolution (MDC) module, which can not only increase the diversity of the receptive fields, but also tackle the “gridding” problem commonly existing in the conventional dilated convolution. By minimizing fine detail lost based on a DUpsample strategy, we further develop a multi-level data-dependent feature aggregation (MDFA) module to enhance the capability of our network in better identifying the fine details especially for the small objects with fuzzy boundaries. We evaluate the optimized HRNet on four different datasets, including Cityscapes, Pascal VOC2012, CamVid and the KITTI. Experimental results validate the effectiveness of our method in improving the accuracy of image semantic segmentation. Comparisons with state-of-the-art methods also verify the advantages of our optimized HRNet in achieving better semantic segmentation performance.

Accurately segmenting retinal vessel from retinal images is essential for the detection and diagnosis of many eye diseases. However, it remains a challenging task due to (1) the large variations of scale in the retinal vessels and (2) the complicated anatomical context of retinal vessels, including complex vasculature and morphology, the low contrast between some vessels and the background, and the existence of exudates and hemorrhage. It is difficult for a model to capture representative and distinguishing features for retinal vessels under such large scale and semantics variations. Limited training data also make this task even harder. In order to comprehensively tackle these challenges, we propose a novel scale and context sensitive network (a.k.a., SCS−Net) for retinal vessel segmentation. We first propose a scale-aware feature aggregation (SFA) module, aiming at dynamically adjusting the receptive fields to effectively extract multi-scale features. Then, an adaptive feature fusion (AFF) module is designed to guide efficient fusion between adjacent hierarchical features to capture more semantic information. Finally, a multi-level semantic supervision (MSS) module is employed to learn more distinctive semantic representation for refining the vessel maps. We conduct extensive experiments on the six mainstream retinal image databases (DRIVE, CHASEDB1, STARE, IOSTAR, HRF, and LES-AV). The experimental results demonstrate the effectiveness of the proposed SCS-Net, which is capable of achieving better segmentation performance than other state-of-the-art approaches, especially for the challenging cases with large scale variations and complex context environments.

Perceptual hashing has been widely used in the field of multimedia security. The difficulty of the traditional perceptual hashing algorithm is to find suitable perceptual features. In this paper, we propose a perceptual hashing learning method for tamper detection based on convolutional neural network, where a hashing layer in the convolutional neural network is introduced to learn the features and hash functions. Specifically, the video is decomposed to obtain temporal representative frame (TRF) sequences containing temporal and spatial domain information. Convolutional neural network is then used to learn visual features of each TRF. We further put each feature into the hashing layer to learn independent hash functions and fuse these features to generate the video hash. Finally, the hash functions and the corresponding video hash are obtained by minimizing the classification loss and quantization error loss. Experimental results and comparisons with state-of-the-art methods show that the algorithm has better classification performance and can effectively perform tamper detection.

Outdoor images captured in poor weather conditions (e.g., fog or haze) commonly suffer from reduced contrast and visibility. Increasing attention has recently been paid to single image dehazing, i.e., improving image contrast and visibility. It is generally thought that the dehazing performance highly depends on the accurate depth information. In this work, we first obtain the initial depth map by using the popular dark channel prior. A unified second-order variational framework is then proposed to refine the depth map and restore the haze-free image. The introduced second-order framework has the capacity of preserving important structures in both depth map and haze-free image. Furthermore, the proposed framework performs well for several different types of haze situations. The resulting optimization problems related to depth map estimation and latent image restoration can be effectively handled using the primal-dual algorithm under a two-step numerical framework. The effectiveness of our proposed method has been demonstrated by comparing the imaging performance with several state-of-the-art dehazing methods.

Automatic plant recognition has become a research focus and received more and more attentions recently. However, existing methods usually only focused on leaf recognition from small databases that usually only contain no more than hundreds of species, and none of them reported a stable performance in either recognition accuracy or recognition speed when compared with a big image database. In this paper, we present a novel method for leaf recognition from a big hierarchical image database. Unlike the existing approaches, our method combines the textural gradient histogram with the shape context to form a more distinctive feature for leaf recognition. To achieve efficient leaf image retrieval, we divided the big database into a set of subsets based on mean-shift clustering on the extracted features and build hierarchical k-dimensional trees KD-trees to index each cluster in parallel. Finally, the proposed parallel indexing and searching schemes are implemented with MapReduce architectures. Our method is evaluated with extensive experiments on different databases with different sizes. Comparisons to state-of-the-art techniques were also conducted to validate the proposed method. Both visual results and statistical results are shown to demonstrate its effectiveness.

Texture exemplar has been widely used in example-based texture synthesis and feature analysis. Unfortunately, manually cropping texture exemplars is a burdensome and boring task. Conventional method over emphasizes the synthesis algorithm analysis and requires frequent user interactions. In this paper, we employ K-means clustering to generate patch distribution maps and calculate K-center similarity as our measurement on patch merge. Patch merging is the key to reduce over-segmentation. Even defective texture exemplars could show high global homogeneity. We detect this kind of exemplars by partitioning patch maps into non-overlapping subblocks. Comparing visual similarity between each block and the global patch map could detect the heterogeneous areas. We also introduce the Poisson disk sampling for achieving uniform exemplar cropping. Visual results show that our approach could accurately extract texture exemplars from arbitrary source images.

Texture synthesis has become a well-established area. However, researchers are mostly concerned with learning the algorithm of texture synthesis to achieve higher quality and better efficiency. We hereby propose a repetitiveness metric method to pick out an optimal texture exemplar which is used to synthesize texture. Different from conventional methods of texture analysis that emphasize on texture feature analysis for the target textures, our method focuses on repetitiveness metric of texture exemplar. To achieve a more efficient method, we firstly perform a Poisson disk sampling to extract unordered texture exemplars from the input image. Using normalized cross correlation (NCC) based on fast Fourier transformation (FFT) for each exemplar, we can get some matrices. Based on repetitiveness metric, we can assign each exemplar a score. Our method can satisfy visual requirement and accomplish high-quality work in a shorter time due to FFT. Compelling visual results and computational complexity analyses prove the validity of our work.

Traditional texture synthesis methods usually emphasized the final effect of the target textures. However, none of them focus on auto-extraction of the source texture exemplar. In this paper, we present a novel textureness metric based on Gist descriptor to accurately extract texture exemplar from an arbitrary image including texture regions. Our method emphasizes the importance of the exemplar for the example-based texture synthesis and focus on ideal texture exemplar auto-extraction. To improve the efficiency of the texture patch searching, we perform a Poisson disk sampling to crop exemplar randomly and uniformly from images. To improve the accuracy of texture recognition, we also use a SVM for the UIUC database to distinguish the texture regions and non-texture regions. The proposed method is evaluated on a variety of images with different kinds of textures. Convincing visual and statistics results demonstrated its effectiveness.

Symmetry analysis for brain images has been considered as a promising technique for automatically extracting the pathological brain slices in conventional scanning. In this article, we present a fast and robust symmetry detection method for automatically extracting symmetry axis (fissure line) from a brain image. Unlike the existing brain symmetry detection methods which mainly rely on the intensity or edges to determine the symmetry axis, our proposed method is based on a set of scale-invariant feature transform (SIFT) features, where the symmetry axis is determined by parallel matching and voting of distinctive features within the brain image. By clustering and indexing the extracted SIFT features using a GPU KD-tree, we can match multiple pairs of features in parallel based on a novel symmetric similarity metric, which combines the relative scales, orientations, and flipped descriptors to measure the magnitude of symmetry between each pair of features. Finally, the dominant symmetry axis presented in the brain image is determined using a parallel voting algorithm by accumulating the pair-wise symmetry score in a Hough space. Our method was evaluated on both synthetic and in vivo datasets, including both normal and pathological cases. Comparisons with state-of-the-art methods were also conducted to validate the proposed method. Experimental results demonstrated that our method achieves a real-time performance and with a higher accuracy than previous methods, yielding an average polar angle error within 0.69° and an average radius error within 0.71 mm. © 2013 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 23, 314–326, 2013

Single-image blind deblurring for imaging sensors in the Internet of Things (IoT) is a challenging ill-conditioned inverse problem, which requires regularization techniques to stabilize the image restoration process. The purpose is to recover the underlying blur kernel and latent sharp image from only one blurred image. Under many degraded imaging conditions, the blur kernel could be considered not only spatially sparse, but also piecewise smooth with the support of a continuous curve. By taking advantage of the hybrid sparse properties of the blur kernel, a hybrid regularization method is proposed in this paper to robustly and accurately estimate the blur kernel. The effectiveness of the proposed blur kernel estimation method is enhanced by incorporating both the L 1 -norm of kernel intensity and the squared L 2 -norm of the intensity derivative. Once the accurate estimation of the blur kernel is obtained, the original blind deblurring can be simplified to the direct deconvolution of blurred images. To guarantee robust non-blind deconvolution, a variational image restoration model is presented based on the L 1 -norm data-fidelity term and the total generalized variation (TGV) regularizer of second-order. All non-smooth optimization problems related to blur kernel estimation and non-blind deconvolution are effectively handled by using the alternating direction method of multipliers (ADMM)-based numerical methods. Comprehensive experiments on both synthetic and realistic datasets have been implemented to compare the proposed method with several state-of-the-art methods. The experimental comparisons have illustrated the satisfactory imaging performance of the proposed method in terms of quantitative and qualitative evaluations.

Leaf recognition remains a hot research topic receiving intensive attention in computer vision. In this paper, we propose deep convolutional networks with deep learning framework on the large scale of leaf databases. Different from the existing leaf recognition algorithms that mainly depend on traditional feature extractions and pattern matching operations, our method can achieve automatic leaf recognition based on deep convolutional networks without any explicit feature extraction or matching. Because it does not require any feature detection and selection, the advantages of our framework are obvious, especially for the large scale leaf databases. Specifically, we design deep convolutional networks structure and adopt fine-tuning strategy for our network initialization. In addition, we also develop a visualization-guided parameter tuning scheme to guarantee the accuracy of our deep learning framework. Our method is evaluated on several different databases with different scales. Comparison experiments are performed and demonstrate that the accuracy of our method outperforms traditional methods.

This paper present a novel algorithm for cartoon image segmentation based on the simple linear iterative clustering (SLIC) superpixels and adaptive region propagation merging. To break the limitation of the original SLIC algorithm in confirming to image boundaries, this paper proposed to improve the quality of the superpixels generation based on the connectivity constraint. To achieve efficient segmentation from the superpixels, this paper employed an adaptive region propagation merging algorithm to obtain independent segmented object. Compared with the pixel-based segmentation algorithms and other superpixel-based segmentation methods, the method proposed in this paper is more effective and more efficient by determining the propagation center adaptively. Experiments on abundant cartoon images showed that our algorithm outperforms classical segmentation algorithms with the boundary-based and region-based criteria. Furthermore, the final cartoon image segmentation results are also well consistent with the human visual perception.

Face detection is an important task in the field of computer vision, which is widely used in the field of security, human-machine interaction, identity recognition, and etc. Many existing methods are developed for image based face pose estimation, but few of them can be directly extended to videos. However, video-based face pose estimation is much more important and frequently used in real applications. This paper describes a method of automatic face pose estimation from videos based on mixture-of-trees model and optical flow. Unlike the traditional mixture-of-trees model, which may easily incur errors in losing faces or with wrong angles for a sequence of faces in video, our method is much more robust by considering the spatio-temporal consistency on the face pose estimation for video. To preserve the spatio-temporal consistency from one frame to the next, this method employs an optical flow on the video to guide the face pose estimation based on mixture-of-trees. Our method is extensively evaluated on videos including different faces and with different pose angles. Both visual and statistics results demonstrated its effectiveness on automatic face pose estimation.

Automatic plant leaf recognition has been a hot research spot in the recent years, where encouraging improvements have been achieved in both recognition accuracy and speed. However, existing algorithms usually only extracted leaf features such as shape or texture or merely adopt traditional neural network algorithm to recognize leaf, which still showed limitation in recognition accuracy and speed especially when facing a large leaf database. In this paper, we present a novel method for leaf recognition by combining feature extraction and machine learning. To break the weakness exposed in the traditional algorithms, we applied binary Gabor pattern BGP and extreme learning machine ELM to recognize leaves. To accelerate the leaf recognition, we also extract BGP features from leaf images with an offline manner. Different from the traditional neural network like BP and SVM, our method based on the ELM only requires setting one parameter, and without additional fine-tuning during the leaf recognition. Our method is evaluated on several different databases with different scales. Comparisons with state-of-the-art methods were also conducted to evaluate the combination of BGP and ELM. Visual and statistical results have demonstrated its effectiveness.

Midsagittal plane (MSP) extraction from 3D brain images is considered as a promising technique for human brain symmetry analysis. In this paper, we present a fast and robust MSP extraction method based on 3D scale-invariant feature transform (SIFT). Unlike the existing brain MSP extraction methods, which mainly rely on the gray similarity, 3D edge registration or parameterized surface matching to determine the fissure plane, our proposed method is based on distinctive 3D SIFT features, in which the fissure plane is determined by parallel 3D SIFT matching and iterative least-median of squares plane regression. By considering the relative scales, orientations and flipped descriptors between two 3D SIFT features, we propose a novel metric to measure the symmetry magnitude for 3D SIFT features. By clustering and indexing the extracted SIFT features using a k-dimensional tree (KD-tree) implemented on graphics processing units, we can match multiple pairs of 3D SIFT features in parallel and solve the optimal MSP on-the-fly. The proposed method is evaluated by synthetic and in vivo datasets, of normal and pathological cases, and validated by comparisons with the state-of-the-art methods. Experimental results demonstrated that our method has achieved a real-time performance with better accuracy yielding an average yaw angle error below 0.91° and an average roll angle error no more than 0.89°.

Since human face is often not clear in the surveillance video, this paper proposes a retrieval method on the whole body with fine-grained feature extraction. This method first extracts foreground region of human movement based on Gaussian mixed model (GMM). The human body is divided into two parts, head and below the head, based on the human morphological features and skin color. There are three parts coupled with the whole body, and then, we extract color feature for each part. Secondly, the human body is divided into front and back feature samples according to the a priori knowledge. Calculate the gap between the retrieval eigenvectors and the eigenvectors of the target body, then determine whether match. The experimental results show that this method maintains better retrieval precision when the recall rate is high. This paper target retrieve is real time on video.

As leaf images can be captured more and more conveniently, automatic leaf recognition has been the key to help us identify different kinds of plant. However, fast and robust leaf recognition is still an unsolved problem, because the leaf images can be collected among different growing stages and with different shapes and colors. In this paper, we present a fast and robust method for leaf recognition by identifying leaves based on rotation invariant shape context (RISC) and summed squared differences (SSD) color matching. Unlike the existing shape context, which is only scale and translational invariant, our proposed method can recognize the leaves with different rotational angles, namely rotation invariant. To distinguish plants having the same shape context but with different colors, we use SSD color matching to measure the similarity of different leaves. The combination of RISC and SSD makes our leaf recognition method faster and much more robust than conventional shape context method. In our experiment, we obtained convincing results to demonstrate its effectiveness.

Automatic leaf recognition has been a hot research topic as digital leaf images capturing becomes more and more convenient and popular, which is also essential for plant education. However, fast and robust automatic recognition for leafs remains a challenging problem. In this paper, we present a novel method for leaf recognition based on texture matching. To measure the similarity of two leaves which normally have different color distributions, lighting distributions, and viewing angles, we use binary Gabor pattern (BGP) matching to efficiently extract the texture feature by transforming an image into a pattern histogram. Support vector machine (SVM) classifier is then used to determine the final recognition results. Due to the robustness of combination of BGP and SVM, our method achieves an average recognition rate of up to 95.2 %.

Images and videos usually express perceptual information with meaningful structures. By removing fine details and reserving important structures, video texture smoothing can better display the useful structural information, and thus is significant in the video understandings for both human and computers. Compared with the image texture smoothing, video texture smoothing is much more challenging. This paper proposes a novel video texture smoothing method through combining existing Relative Total Variation (RTV) and optical flow matching. By considering both special relationship and color/gradient similarity between adjacent frames, we build an optimization framework with two novel regularization terms and solve the smoothed video texture via iterations. Convincing experiment results demonstrate the effectiveness of our method.

In this paper, we propose a novel method for automatic hand gesture recognition from images based on shape context. Unlike conventional approaches, our method can robustly detect hand gestures rotated with arbitrary angle. Specifically, we improve the existing shape context to rotational invariant by creating a new log-polar space based on the tangent line of the boundary points. We first align the two hand gestures by solving a correspondence problem. The similarity of two hand gestures are obtained by calculating the shape distance based on our proposed rotational invariant shape context. Finally, the best matched result is identified by retrieving the gesture with the maximal shape similarity. Our method is evaluated using a standard simulated gesture dataset. Experimental results show that our method can accurately identify hand gestures, either with or without rotation. Comparison experiments also suggest that our method outperforms existing hand gesture recognition methods based on conventional shape context.

Textures are ubiquitous and usually have fine details as well as meaningful structures on the surfaces. Many algorithms have been proposed for texture smoothing and structure extraction, but none of them obtained a satisfactory effect because the optimization procedure is very challenging. In this paper, we present a texture smoothing method based on a novel adaptive total variation framework. We propose using absolute variation to separate the important structures and the fine details of a texture. Then, a sharp total variation (STV) based on absolute variation and inherent variation is used to reinforce the structure edges during the smoothing process. Finally, by integrating our proposed STV and the existing relative total variation (RTV), we can not only smooth the fine detail of the textures, but also maintain the salient structures. Experiments show that our method outperforms the existing methods in terms of detail smoothing and salient structures preserving.

The feature extraction is the most critical step in image retrieval. Among various local feature extraction methods, scale-invariant feature transform (SIFT) has been proven to be the most robust local invariant feature descriptor, which is widely used in the field of image matching and retrieval. However, the SIFT algorithm has a disadvantage that the algorithm will produce a large number of feature points and is not suited for widely using in the field of image retrieval. Firstly, a novel significant measure algorithm is proposed in this paper, and the regions of interest in images are obtained. Then, SIFT features are extracted from salient regions, reducing the number of SIFT features. Our algorithm also abstracts color features from salient regions, and this method overcomes SIFT algorithm’s drawback that could not reflect image’s color information. The experiments demonstrate that the integrated visual saliency analysis-based feature selection algorithm provides significant benefits both in retrieval accuracy and in speed.

level set formulation and partial differential equations. The weighted endocardium surface and epicardium surface are then varying according to the local echo sampling density. To handle the noisy and non-uniform regions within the echo data, we used a regularization term to adaptively and continuously following the gradient descent of the topological change. Finally, the optimal the endocardium surface and epicardium surface are obtained by solving the minimal energy for the two weighted surfaces using a brief propagation algorithm. Results: Even for the typically low signal-to-noise and small field-ofview in echo images, our method can successfully segment the left ventricle by robustly extracting the endocardium surface and epicardium surface. The proposed method achieves a mean accuracy of 6.1±1.3% volume differences against the gold standard for MRI segmentation method and a mean accuracy of 14.5±2.1% for segmentation of all four chambers. Conclusion: Using a level set approach to segment left ventricle in 3D echocardiography provides a fully automatic tool for physicians to obtain the anatomical and diagnostic information for cardiac functional analysis, without requiring any user input or any other assistants. The ability of our method to automatically and accurately extract endocardium and epicardium from echo has promising clinical potential in further medical diagnosis and surgery planning.