Search

Recent advancements in Generative Adversarial Networks (GANs) have led to significant improvements in various image generation tasks including image synthesis based on text descriptions. In this paper, we present an enhanced Attentional Generative Adversarial Network (e-AttnGAN) with improved training stability for text-to-image synthesis. e-AttnGAN’s integrated attention module utilizes both sentence and word context features and performs feature-wise linear modulation (FiLM) to fuse visual and natural language representations. In addition to multimodal similarity learning for text and image features of AttnGAN [1], similarity and feature matching losses between real and generated images are included while employing classification losses for “significant attributes”. In order to improve the stability of the training and solve the mode collapse issue, spectral normalization and two-time scale update rule are used for the discriminator together with instance noise. Our experiments show that e-AttnGAN outperforms state-of-the-art methods using the FashionGen and DeepFashion-Synthesis datasets in terms of inception score, R-precision and classification accuracy. A detailed ablation study has been conducted to observe the effect of each component.

In this paper, we present a query & retrieval system specially designed for upper-body clothing products such as Shirts, Jackets etc. To facilitate precise learning, each clothing product image is divided into its constituent parts (e.g. torso and sleeves) to enable the learning of the respective attributes of the parts. The regions associated with the constituent clothing parts are first identified by an algorithm which exploits the structure of the upper-body clothing items. Our proposed “Part Guided” Convolutional Neural Network (CNN) is then “steered” towards different regions according to the attributes and expedite the training. In order to extract compact color information and enable flexible query formulation in a richer space, a Markov Random Field based color encoding method is adopted. A detailed dataset called “Shirt Attributes Dataset” was constructed to evaluate the performance of the proposed system. The proposed system allows more specific query formulations and achieves better results compared to different baselines in terms of classification and retrieval accuracy.

Autoimmune diseases (AD) are the abnormal response of the immune system of the body to healthy tissues. ADs have generally been on the increase. Efficient computer aided diagnosis of ADs through classification of the human epithelial type 2 (HEp-2) cells become beneficial. These methods make lower diagnosis costs, faster response and better diagnosis repeatability. In this paper, we present an automated HEp-2 cell image classification technique that exploits the sparse coding of the visual features together with the Bag of Words model (SBoW). In particular, SURF (Speeded Up Robust Features) and SIFT (Scale-invariant feature transform) features are specially integrated to work in a complementary fashion. This method helps greatly improve the cell classification accuracy. Additionally, a hierarchical max-pooling method is proposed to aggregate the local sparse codes in different layers to provide final feature vector. Furthermore, various parameters of the dictionary learning including the dictionary size, the learning iteration number, and the pooling strategy is also investigated. Experiments conducted on publicly available datasets show that the proposed technique clearly outperforms state-of-the-art techniques in cell and specimen levels.

In this paper, we introduce a multi-dimensional approach to the problem of reconstruction of MR image sequences that are highly undersampled in k-space. By formulating the reconstruction as a high-order low rank tensor plus sparse tensor decomposition problem, we propose an efficient numerical algorithm based on the alternating direction method of multipliers (ADMM) to solve the optimization. Using Tucker representation, the sparse component is learnt efficiently with different sparsifying matrices along the modes of dynamic MR data. To estimate the low rank tensor, a convex cost function is defined to be the weighted sum of nuclear norms of its 3 unfolding matrices. Through extensive experimental results we show that our proposed method achieves superior reconstruction quality, compared to the state-of-the-art reconstruction methods. Graphical abstractDisplay Omitted HighlightsA novel multi-dimensional analysis model is learnt to recover higher quality MRI sequences.The dynamic MRI reconstruction is formulated as a higher-dimensional low rank plus sparse tensor reconstruction problem.An efficient numerical algorithm based on ADMM is proposed to solve the optimization problem.

In this paper, we introduce a dictionary learning based approach applied to the problem of real-time reconstruction of MR image sequences that are highly undersampled in k-space. Unlike traditional dictionary learning, our method integrates both global and patch-wise (local) sparsity information and incorporates some priori information into the reconstruction process. Moreover, we use a Dependent Hierarchical Beta-process as the prior for the group-based dictionary learning, which adaptively infers the dictionary size and the sparsity of each patch; and also ensures that similar patches are manifested in terms of similar dictionary atoms. An efficient numerical algorithm based on the alternating direction method of multipliers (ADMM) is also presented. Through extensive experimental results we show that our proposed method achieves superior reconstruction quality, compared to the other state-of-the- art DL-based methods.

In this paper, we present the enhanced Attentional Generative Adversarial Network (e-AttnGAN) with improved training stability for text-to-image synthesis. e-AttnGAN's integrated attention module utilizes both sentence and word context features and performs feature-wise linear modulation (FiLM) to fuse visual and natural language representations. In addition to multimodal similarity learning for text and image features of AttnGAN, cosine and feature matching losses of real and generated images are included while employing a classification loss for "significant attributes". In order to improve the stability of the training and solve the issue of model collapse, spectral normalization and two-time scale update for the discriminator are used together with instance noise. Our experiments show that e-AttnGAN outperforms state-of-the-art methods on the FashionGen and DeepFashion-Synthesis datasets.

Recent advances in Generative Adversarial Networks (GANs) have made it possible to conduct multi-domain image-to-image translation using a single generative network. While recent methods such as Ganimation and SaGAN are able to conduct translations on attribute-relevant regions using attention, they do not perform well when the number of attributes increases as the training of attention masks mostly rely on classification losses. To address this and other limitations, we introduce Attribute Manipulation Generative Adversarial Networks (AMGAN) for fashion images. While AMGAN's generator network uses class activation maps (CAMs) to empower its attention mechanism, it also exploits perceptual losses by assigning reference (target) images based on attribute similarities. AMGAN incorporates an additional discriminator network that focuses on attribute-relevant regions to detect unrealistic translations. Additionally, AMGAN can be controlled to perform attribute manipulations on specific regions such as the sleeve or torso regions. Experiments show that AMGAN outperforms state-of-the-art methods using traditional evaluation metrics as well as an alternative one that is based on image retrieval.

Proposing a new superpixel extraction method by adding the gradient information.Intelligently select the input patches based on proposed superpixel method.Proposing a technique for cell extraction from input masks of specimen images.Increasing the accuracy of cell and image level classification.Reduce the timing of HEp-2 classification by decreasing the number of input patches. Display Omitted Autoimmune Diseases (AD) are among the top 10 leading causes of death in female children and women in all age groups up to 64 years. They are widely diagnosed by various antibody tests that typically apply the Indirect Immunofluorescence (IIF) to the Human Epithelial Type-2 (HEp-2) cells. Automated classification of Hep-2 cells has attracted much research interest in recent years, and many of these approaches employ patch-based models and the Bag of Words (BoW) scheme, but often face several typical constraints such as the need to process a huge number of overlapped image patches, tuning of various parameters and etc. We propose a superpixel based Hep-2 cell classification technique by calculating the sparse codes of image patches which are prepared in a more intelligent way. In particular, the superpixel approach guides the determination of the right image patches while aggregating the neighboring pixels of similar patterns. In addition, we introduce extended superpixels which is able to capture the most discriminative gradient information across the boundary of the HEp-2 cell images. The proposed technique has been evaluated over two public datasets (ICPR2012 and ICIP2013) and experiments show superior performance in both classification accuracy and speed of model training and cell classification.

Face identification (FI) has made significant amount of progress in the last three decades. Its application is now moving towards wearable devices (like Google Glass and mobile devices) leading to the problem of FI on first-person-views (FPV) or ego-centric videos for scenarios like business networking, memory assistance, etc. In the existing literature, performance analysis of various image descriptors on FPV data is little known. In this paper, we evaluate six popular image descriptors: local binary patterns (LBP), scale invariant feature transform (SIFT), local phase quantization (LPQ), local intensity order pattern (LIOP), histogram of oriented gradients (HOG) and binarized statistical image features (BSIF) and ten different distance measures: Euclidean, Cosine, Chi square, Spearman, Cityblock, Minkowski, Correlation, Hamming, Jaccard and Chebychev with first nearest neighbor (1-NN) and support vector machines (SVM) as classifiers for FI task on both benchmark databases: FERET, AR, GT and FPV database collected using wearable devices like Google Glass (GG). Comparative analysis on these databases using various descriptors shows the superiority of BSIF with Cosine, Chi square and Cityblock distance measures using 1-NN as classifier over other descriptors and distance measures and even some of the current state-of-art benchmark database results.

In this paper, we investigate ways of conducting a detailed fashion search using query images and attributes. A credible fashion search platform should be able to (1) find images that share the same attributes as the query image, (2) allow users to manipulate certain attributes, e.g. replace collar attribute from round to v-neck, and (3) handle region-specific attribute manipulations, e.g. replacing the color attribute of the sleeve region without changing the color attribute of other regions. A key challenge to be addressed is that fashion products have multiple attributes and it is important for each of these attributes to have representative features. To address these challenges, we propose the FashionSearchNet which uses a weakly supervised localization method to extract regions of attributes. By doing so, unrelated features can be ignored thus improving the similarity learning. Also, FashionSearchNet incorporates a new procedure that enables region awareness to be able to handle region-specific requests. FashionSearchNet outperforms the most recent fashion search techniques and is shown to be able to carry out different search scenarios using the dynamic queries.

In this paper, we propose an attribute-based query & retrieval system designed for fashion products. Our system addresses the problem of carrying out fashion searches by the query image and attribute manipulation, e.g. replacing long sleeve attribute of a dress to sleeveless. We present the attributes in two groups: (1) general attributes (category, gender etc.) and (2) special attributes (sleeve length, collar etc.). The special attributes are more suitable for the attribute manipulation and thus conducting searches. In order to solve the mentioned fashion search problem, it is crucial for the deep neural networks to understand attribute similarities. To facilitate more specific similarity learning, clothing items are represented by their structural subcomponents or "parts". The parts are estimated using an unsupervised segmentation method and used inside the proposed Convolutional Neural Network (CNN) as an attention mechanism. Meaning, different parts are connected to the special attributes, e.g. sleeve part is connected with sleeve length attribute. With this mechanism, part-based triplet ranking constraint is applied to learn similarity of each special attribute independently from one another in a single network. In the end, the well-defined features are used to conduct the fashion search. Additionally, an adaptive relevance feedback module is used to personalize the fashion search process with the feature descriptions. For our experiments, a new dataset is constructed containing 101,021 images which consist of pure clothing items. Besides achieving decent retrieval results in our dataset, the experiments show that proposed technique outperforms different baselines and is able to adapt towards user's requests.

Despite remarkable progress of face analysis techniques, detecting landmarks on large-pose faces is still difficult due to self-occlusion, subtle landmark difference and incomplete information. To address these challenging issues, we introduce a novel recurrent 3D-2D dual learning model that alternatively performs 2D-based 3D face model refinement and 3D-to-2D projection based 2D landmark refinement to reliably reason about self-occluded landmarks, precisely capture the subtle landmark displacement and accurately detect landmarks even in presence of extremely large poses. The proposed model presents the first loop-closed learning framework that effectively exploits the informative feedback from the 3D-2D learning and its dual 2D-3D refinement tasks in a recurrent manner. Benefiting from these two mutual-boosting steps, our proposed model demonstrates appealing robustness to large poses (up to profile pose) and outstanding ability to capture fine-scale landmark displacement compared with existing 3D models. It achieves new state-of-the-art on the challenging AFLW benchmark. Moreover, our proposed model introduces a new architectural design that economically utilizes intermediate features and achieves 4× faster speed than its deep learning based counterparts.

We propose a novel 3D-assisted coarse-to-fine extreme-pose facial landmark detection system in this work. For a given face image, our system first refines the face bounding box with landmark locations inferred from a 3D face model generated by a Recurrent 3D Regressor at coarse level. Another R3R is then employed to fit a 3D face model onto the 2D face image cropped with the refined bounding box at fine-scale. 2D landmark locations inferred from the fitted 3D face are further adjusted with the popular 2D regression method, i.e. LBF. The 3D-assisted coarse-to-fine strategy and the 2D adjustment process explicitly ensure both the robustness to extreme face poses and bounding box disturbance and the accuracy towards pixel-level landmark displacement. Extensive experiments on the Menpo Challenge test sets demonstrate the superior performance of our system.

In this paper, we propose a Compressive Sensing based approach to the problem of real-time reconstruction of MR image sequences. Our proposed method is able to extract useful priori information and incorporate it into a modified iterative thresholding algorithm for fast casual reconstruction of MR images from highly undersampled k-space data. Through extensive experimental results we show that our proposed method achieves superior reconstruction quality, while having a lower computational complexity and memory requirements compared to the other state-of-the-art methods.

In this paper, we study the problem of recursively reconstructing time sequences of sparse signals, where sparsity changes smoothly with time. The idea is to use the signal/image of the previous time instance to extract an estimated probability model for the signal/image of interest, and then use this model to guide the reconstruction process. We examine and illustrate the performance of our approach, ''Weighted-CS'', with both synthetic and real medical signals/images. It is shown that we can achieve significant performance improvement, using fewer number of samples, compared to other state-of-art Compressive Sensing methods.

In this technical demonstration, we propose a face swapping framework, which is able to interactively change the appearance of a face in the wild to a different person/creature's face in real time on a mobile device. To realize this objective, we develop a deep learning-based face detector which is able to accurately detect faces in the wild. Our face feature points tracking system based on progressive initialization ensures accurate and robust localization of facial landmarks under extreme poses and expressions in real time. Relying on the advances of our face detector and face feature points tracker, we construct the Face Swapper which can smoothly replace the face appearance of a user in real time.

We propose a novel computationally efficient hierarchical dictionary learning (HDL) approach for data-driven unmixing and functional connectivity analysis of functional magnetic resonance imaging (fMRI) data. It is shown that by simultaneously exploiting the sparsity of the spatial brain maps and the incoherence among their evolution in time or task functions, one can achieve better performance while overcoming the drawbacks of existing approaches. The task functions constituting the dictionary, are learned using a hierarchical subset selection approach. Here, to enforce incoherence among atoms, any new atom is selected from suitable training candidates if it does not lie in the column span of past selected atoms. Also, since the sparsity of spatial maps is generally unknown and affected due to acquisition artifacts, HDL doesn't make use of an implicit sparse coding stage while dictionary update. This makes HDL a very fast and efficient data-driven approach for fMRI analysis. Experimental results on synthetic and real fMRI datasets provide compelling evidences that HDL performs better than existing state-of-the-art methods.

In this paper, we introduce a multi-dimensional approach to the problem of reconstruction of MR image sequences that are highly undersampled in k-space. By formulating the reconstruction as a high-order low-rank plus sparse tensor decomposition problem, we propose an efficient numerical algorithm based on the alternating direction method of multipliers (ADMM) to solve the optimization. Through extensive experimental results we show that our proposed method achieves superior reconstruction quality, compared to the state-of-the-art reconstruction methods.

Automatic classification of human epithelial type-2 (HEp-2) cells can improve the diagnostic process of autoimmune diseases (ADs) in terms of lower cost, faster response, and better repeatability. However, most of the proposed methods for classification of HEp-2 cells suffer from several constraints including tedious parameter tuning, massive memory requirement, and high computational costs. We propose an adaptive distributed dictionary learning (ADDL) method where the dictionary learning problem is reformulated as a distributed learning task. With the help of this approach, we develop an automatic and robust method that effectively handles the complexity of the problem in terms of memory and computational cost and also obtains superior classification accuracy.

We investigate the feature design and classification architectures in temporal action localization. This application focuses on detecting and labeling actions in untrimmed videos, which brings more challenge than classifying presegmented videos. The major difficulty for action localization is the uncertainty of action occurrence and utilization of information from different scales. Two innovations are proposed to address this issue. First, we propose a Pyramid of Score Distribution Feature (PSDF) to capture the motion information at multiple resolutions centered at each detection window. This novel feature mitigates the influence of unknown action position and duration, and shows significant performance gain over previous detection approaches. Second, inter-frame consistency is further explored by incorporating PSDF into the state-of-the-art Recurrent Neural Networks, which gives additional performance gain in detecting actions in temporally untrimmed videos. We tested our action localization framework on the THUMOS'15 and MPII Cooking Activities Dataset, both of which show a large performance improvement over previous attempts.

We propose a new linear model, based on resampling 3D face meshes to convert them to 3D matrices, to recognize identity and expression simultaneously. This contrasts with bilinear models currently used with 3D meshes. The matrices are amenable to algebraic operations for facial data analysis and synthesis. Facial emotion is represented as a linear combination of its identity and expression using principal components extracted from training data as neutral-to-emotion deformations. The linear model is applicable to other mesh data with pose variations after correction using recently available techniques. The proposed approach avoids the problem of correspondence between pairs of person's neutral and emotion meshes for estimating facial deformations used as features. Identity and expression recognition accuracies, obtained by representing resampled matrices as linear combinations of composite depth-color (gray) PCs, are better than the results in the literature on both simultaneous identity-expression using bilinear models and expression-only recognition using deformable models, facial action codes, distances between pairs of annotated facial points as features and others. The proposed framework can also be used to generate synthetic matrices displaying a wide array of natural and mixed emotions for any chosen identity. A byproduct is the result that second-order deformations as features do not seem to perform as effectively as first-order deformations for identity and expression recognition.

In this paper, we present a novel framework, called learning by propagability, for two essential data mining tasks, i.e., classification and regression. The whole learning process is driven by the philosophy that the data labels and the optimal feature representation jointly constitute a harmonic system, where the data labels are invariant with respect to the propagation on the similarity graph constructed based on the optimal feature representation. Based on this philosophy, a unified framework of learning by propagability is proposed for the purposes of both classification and regression. Specifically, this framework has three characteristics: 1) the formulation unifies the label propagation and optimal feature representation pursuing, and thus the label propagation process is enhanced by benefiting from the refined similarity graph constructed with the derived optimal feature representation instead of the original representation; 2) it unifies the formulations for supervised and semisupervised learning in both classification and regression tasks; and 3) it can directly deal with the multiclass classification problems. Extensive experiments for the classification task on UCI toy data sets, handwritten digit recognition, face recognition, and microarray recognition as well as for the regression task of human age estimation on the FG-NET aging database, all validate the effectiveness of our proposed learning framework, compared with the state-of-the-art counterparts.

Sparsity is a fundamental concept in compressive sampling of signals/images, which is commonly measured using the l0 norm, even though, in practice, the l1 or the lp ( 0

We propose a novel approach to human facial expression recognition using only the shape information at a finite set of fiducial points, extracted from the 3D neutral and expressive faces. In the course of applying the technique to the facial database, BU-3DFE, which contains facial shape and 2D color (''texture'') information, we extract from the images of neutral and expressive faces, salient contours in the facial interest-regions around the eyebrows, eyes, nose and mouth by invoking an active contour algorithm. The contours are then uniformly sampled and mapped onto the 3D-mesh dataset in order to generate a shape (and color) description of the interest-regions. By a matrix-algebraic operation on the shape of the neutral and expressive faces, a shape feature-matrix is computed for each expression and for each person, which is then subjected to the proposed modified PCA approach to recognize expressions. Classification results are presented to demonstrate the effectiveness of the proposed approach. It is also found that accuracy estimates compare favorably with those in the literature on facial expression recognition from 3D-mesh datasets.

A novel segmentation-based image approximation and coding technique is proposed. A hybrid quad-binary (QB) tree structure is utilized to efficiently model and code geometrical information within images. Compared to other tree-based representation such as wedgelets, the proposed QB-tree based method is more efficient for a wide range of contour features such as junctions, corners and ridges, especially at low bit rates.

This paper presents a new method for segmenting medical images by modeling interaction between neighboring structures. Compared to previously reported methods, the proposed approach enables simultaneous segmentation of multiple neighboring structures for improved robustness. During the segmentation process, the object contour evolution and shape prior estimates are influenced by the interactions between neighboring shapes consisting of attraction, repulsion, and competition. Instead of estimating the a priori shape of each structure independently, an interactive maximum a posteriori shape estimation method is used for estimating the shape priors by considering shape prior distribution, neighboring shapes, and image features. Energy functionals are then formulated to model the interaction and segmentation. With the proposed method, neighboring structures with similar intensities and/or textures, and blurred boundaries can be extracted simultaneously. Experimental results obtained on both synthetic data and medical images demonstrate that the introduced interaction between neighboring structures improves segmentation performance compared with other existing approaches.

This paper introduces the 3D color set partitioning in hierarchical trees (3D-CSPIHT) low bit rate embedded video coding scheme. The codec exploits the correlation between temporal and spatial wavelet coefficients and the interdependency between luminance and chrominance components to code color video sequences without the need for explicit bit allocation. Besides offering rate scalability, the new codec also produces multi-resolution scalable code streams. The hierarchical variable size block matching motion estimation technique is also integrated to demonstrate the motion estimation option with 3D-CSPIHT. The coding results show that 3D-CSPIHT produces better performance and visual quality compared to 3D-SPIHT.

In this paper, we present a hybrid framework for articulated 3-D human motion tracking from multiple synchronized cameras with potential uses in surveillance systems. Although the recovery of 3-D motion provides richer information for event understanding, existing methods based on either deterministic search or stochastic sampling lack robustness or efficiency. We therefore propose a hybrid sample-and-refine framework that combines both stochastic sampling and deterministic optimization to achieve a good compromise between efficiency and robustness. Similar motion patterns are used to learn a compact low-dimensional representation of the motion statistics. Sampling in a low-dimensional space is implemented during tracking, which reduces the number of particles drastically. We also incorporate a local optimization method based on simulated physical force/moment into our framework, which further improves the optimality of the tracking. Experimental results on several real human motion sequences show the accuracy and robustness of our method, which also has a higher sampling efficiency than most particle filtering-based methods.

Precise segmentation of vasculature from three-dimensional (3D) magnetic resonance angiography (MRA) images is playing an important role in image-guided neurosurgery, pre-operation planning and clinical analysis. Active Contour based evolution algorithms are being widely applied to MRA data sets, however existing approaches exhibit some difficulties in extracting tiny parts of the vessels. Our objective is to develop an automated segmentation scheme to accurately extract vasculature of the brain, especially tiny vessels. Inspired by the intrinsic properties of MRA, we have proposed a scheme called the gradient compensated geodesic active contours (GCGAC), which compensates for low gradients near edges of thin vessels. The GCGAC, which is implemented based on level set, has been tested on both synthetic volumetric image and real 3D MRA images. Our experiments show that the introduced gradient compensation can facilitate more accurate segmentation of tiny blood vessels.

Occlusion is a difficult problem for visual tracking and we use multiple wide baseline cameras to deal with occlusion. We propose a data fusion approach for visual tracking using multiple cameras with overlapping fields of view. First, we present a spatial and temporal recursive Bayesian filter to fuse information from multiple cameras. An adaptive particle filter is formulated to realize the spatial and temporal recursive Bayesian filter. Our algorithm is able to recover the target's position even under complete occlusion in a camera.

Tool wear dramatically affects the texture of the machined surface. Analyzing the texture of machined surfaces has been shown to be promising for tool wear monitoring. However, most methods have their limitations when applied to real environments, where the geometric features of machined surface depend on the machining operation, and where image quality is affected by illumination and other factors. Problems of non-uniform illumination and image noise can be reduced by applying image segmentation and image enhancement techniques. This paper discusses our work on statistical and structural approaches for analyzing machined surfaces and investigates the correlation between tool wear and quantities characterizing machined surfaces. The column projection method can be used for machined surfaces with highly repetitive and regular textures while the connectivity oriented fast Hough transform based method is able to characterize surfaces produced by various machining processes and conditions. Our results clearly indicate that tool condition monitoring which is defined as the ability to distinguish between a sharp, a semi-dull, or a dull tool can be successfully accomplished by analysis of statistical and structural information extracted from the machined surface.

In signal approximation, classical wavelet synthesis are known to produce Gibbs-like phenomenon around discontinuities when wavelet coefficients in the cone of influence of the discontinuities are quantized. By analyzing a function in a piecewise manner, filtering across discontinuities can be avoided. Using this principle, the interval wavelet transform can generate sparser representations in the vicinity of discontinuities than classical wavelet transforms. This work introduces two new constructions of interval wavelets and shows how they can be used for image compression and upscaling.

This paper studies automated classification of Human Epithelial Type-2 (HEp-2) cell images which is essential in diagnosing the Autoimmune Diseases (AD). The prevalent approach for this problem makes use of the Bag of Words (BoW) model and sparse coding scheme on over complete dictionaries, where the dictionary dimension is usually much larger than feature dimension. In addition, this approach usually requires manual selection of the dictionary dimension which is often troublesome and dependent highly on specific applications and datasets. We proposed a non-parametric Bayesian model that is capable of determining the dictionary dimension automatically by exploiting the Indian Buffet Process (IBP). This proposed model has been evaluated on two public HEp-2 benchmarking datasets, i.e., ICPR2012 and ICIP2013 where the SIFT and SURF features of the cell image are extracted in a grid manner and used as the input. Experiments show that the proposed model obtained state-of-the-art cell classification accuracy. More importantly, the dictionary dimension learned by the proposed model is around 29 and 8 times lower than the over complete dictionaries, respectively, for two benchmarking datasets. This low-dimensional dictionary helps to reduce the computational cost significantly for the cell classification task.

This paper presents a new method for segmentation of medical images by extracting organ contours, using minimal path deformable models incorporated with statistical shape priors. In our approach, boundaries of structures are considered as minimal paths, i.e., paths associated with the minimal energy, on weighted graphs. Starting from the theory of minimal path deformable models, an intelligent "worm" algorithm is proposed for segmentation, which is used to evaluate the paths and finally find the minimal path. Prior shape knowledge is incorporated into the segmentation process to achieve more robust segmentation. The shape priors are implicitly represented and the estimated shapes of the structures can be conveniently obtained. The worm evolves under the joint influence of the image features, its internal energy, and the shape priors. The contour of the structure is then extracted as the worm trail. The proposed segmentation framework overcomes the short-comings of existing deformable models and has been successfully applied to segmenting various medical images.

The texture of a machined surface generated by a cutting tool, with geometrically well-defined cutting edges, carries essential information regarding the extent of tool wear. There is a strong relationship between the degree of wear of the cutting tool and the geometry imparted by the tool on to the workpiece surface. The monitoring of a tool’s condition in production environments can easily be accomplished by analyzing the surface texture and how it is altered by a cutting edge experiencing progressive wear and micro-fractures. This paper discusses our work which involves fractal analysis of the texture of surfaces that have been subjected to machining operations. Two characteristics of the texture, high directionality and self-affinity, are dealt with by extracting the fractal features from images of surfaces machined with tools with different levels of tool wear. The Hidden Markov Model is used to classify the various states of tool wear. In this paper, we show that fractal features are closely related to tool condition and HMM-based analysis provides reliable means of tool condition prediction.

Just-noticeable distortion (JND), which refers to the maximum distortion that the human visual system (HVS) cannot perceive, plays an important role in perceptual image and video processing. In comparison with JND estimation for images, estimation of the JND profile for video needs to take into account the temporal HVS properties in addition to the spatial properties. In this paper, we develop a spatio-temporal model estimating JND in the discrete cosine transform domain. The proposed model incorporates the spatio-temporal contrast sensitivity function, the influence of eye movements, luminance adaptation, and contrast masking to be more consistent with human perception. It is capable of yielding JNDs for both still images and video with significant motion. The experiments conducted in this study have demonstrated that the JND values estimated for video sequences with moving objects by the model are in line with the HVS perception. The accurate JND estimation of the video towards the actual visibility bounds can be translated into resource savings (e.g., for bandwidth/storage or computation) and performance improvement in video coding and other visual processing tasks (such as perceptual quality evaluation, visual signal restoration/enhancement, watermarking, authentication, and error protection)

Precise segmentation of three-dimensional (3D) magnetic resonance angiography (MRA) images can be a very useful computer aided diagnosis (CAD) tool for clinical routines. Level sets based evolution schemes, which have been shown to be effective and easy to implement for many segmentation applications, are being applied to MRA data sets. In this paper, we present a segmentation scheme for accurately extracting vasculature from MRA images. Our proposed algorithm models capillary action and derives a capillary active contour for segmentation of thin vessels. The algorithm is implemented using the level set method and has been applied successfully on real 3D MRA images. Compared with other state-of-the-art MRA segmentation algorithms, experiments show that our method facilitates more accurate segmentation of thin blood vessels.

This paper proposes a method for progressive lossy-to-lossless compression of four-dimensional (4-D) medical images (sequences of volumetric images over time) by using a combination of three-dimensional (3-D) integer wavelet transform (IWT) and 3-D motion compensation. A 3-D extension of the set-partitioning in hierarchical trees (SPIHT) algorithm is employed for coding the wavelet coefficients. To effectively exploit the redundancy between consecutive 3-D images, the concepts of key and residual frames from video coding is used. A fast 3-D cube matching algorithm is employed to do motion estimation. The key and the residual volumes are then coded using 3-D IWT and the modified 3-D SPIHT. The experimental results presented in this paper show that our proposed compression scheme achieves better lossy and lossless compression performance on 4-D medical images when compared with JPEG-2000 and volumetric compression based on 3-D SPIHT.

The cathode ray tube (CRT) monitor consists of alternating patterns of red, green and blue color phosphor pixels at fixed spacing. Convergence relates to a monitor's ability to match red, green and blue colors to form white. The misconvergence or misalignment of the colors results in incorrect rendition of colors, particularly at the edges. Misconvergence is an important measure of the performance of a CRT monitor. This paper presents a technique for measuring misconvergence of a color monitor by processing monochrome images of specific patterns displayed on the monitor. The technique overcomes problems due to color cross talk, video/image jitter, and lens distortion. Also, the measurement time is shorter by an order of a magnitude over traditional methods, opening up possibilities for real time feedback for accurate alignment.

This paper describes a variation of the set partitioning in hierarchical trees (SPIHT) scheme for color image coding. By using partially linked spatial orientation tree structures across different spectral planes, the new color-SPIHT scheme is able to embed both chrominance and luminance data in the coded bit stream. The performance is comparable to that of a SPIHT-based coding scheme but with significantly lower computational complexity.

With the prevalence of brain-related diseases like Alzheimer in an increasing ageing population, Connectomics, the study of connections between neurons of the human brain, has emerged as a novel and challenging research topic. Accurate and fully automatic algorithms are needed to deal with the increasing amount of data from the brain images. This paper presents an automatic 3D neuron reconstruction technique where neurons within each slice image are first segmented and then linked across multiple slices within the publicly available Electron Microscopy dataset (SNEMI3D). First, random Forest classifier is adapted on top of superpixels for the neuron segmentation within each slice image. The maximum overlap between two consecutive images is then calculated for neuron linking, where the adjacency matrix of two different labeling of the segments is used to distinguish neuron merging and splitting. Experiments over the SNEMI3D dataset show that the proposed technique is efficient and accurate.

Face identification (FI) has made significant amount of progress in the last three decades. Its application is now moving towards wearable devices (like Google Glass and mobile devices) leading to the problem of FI on first-person-views (FPV) or ego-centric videos for scenarios like business networking, memory assistance, etc. In the existing literature, performance analysis of various image descriptors on FPV data are little known. In this paper, we evaluate four popular image descriptors: local binary patterns (LBP), scale invariant feature transform (SIFT), local phase quantization (LPQ) and binarized statistical image features (BSIF) and ten different distance measures: Euclidean, Cosine, Chi square, Spearman, Cityblock, Minkowski, Correlation, Hamming, Jaccard and Chebychev with first nearest neighbor (1-NN) and support vector machines (SVM) as classifiers for FI task on both benchmark databases: FERET, AR, GT and FPV database collected using wearable devices like Google Glass (GG). Comparative analysis on these databases using various descriptors shows the superiority of BSIF with Cosine, Chi square and Cityblock distance measures using 1-NN as classifier over other descriptors and distance measures and even some of the current state-of-art benchmark database results.

Use of automatic classification for Indirect Immunofluorescence (IIF) images of HEp-2 cells is increasingly gaining interest in Antinuclear Autoantibodies (ANAs) detection. In order to improve the classification accuracy, we propose a multi-modal joint dictionary learning method, to obtain a discriminative and reconstructive dictionary while training a classifier simultaneously. Here, the term ‘multi-modal’ refers to features extracted using different algorithms from the same data set. To utilize information fusion between feature modalities the algorithm is designed so that sparse codes of all modalities of each sample share the same sparsity pattern. The contribution of this paper is two-fold. First, we propose a new framework for multi-modal fusion at the feature level. Second, we impose an additional constraint on consistency of sparse coefficients among different modalities of the same class. Extensive experiments are conducted on the ICPR2012 and ICIP2013 HEp-2 data sets. All results confirm the higher level of accuracy of the proposed method compared with state-of-the-art.

In this paper, we propose a new framework to perform motion compression for time-dependent 3D geometric data. Temporal coherence in dynamic geometric models can be used to achieve significant compression, thereby leading to efficient storage and transmission of large volumes of 3D data. The displacement of the vertices in the geometric models is computed using the iterative closest point (ICP) algorithm. This forms the core of our motion prediction technique and is used to estimate the transformation between two successive 3D data sets. The motion between frames is coded in terms of a few affine parameters with some added residues. Our motion segmentation approach separates the vertices into two groups. Within the first group, motion can be encoded with a few affine parameters without the need of residues. In the second group, the vertices need further encoding of residual errors. Also in this group, for those vertices associated with large residual errors under affine mapping, we encode their motion effectively using Newtonian motion estimates. This automatic segmentation enables our algorithm to he very effective in compressing time-dependent geometric data. Dynamic range data captured from the real world, as well as complex animations created using commercial tools, can be compressed efficiently using this scheme.

It has been estimated that autoimmune diseases are among the top ten leading causes of death among women in all age groups up to 65 years. However, the detection of it by indirect immunofluorescence (IIF) image analysis depends heavily on the experience of the physicians. An accurate and automatic Computer Aided Diagnosis (CAD) system will help greatly for the classification of the Human Epithelial type 2 (HEp-2) cell images with little human intervention. In this paper we present an automatic HEp-2 cell image classification technique that exploits different spatial scaled image representation and sparse coding of SIFT features. Additionally, spatial max pooling of sparse coding at different scales is used to boost the classification performance. The proposed method is tested on the ICPR 2012 contest dataset and experiments show that it clearly outperforms state-of-the-art techniques in cell and image level as well as two intensity level images.

The potential of the new weighted-compressive sensing approach for efficient reconstruction of electrocardiograph (ECG) signals is investigated. This is motivated by the observation that ECG signals are hugely sparse in the frequency domain and the sparsity changes slowly over time. The underlying idea of this approach is to extract an estimated probability model for the signal of interest, and then use this model to guide the reconstruction process. The authors show that the weighted-compressive sensing approach is able to achieve reconstruction performance comparable with the current state-of-the-art discrete wavelet transform-based method, but with substantially less computational cost to enable it to be considered for use in the next generation of miniaturised wearable ECG monitoring devices.

This paper describes the implementation of the recently introducedcolor set partitioning in hierarchical tree (CSPIHT)-based scheme for video coding. The intra- and interframe coding performance of a CSPIHT-based video coder (CVC) is compared against that of the H.263 at bit rates lower than 64 kbit/s. The CVC performs comparably or better than the H.263 at lower bit rates, whereas the H.263 performs better than the CVC at higher bit rates. We identify areas that hamper the performance of the CVC and propose an improved scheme that yields better performance in image and video coding in low bit-rate environments.

Tool wear dramatically affects the texture of the machined surface and the sound generated by the cutting process. This paper discusses our work on texture analysis of machined surfaces and signal processing of sound generated by machining process and investigates the correlation between tool wear and quantities characterizing machined surfaces and sound pattern. Our results clearly indicate that tool condition monitoring which is defined as the ability to distinguish between a sharp, a semi-dull, or a dull tool can be successfully accomplished by combining sensory data from a CCD camera (image analysis) and a microphone (sound analysis).

With the proliferation of different types of IC packages, there is a need for machine-vision-based inspection systems to be able to efficiently identify the orientation of IC packages to ensure that they are shipped to end-users in the proper orientation. The orientation of IC packages can be determined by locating special features called notches or dimples, which are molded on the IC packages. This paper presents an algorithm that automatically checks the orientation of different typ es of IC packages. Our algorithm incorporates several procedures to achieve a fast and robust operation. Firstly, the boundary of the IC package is detected and aligned; then regions where the notch or dimple may exist are isolated. Adaptive double thresholding is applied in the selected regions to extract the possible notches or dimples. The extracted objects are then analyzed to determine whether they represent the notch or the dimple. In these procedures, intensity-based information as well as the gradie nt-based information is analyzed to generate a more comprehensive description of the features of interest.