Search

The quality of patient care associated with diagnostic radiology is proportionate to a physician's workload. Segmentation is a fundamental limiting precursor to diagnostic and therapeutic procedures. Advances in machine learning aims to increase diagnostic efficiency to replace single applications with generalized algorithms. We approached segmentation as a multitask shape regression problem, simultaneously predicting coordinates on an object's contour while jointly capturing global shape information. Shape regression models inherent point correlations to recover ambiguous boundaries not supported by clear edges and region homogeneity. Its capabilities was investigated using multi-output support vector regression (MSVR) on head and neck (HaN) CT images. Subsequently, we incorporated multiplane and multimodality spinal images and presented the first deep learning multiapplication framework for shape regression, the holistic multitask regression network (HMR-Net). MSVR and HMR-Net's performance were comparable or superior to state-of-the-art algorithms. Multiapplication frameworks bridges any technical knowledge gaps and increases workflow efficiency.

Automated capture of animal pose is transforming how we study neuroscience and social behavior. Movements carry important social cues, but current methods are not able to robustly estimate pose and shape of animals, particularly for social animals such as birds, which are often occluded by each other and objects in the environment. To address this problem, we first introduce a model and multi-view optimization approach, which we use to capture the unique shape and pose space displayed by live birds. We then introduce a pipeline and experiments for keypoint, mask, pose, and shape regression that recovers accurate avian postures from single views. Finally, we provide extensive multi-view keypoint and mask annotations collected from a group of 15 social birds housed together in an outdoor aviary. The project website with videos, results, code, mesh model, and the Penn Aviary Dataset can be found at https://marcbadger.github.io/avian-mesh., Comment: In ECCV 2020

Objectives The purpose of this study was to put forth a precise landmark-based technique for reconstructing the three-dimensional shape of human entheseal surfaces, to investigate whether the shape of human entheses is related to their size. The effects of age-at-death and bone length on entheseal shapes were also assessed. Materials and methods The sample comprised high-definition three-dimensional models of three right hand entheseal surfaces, which correspond to 45 male adult individuals of known age. For each enthesis, a particular landmark configuration was introduced, whose precision was tested both within and between observers. The effect of three-dimensional size, age-at-death, and bone length on shape was investigated through shape regression. Results The method presented high intra-observer and inter-observer repeatability. All entheses showed significant allometry, with the area of opponens pollicis demonstrating the most substantial relationship. This was particularly due to variation related to its proximal elongated ridge. The effect of age-at-death and bone length on entheses was limited. Discussion The introduced methodology can set a reliable basis for further research on the factors affecting entheseal shape. Using both size and shape, variables can provide further information on entheseal variation and its biomechanical implications. The low entheseal variation by age verifies that specimens under 50 years of age are not substantially affected by age-related changes. The lack of correlation between entheseal shape and bone length or age implies that other factors may regulate entheseal surfaces. Future research should focus on multivariate shape patterns among entheses and their association with occupation.

Crossbreeding between individuals of different breeds and introgression, the transfer of genes between breeds and/or populations mediated primarily by backcrossing, have been characteristic tools used in the refinement or optimisation of practical horse breeding. In this study we analysed the genetic contribution of the Arabian horse to the gene pool of the Lipizzan horse and its association with the overall type via shape regression analysis in 158 Lipizzan horses from the Austrian federal stud farm of Piber and the Spanish Riding School. Although crossbreeding with Arabian horses took place between 1776 and 1945, we found a significant association between Lipizzan body shape (p

State-of-the-art scene text detection techniques predict quadrilateral boxes that are prone to localization errors while dealing with straight or curved text lines of different orientations and lengths in scenes. This paper presents a novel multi-scale shape regression network (MSR) that is capable of locating text lines of different lengths, shapes and curvatures in scenes. The proposed MSR detects scene texts by predicting dense text boundary points that inherently capture the location and shape of text lines accurately and are also more tolerant to the variation of text line length as compared with the state of the arts using proposals or segmentation. Additionally, the multi-scale network extracts and fuses features at different scales which demonstrates superb tolerance to the text scale variation. Extensive experiments over several public datasets show that the proposed MSR obtains superior detection performance for both curved and straight text lines of different lengths and orientations., Comment: Accepted by IJCAI19

This work presents a Bayesian predictive approach to statistical shape analysis. A modeling strategy that starts with a Gaussian distribution on the configuration space, and then removes the effects of location, rotation and scale, is studied. This boils down to an application of the projected normal distribution to model the configurations in the shape space, which together with certain identifiability constraints, facilitates parameter interpretation. Having better control over the parameters allows us to generalize the model to a regression setting where the effect of predictors on shapes can be considered. The methodology is illustrated and tested using both simulated scenarios and a real data set concerning eight anatomical landmarks on a sagittal plane of the corpus callosum in patients with autism and in a group of controls.

Spatiotemporal shape models capture the dynamics of shape change over time and are an essential tool for monitoring and measuring anatomical growth or degeneration. In this paper we evaluate non-parametric shape regression on the challenging problem of modeling early childhood sub-cortical development starting from birth. Due to the flexibility of the model, it can be challenging to choose parameters which lead to a good model fit yet does not over fit. We systematically test a variety of parameter settings to evaluate model fit as well as the sensitivity of the method to specific parameters, and we explore the impact of missing data on model estimation.

This thesis concerns the geometric problem of finding a convex set that best fits a given dataset. Our question serves as an abstraction for data-analytical tasks arising in a range of scientific and engineering applications. We focus on two specific instances: 1. A key challenge that arises in solving inverse problems is ill-posedness due to a lack of measurements. A prominent family of methods for addressing such issues is based on augmenting optimization-based approaches with a convex penalty function so as to induce a desired structure in the solution. These functions are typically chosen using prior knowledge about the data. In Chapter 2, we study the problem of learning convex penalty functions directly from data for settings in which we lack the domain expertise to choose a penalty function. Our solution relies on suitably transforming the problem of learning a penalty function into a fitting task. 2. In Chapter 3, we study the problem of fitting tractably-described convex sets given the optimal value of linear functionals evaluated in different directions. Our computational procedures for fitting convex sets are based on a broader framework in which we search among families of sets that are parameterized as linear projections of a fixed structured convex set. The utility of such a framework is that our procedures reduce to the computation of simple primitives at each iteration, and these primitives can be further performed in parallel. In addition, by choosing structured sets that are non-polyhedral, our framework provides a principled way to search over expressive collections of non-polyhedral descriptions; in particular, convex sets that can be described via semidefinite programming provide a rich source of non-polyhedral sets, and such sets feature prominently in this thesis. We provide performance guarantees for our procedures. Our analyses rely on understanding geometrical aspects of determinantal varieties, building on ideas from empirical processes as well as random matrix theory. We demonstrate the utility of our framework with numerical experiments on synthetic data as well as applications in image denoising and computational geometry. As secondary contributions, we consider the following: 1. In Chapter 4, we consider the problem of optimally approximating a convex set as a spectrahedron of a given size. Spectrahedra are sets that can be expressed as feasible regions of a semidefinite program. 2. In Chapter 5, we consider change-point estimation in a sequence of high-dimensional signals given noisy observations. Our method integrates classical approaches with a convex optimization-based step that is useful for exploiting structure in high-dimensional data.

For smart living applications, personal identification as well as behavior and emotion detection becomes more and more important in our daily life. For identity classification and facial expression detection, facial features extracted from face images are the most popular and low-cost information. The face shape in terms of landmarks estimated by a face alignment method can be used for many applications including virtual face animation and real face classification. In this paper, we propose a robust face alignment method based on the multi-feature shape regression (MSR), which is evolved from the explicit shape regression (ESR) proposed in Cao et al. (Int, Vis, 2014, 107:177–190, Comput). The proposed MSR face alignment method successfully utilizes color, gradient, and regional information to increase accuracy of landmark estimation. For face recognition algorithms, we further suggest a face warping algorithm, which can cooperate with any face alignment algorithm to adjust facial pose variations to improve their recognition performances. For performance evaluations, the proposed and the existing face alignment methods are compared on the face alignment database. Based on alignment-based face recognition concept, the face alignment methods with the proposed face warping method are tested on the face database. Simulation results verify that the proposed MSR face alignment method achieves better performances than the other existing face alignment methods.

Longitudinal shape analysis has shown great potential to model anatomical processes from baseline to follow-up observations. Shape regression estimates a continuous trajectory of time-discrete anatomical shapes to quantify temporal changes. The need for shape alignment and point-to-point correspondences represent limitations of current shape analysis methodologies, and present significant challenges in shape evaluation. We propose a method that estimates a continuous trajectory of continuous medial representations (CM-Rep) from a set of time-discrete observed shapes. To avoid the traditional step of aligning individual objects, shape changes are modeled via diffeomorphic ambient space deformations. Using a medial shape representation, we separately capture object pose changes and intrinsic geometry changes. Tests and validation with synthetic and real anatomical shapes demonstrate that the new method captures extrinsic shape changes as well as intrinsic shape changes encoded with CM-Reps, a highly relevant property for studying growth and disease processes.

Road shape estimation is important for the safe driving of intelligent vehicles. The common road shape models such as line/parabola, spline and clothoid are lacking of flexibility in various urban traffic scenes. In this paper, a robust road shape model which consists of multiple overlapped submaps is proposed. Each individual submap is represented by a smooth curve generated through Gaussian process(GP). To estimate parameters of a GP submap, a framework involving pre-processing, pose correction, road shape regression and map updating/creating is proposed. Pose correction is achieved by fusion of vehicle motion model and simplified GP-based observation model. Road shape regression is used to extract a coarse road shape. Map updating/creating is used to adapt to the new coming data and generates refined road shape. A robust iterative Gaussian process regression(iGPR) is utilized in both road shape regression and map updating/creating. Extensive experimental results show the efficiency of the proposed method.

International audience; Many problems in medicine are inherently dynamic processes which include the aspect of change over time, such as childhood development, aging, and disease progression. From medical images, numerous geometric structures can be extracted with various representations, such as landmarks, point clouds, curves, and surfaces. Different sources of geometry may characterize different aspects of the anatomy, such as fiber tracts from DTI and subcortical shapes from structural MRI, and therefore require a modeling scheme which can include various shape representations in any combination. In this paper, we present a geodesic regression model in the large deformation (LDDMM) framework applicable to multi-object complexes in a variety of shape representations. Our model decouples the deformation parameters from the specific shape representations, allowing the complexity of the model to reflect the nature of the shape changes, rather than the sampling of the data. As a consequence, the sparse representation of diffeomorphic flow allows for the straightforward embedding of a variety of geometry in different combinations, which all contribute towards the estimation of a single deformation of the ambient space. Additionally, the sparse representation along with the geodesic constraint results in a compact statistical model of shape change by a small number of parameters defined by the user. Experimental validation on multi-object complexes demonstrate robust model estimation across a variety of parameter settings. We further demonstrate the utility of our method to support the analysis of derived shape features, such as volume, and explore shape model extrapolation. Our method is freely available in the software package deformetrica which can be downloaded at www.deformetrica.org.

Facial landmark detection is a challenging task with broad applications. Many approaches have been proposed with varying degrees of success. Regression based methods update the facial point positions iteratively. The mean shape or shapes sampled from training set is often used as the initialization, which sometimes may lead to a local minimum in update due to the offset of initial positions and target positions. On the other hand, convolution network based method shows superior accuracy, at the cost of a complex and unwieldy architecture of deep model. To address the above limitations, a new approach for facial landmark detection is proposed in this paper. The key idea is to combine deep convolution networks with shape regression approach. Deep convolution networks in the first level provide a highly robust initial shape, while the following regression finely tunes the initial prediction to achieve high accuracy. Extensive experiments show that the regression based methods are very sensitive to initializations and the proposed approach (i) achieves high accuracy on public datasets, especially outperforms existing methods in challenging conditions like large pose expression variation, and (ii) reduces model complexity drastically compared to previous methods.

We present a very efficient, highly accurate, “Explicit Shape Regression” approach for face alignment. Unlike previous regression-based approaches, we directly learn a vectorial regression function to infer the whole facial shape (a set of facial landmarks) from the image and explicitly minimize the alignment errors over the training data. The inherent shape constraint is naturally encoded into the regressor in a cascaded learning framework and applied from coarse to fine during the test, without using a fixed parametric shape model as in most previous methods. To make the regression more effective and efficient, we design a two-level boosted regression, shape-indexed features and a correlation-based feature selection method. This combination enables us to learn accurate models from large training data in a short time (20 minutes for 2,000 training images), and run regression extremely fast in test (15 ms for a 87 landmarks shape). Experiments on challenging data show that our approach significantly outperforms the state-of-the-art in terms of both accuracy and efficiency.

We present a real-time performance-driven facial animation system based on 3D shape regression. In this system, the 3D positions of facial landmark points are inferred by a regressor from 2D video frames of an ordinary web camera. From these 3D points, the pose and expressions of the face are recovered by fitting a user-specific blendshape model to them. The main technical contribution of this work is the 3D regression algorithm that learns an accurate, user-specific face alignment model from an easily acquired set of training data, generated from images of the user performing a sequence of predefined facial poses and expressions. Experiments show that our system can accurately recover 3D face shapes even for fast motions, non-frontal faces, and exaggerated expressions. In addition, some capacity to handle partial occlusions and changing lighting conditions is demonstrated.

Accurate segmentation of optic cup and disc in retinal fundus images is required to derive the cup-to-disc ratio (CDR) parameter which is the main indicator for Glaucoma assessment. In this paper, we propose a coupled regression method for accurate segmentation of optic cup and disc in retinal colour fundus image. The proposed coupled regression framework consists of a parameter regressor which directly predicts CDR from a given image, as well as an ensemble shape regressor which iteratively estimates the OD-OC boundary by taking into account the CDR estimated by the parameter regressor. The parameter regressor and the shape regressor are then coupled together within a feedback loop so that estimation of one reinforces the other. Both parameter regressor and the ensemble shape regressor are modeled using Boosted Regression Trees. The proposed optic cup and disc segmentation method is applied on an image set of 50 patients and demonstrates high segmentation accuracy. A comparative study shows that our proposed method outperforms state of the art methods for cup segmentation.

Cascade shape regression has been proven to be an accurate, robust and fast framework for face alignment. Recently, a lot of methods based on this framework have emerged which focus on boosting learning method or extracting geometric invariant features. Despite the great success of these methods, none of them are initialization independent, which limits their prediction performance to some complex face shapes. In this paper, we propose a novel initialization scheme called high-quality initial shape estimation to generate high-quality initial face shapes. First, we extract Gabor features to represent facial appearance. Then we minimize the square error between the target shapes and the estimated initial shapes using a random regression forest and binary comparison features. Finally, we use a standard cascade shape regressor to regress the estimated initial shape for robust face alignment. Experimental results show that our method achieves state-of-the-art performance on the 300-W dataset, which is the most challenging dataset today.

The emergence of mobile systems brings new problematics in computer vision. Static camera-based methods for re-identification need to be adapted in this new context. To deal with dynamical background, this thesis proposes to employ the well known Point Distribution Model (PDM), usually applied for face alignment, on the human body. Three advantages come from this pre-processing before re-identification, segment the person from background, enhance robustness to the person pose and extract spatial key points to build a behavioural-based signature.We implement and evaluate a complete framework for re-identification, divided in three sequential modules. The first one corresponds to the pedestrian detection. We use an efficient method of the state of the art employing the Channel Features with the algorithm AdaBoost.The second one is the PDM alignment within the bounding box provided by the detection step. Two distinct approaches are presented in this thesis. The first method relies on a parametric formulation to describe the shape, similar to the ASM or AAM. To fit this shape model, we maximize the score of an appearance model defined by GentleBoost, which employs local histograms of oriented gradients. The second approach is based on the cascade regression shape scheme. The main idea is the approximation for each step into a classification of homogeneous deformations, grouped by unsupervised clustering.The third module is the re-identfication one. We show that employing a PDM as a structural support improves re-identification results. We experiment classic appearance-based signatures, color histograms and the shape descriptor Shape Context. The results are encouraging for application perspective of PDM for the gait recognition.; L'essor des systèmes mobiles pose de nouvelles problématiques dans le domaine de vision par ordinateur. Les techniques de ré-identification s'appuyant sur un réseau de caméras fixes doivent être repensées afin de s'adapter à un décor changeant. Pour répondre à ces besoins, cette thèse explore, dans le cadre du corps humain, l'utilisation d'un modèle structurel habituellement employé pour de la reconnaissance faciale. Il s'agit de l'alignement d'un modèle à distribution de points (Point Distribution Model ou PDM). L'objectif de ce pré-traitement avant la ré-identification est triple, segmenter la personne du décor, améliorer la robustesse vis-à-vis de sa pose et extraire des points clés spatiaux pour construire une signature basée sur son comportement.Nous concevons et évaluons un système complet de ré-identification, découpé en trois modules mis en séquence. Le premier de ces modules correspond à la détection de personnes. Nous proposons de nous baser sur une méthode de l'état de l'art utilisant les Channel Features avec l'algorithme AdaBoost.Le second module est l'alignement du PDM au sein de la boîte englobante fournie par la détection. Deux approches sont présentées dans cette thèse. La première s'appuie sur une formulation paramétrique du modèle de forme. L'alignement de ce modèle est guidé par la maximisation d'un score d'un modèle d'apparence GentleBoost utilisant des caractéristiques locales de type histogrammes de gradients orientés. La seconde approche exploite une technique de cascade de régressions de forme. L'idée principale est le regroupement de déformations homogènes en clusters et la classification de ces derniers dans le but d'aligner le PDM itérativement.Enfin, le troisième module est celui de la ré-identification. Nous montrons que l'utilisation d'un PDM en support permet d'améliorer les résultats de ré-identification. Nos expérimentations portent sur des signatures d'apparence classique, les histogrammes de couleurs, et sur un descripteur de forme, le Shape Context. L'évaluation de ce dernier fournit des résultats encourageants pour une perspective d'utilisation des PDM au sein d'une reconnaissance de démarches.

We present a facial animation system for ordinary single-cameral videos based on 2D shape regression. Unlike some prior facial animation techniques, our system doesn’t need complex equipment. The system consists of firstly a Cascade Multi-Channel Convolutional Neural Network (CMC-CNN) model to accurately detect facial landmarks from 2D video frames. Based on these detected 2D points, the facial motion parameters, including the head pose and facial expressions, are recovered. Then the system animates a bone-driven 3D avatar with the facial motion parameters. Experiments show that our system can accurately detect facial landmarks and the animation results are visually plausible and similar to the user’s facial motion.

Statistical analysis of time dependent imaging data is crucial for understanding normal anatomical development as well as disease progression. The most promising studies are of longitudinal design, where repeated observations are obtained from the same subjects. Analysis in this case is challenging due to the difficulty in modeling longitudinal changes, such as growth, and comparing changes across different populations. In any case, the study of anatomical change over time has the potential to further our understanding of many dynamic processes. What is needed are accurate computational models to capture, describe, and quantify anatomical change over time. Anatomical shape is encoded in a variety of representations, such as medical imaging data and derived geometric information extracted as points, curves, and/or surfaces. By considering various shape representations embedded into the same ambient space as a shape complex, either in 2D or 3D, we obtain a more comprehensive description of the anatomy than provided by an single isolated shape. In this dissertation, we develop spatiotemporal models of anatomical change designed to leverage multiple shape representations simultaneously. Rather than study directly the geometric changes to a shape itself, we instead consider how the ambient space deforms, which allows all embedded shapes to be included simultaneously in model estimation. Around this idea, we develop two complementary spatiotemporal models: a flexible nonparametric model designed to capture complex anatomical trajectories, and a generative model designed as a compact statistical representation of anatomical change. We present several ways spatiotemporal models can support the statistical analysis of scalar measurements, such as volume, extracted from shape. Finally, we cover the statistical analysis of higher dimensional shape features to take better advantage of the rich morphometric information provided by shape, as well as the trajectory of change captured by spatiotemporal models.

We present a highly accurate and very efficient approach for face alignment, called Extended Robust Cascaded Pose Regression (ERCPR), which is robust to large variations due to differences in expressions and pose. Unlike previous shape regression-based approaches, we propose to reference features weighted by three different face landmarks, which are much more robust to shape variations. Then, a correlation-based feature selection method and a two-level boosted regression are applied to establish accurate relation between features and shapes. Experiments on two challenging face datasets (LFPW, COFW) show that our proposed approach significantly outperforms the state-of-art in terms of both efficiency and accuracy.

Classical dressage and the schools above the ground as performed in the Spanish Riding School (SRS) in Vienna, require special psychological and physical properties from riding horses. To document the training and performing level of the Lipizzan riding stallions from the SRS in Vienna, we analyzed the horses' performance traits retrieved from chief riders' evaluations in relation to training levels and age classes and we studied the interplay of performing status with the horses' body shape. In total, the mean age of all 80 riding stallions was 11.9 years (min 4 years, max 26 years). Completely trained stallions (competition level S and higher) were on average 15.6 years old (min. 10 years and max. 26 years). From 10 recorded performance traits (five physical traits and five psychological traits), walk, trot, and collection ratings showed significant differences for levadeurs, caprioleurs, and courbetteurs; the psychological traits reactability, diligence, and sensibility showed significant differences between age class (3–4 years, 5–8 years, 9–16 years, >16 years) and number of flying gallop changes. Further we found that 80% of the chief riders' ratings of physical performance traits reached significant levels in the shape regressions, indicating an association of their ratings with body shape variation. The resulting mean body shapes from the significant regressions illustrated the requirements of the school above the ground and the classical dressage on the horses' conformation. We showed that the evaluation of subjective ratings on valuating scales applying shape regressions can help to optimize the quality of scoring data in equine performance traits.

In this experiment, which is based on a cohort of 44 Lipizzan mares from the Austrian state stud farm of Piber, we present new statistical techniques for the analysis of shape and equine conformation using image data. In addition, we examined which strategies and procedures of image processing techniques led to a successful interpretation of the traits implemented in horse breeding programs. A total of 246 two-dimensional anatomical and somatometric landmarks were digitized from standardized photographs, and the variation of shape has been analyzed by the use of generalized orthogonal least-squares Procrustes (generalized Procrustes analysis (GPA)) procedures. The resulting shape variables have been regressed on the results from linear type trait classifications. In addition, the rating scores of six conformation classifiers were tested for agreement, yielding an inter-rater correlation (inter-class correlation) ranging from 0.41 to 0.68, respectively, a κ coefficient ranging from 0.16 to 0.53. From the 12 linear type traits assessed on a valuating scale, only the type-related traits (type, breed-type and harmony) revealed significant (P

We present a novel approach to resolve the problem of face alignment with a two-layer shape regression framework. Traditional regression-based methods [4, 6, 7] regress all landmarks in a single shape without consideration of the difference between various landmarks in biologic property and texture, which would lead to a suboptimal prediction. Unlike previous regression-based approach, we do not regress the entire landmarks in a holistic manner without any discrimination. We categorize the geometric constraints into two types, inter-component constraints and intra-component constraints. Corresponding to these two shape constraints, we design a two-layer shape regression framework which can be integrated with regression-based methods. We define a term of “key points” of components to describe inter-component constraints and then determine the sub-shapes. We verify our two-layer shape regression framework on two widely used datasets (LFPW [10] and Helen [11]) for face alignment and experimental results prove its improvements in accuracy.

With quick development of Kinect, depth image has become an important channel for assisting the color/infrared image in diverse computer vision tasks. Kinect can provide depth image as well as color and infrared images, which are suitable for multi-model vision tasks. This paper presents a framework for intensity-depth face alignment based on cascade shape regression. Information from intensity and depth images is combined during feature selection in cascade shape regression. Experimental results show that this combination improves face alignment accuracy notably.

The localization of eye centers technique is used in several applications. In this paper, we combine the face shape model to eye center location method to obtaining robustness in eye center localization from a low-resolution image. We make use of shape regression approach and combine it to the eye-center localization using isophote curvature features which has shown its advantage from previous work. In the experiments, we use BioID datasets to test our approach for accurate eye center location and robustness to changes in illumination and pose. We demonstrate that our approach can achieve an improvement of the robustness of eye center localization while the accuracy at eye center localization e≤0.1 and e≤0.15 have achieve 0.918 and 0.977 respectively.

Facial feature detection offers a wide range of applications, e.g. in facial image processing, human computer interaction, consumer electronics, and the entertainment industry. These applications impose two antagonistic key requirements: high processing speed and high detection accuracy. We address both by expanding upon the recently proposed explicit shape regression [1] to (a) allow usage and mixture of different feature channels, and (b) include head pose information to improve detection performance in non-cooperative environments. Using the publicly available “wild” datasets LFW [10] and AFLW [11], we show that using these extensions outperforms the baseline (up to 10% gain in accuracy at 8% IOD) as well as other state-of-the-art methods.