Search

Deep neural networks have been tremendously successful at segmenting objects in images. However, it has been shown they still have limitations on challenging problems such as the segmentation of medical images. The main reason behind this lower success resides in the reduced size of the object in the image. In this paper we overcome this limitation through a cyclic collaborative framework, CyCoSeg. The proposed framework is based on a deep active shape model (D-ASM), which provides prior information about the shape of the object, and a semantic segmentation network (SSN). These two models collaborate to reach the desired segmentation by influencing each other: SSN helps D-ASM identify relevant keypoints in the image through an Expectation Maximization formulation, while D-ASM provides a segmentation proposal that guides the SSN. This cycle is repeated until both models converge. Extensive experimental evaluation shows CyCoSeg boosts the performance of the baseline models, including several popular SSNs, while avoiding major architectural modifications. The effectiveness of our method is demonstrated on the left ventricle segmentation on two benchmark datasets, where our approach achieves one of the most competitive results in segmentation accuracy. Furthermore, its generalization is demonstrated for lungs and kidneys segmentation in CT scans.

Object recognition and localization is still a very challenging problem, despite recent advances in deep learning (DL) approaches, especially for objects with varying shapes and appearances. Statistical models, such as an Active Shape Model (ASM), rely on a parametric model of the object, allowing an easy incorporation of prior knowledge about shape and appearance in a principled way. To take advantage of these benefits, this paper proposes a new ASM framework that addresses two tasks: $(i)$ comparing the performance of several image features used to extract observations from an input image; and $(ii)$ improving the performance of the model fitting by relying on a probabilistic framework that allows the use of multiple observations and is robust to the presence of outliers. The goal in $(i)$ is to maximize the quality of the observations by exploring a wide set of handcrafted features (HOG, SIFT, and texture templates) and more recent DL-based features. Regarding $(ii)$ , we use the Generalized Expectation-Maximization algorithm to deal with outliers and to extend the fitting process to multiple observations. The proposed framework is evaluated in the context of facial landmark fitting and the segmentation of the endocardium of the left ventricle in cardiac magnetic resonance volumes. We experimentally observe that the proposed approach is robust not only to outliers, but also to adverse initialization conditions and to large search regions (from where the observations are extracted from the image). Furthermore, the results of the proposed combination of the ASM with DL-based features are competitive with more recent DL approaches ( e.g. FCN [1] , U-Net [2] and CNN Cascade [3] ), showing that it is possible to combine the benefits of statistical models and DL into a new deep ASM probabilistic framework.

The way people walk is a strong correlate of their identity. Several studies have shown that both humans and machines can recognize individuals just by their gait, given that proper measurements of the observed motion patterns are available. For surveillance applications, gait is also attractive, because it does not require active collaboration from users and is hard to fake. However, the acquisition of good-quality measures of a person’s motion patterns in unconstrained environments, (e.g., in person re-identification applications) has proved very challenging in practice. Existing technology (video cameras) suffer from changes in viewpoint, daylight, clothing, accessories, and other variations in the person’s appearance. Novel three-dimensional sensors are bringing new promises to the field, but still many research issues are open. This article presents a survey of the work done in gait analysis for re-identification in the past decade, looking at the main approaches, datasets, and evaluation methodologies. We identify several relevant dimensions of the problem and provide a taxonomic analysis of the current state of the art. Finally, we discuss the levels of performance achievable with the current technology and give a perspective of the most challenging and promising directions of research for the future.

This paper addresses the problem of Human-Aware Navigation (HAN), using multi camera sensors to implement a vision-based person tracking system. The main contributions of this paper are as follows: ...

Overall survival (OS) time prediction is one of the most common estimates of the prognosis of gliomas and is used to design an appropriate treatment planning. State-of-the-art (SOTA) methods for OS time prediction follow a pre-hoc approach that require computing the segmentation map of the glioma tumor sub-regions (necrotic, edema tumor, enhancing tumor) for estimating OS time. However, the training of the segmentation methods require ground truth segmentation labels which are tedious and expensive to obtain. Given that most of the large-scale data sets available from hospitals are unlikely to contain such precise segmentation, those SOTA methods have limited applicability. In this paper, we introduce a new post-hoc method for OS time prediction that does not require segmentation map annotation for training. Our model uses medical image and patient demographics (represented by age) as inputs to estimate the OS time and to estimate a saliency map that localizes the tumor as a way to explain the OS time prediction in a post-hoc manner. It is worth emphasizing that although our model can localize tumors, it uses only the ground truth OS time as training signal, i.e., no segmentation labels are needed. We evaluate our post-hoc method on the Multimodal Brain Tumor Segmentation Challenge (BraTS) 2019 data set and show that it achieves competitive results compared to pre-hoc methods with the advantage of not requiring segmentation labels for training., Comment: 5 pages, 2 figure

Meta-training has been empirically demonstrated to be the most effective pre-training method for few-shot learning of medical image classifiers (i.e., classifiers modeled with small training sets). However, the effectiveness of meta-training relies on the availability of a reasonable number of hand-designed classification tasks, which are costly to obtain, and consequently rarely available. In this paper, we propose a new method to unsupervisedly design a large number of classification tasks to meta-train medical image classifiers. We evaluate our method on a breast dynamically contrast enhanced magnetic resonance imaging (DCE-MRI) data set that has been used to benchmark few-shot training methods of medical image classifiers. Our results show that the proposed unsupervised task design to meta-train medical image classifiers builds a pre-trained model that, after fine-tuning, produces better classification results than other unsupervised and supervised pre-training methods, and competitive results with respect to meta-training that relies on hand-designed classification tasks.

This paper describes the field research, design and comparative deployment of a multimodal medical imaging user interface for breast screening. The main contributions described here are threefold: 1) The design of an advanced visual interface for multimodal diagnosis of breast cancer (BreastScreening); 2) Insights from the field comparison of single vs multimodality screening of breast cancer diagnosis with 31 clinicians and 566 images, and 3) The visualization of the two main types of breast lesions in the following image modalities: (i) MammoGraphy (MG) in both Craniocaudal (CC) and Mediolateral oblique (MLO) views; (ii) UltraSound (US); and (iii) Magnetic Resonance Imaging (MRI). We summarize our work with recommendations from the radiologists for guiding the future design of medical imaging interfaces., Comment: AVI 2020 Short Papers, 5 pages, 2 figures, for associated files, see https://github.com/MIMBCD-UI/avi-2020-short-paper

The deployment of automated systems to diagnose diseases from medical images is challenged by the requirement to localise the diagnosed diseases to justify or explain the classification decision. This requirement is hard to fulfil because most of the training sets available to develop these systems only contain global annotations, making the localisation of diseases a weakly supervised approach. The main methods designed for weakly supervised disease classification and localisation rely on saliency or attention maps that are not specifically trained for localisation, or on region proposals that can not be refined to produce accurate detections. In this paper, we introduce a new model that combines region proposal and saliency detection to overcome both limitations for weakly supervised disease classification and localisation. Using the ChestX-ray14 data set, we show that our proposed model establishes the new state-of-the-art for weakly-supervised disease diagnosis and localisation. We make our code available at https://github.com/renato145/RpSalWeaklyDet.

Pedestrian detection is one of the most explored topics in computer vision and robotics. The use of deep learning methods allowed the development of new and highly competitive algorithms. Deep Reinforcement Learning has proved to be within the state-of-the-art in terms of both detection in perspective cameras and robotics applications. However, for detection in omnidirectional cameras, the literature is still scarce, mostly because of their high levels of distortion. This paper presents a novel and efficient technique for robust pedestrian detection in omnidirectional images. The proposed method uses deep Reinforcement Learning that takes advantage of the distortion in the image. By considering the 3D bounding boxes and their distorted projections into the image, our method is able to provide the pedestrian’s position in the world, in contrast to the image positions provided by most state-of-the-art methods for perspective cameras. Our method avoids the need of pre-processing steps to remove the distortion, which is computationally expensive. Beyond the novel solution, our method compares favorably with the state-of-the-art methodologies that do not consider the underlying distortion for the detection task.

Providing reliable descriptions of the agents in a video scene is an essential task in many applications, such as surveillance. However, most works focus solely on the characterization of pedestrians, which is not sufficient to describe complex scenes, where a variety of vehicles ($e.g$., bikes and cars) are also present. In this work we address this limitation and propose a framework based on switching motion fields to efficiently characterize the different agents in a scene. Our method achieves a balanced accuracy of 91.9% on the identification of bikers and pedestrian classes on three challenging scenarios, and provides comprehensive information about their behaviors.

We present 3DRegNet, a novel deep learning architecture for the registration of 3D scans. Given a set of 3D point correspondences, we build a deep neural network to address the following two challenges: (i) classification of the point correspondences into inliers/outliers, and (ii) regression of the motion parameters that align the scans into a common reference frame. With regard to regression, we present two alternative approaches: (i) a Deep Neural Network (DNN) registration and (ii) a Procrustes approach using SVD to estimate the transformation. Our correspondence-based approach achieves a higher speedup compared to competing baselines. We further propose the use of a refinement network, which consists of a smaller 3DRegNet as a refinement to improve the accuracy of the registration. Extensive experiments on two challenging datasets demonstrate that we outperform other methods and achieve state-of-the-art results. The code is available., 15 pages, 8 figures, 6 tables

Achieving robust object segmentations requires the ability to discard outliers (invalid observations) in the segmentation process. When considering an object model, such as an active shape model (ASM), described by a set of points, a one-to-one mapping from one model point to one valid observation would be ideal. However, in general, a one-to-many mapping is necessary to ensure the valid observation is detected and, thus, to have a reliable and robust fitting process. In this work, we compare three observation detectors: two of them based on handcrafted texture features and another based on a deep CNN classifier. Furthermore, to reduce from many detections to just one valid observation, we incorporate the Generalized Expectation-Maximization (GEM) algorithm in the ASM framework. This algorithm is able to neglect outliers by assigning them low weights. The proposed methodology exhibits remarkable accuracy with all the detectors in the context of the segmentation of the left ventricle in two publicly available MRI datasets, for which the proposed approach is competitive with other state-of-the-art methods.

In this research, we take an HCI perspective on the opportunities provided by AI techniques in medical imaging, focusing on workflow efficiency and quality, preventing errors and variability of diagnosis in Breast Cancer. Starting from a holistic understanding of the clinical context, we developed BreastScreening to support Multimodality and integrate AI techniques (using a deep neural network to support automatic and reliable classification) in the medical diagnosis workflow. This was assessed by using a significant number of clinical settings and radiologists. Here we present: i) user study findings of 45 physicians comprising nine clinical institutions; ii) list of design recommendations for visualization to support breast screening radiomics; iii) evaluation results of a proof-of-concept BreastScreening prototype for two conditions Current (without AI assistant) and AI-Assisted; and iv) evidence from the impact of a Multimodality and AI-Assisted strategy in diagnosing and severity classification of lesions. The above strategies will allow us to conclude about the behaviour of clinicians when an AI module is present in a diagnostic system. This behaviour will have a direct impact in the clinicians workflow that is thoroughly addressed herein. Our results show a high level of acceptance of AI techniques from radiologists and point to a significant reduction of cognitive workload and improvement in diagnosis execution.

The performance of deep learning (DL) models is highly dependent on the quality and size of the training data, whose annotations are often expensive and hard to obtain. This work proposes a new strategy to train DL models by Learning Optimal samples Weights (LOW), making better use of the available data. LOW determines how much each sample in a batch should contribute to the training process, by automatically estimating its weight in the loss function. This effectively forces the model to focus on more relevant samples. Consequently, the models exhibit a faster convergence and better generalization, specially on imbalanced data sets where class distribution is long-tailed. LOW can be easily integrated to train any DL model and can be combined with any loss function, while adding marginal computational burden to the training process. Additionally, the analysis of how sample weights change during training provides insights on what the model is learning and which samples or classes are more challenging. Results on popular computer vision benchmarks and on medical data sets show that DL models trained with LOW perform better than with other state-of-the-art strategies 1 .

Trajectory prediction is a crucial element of many automated tasks, such as autonomous navigation or video surveillance. To automatically predict the motion of an agent (e.g., pedestrian or car), the model needs to efficiently represent human motion and “understand” the external stimuli that may influence human behavior. In this work we propose a methodology to model the motion of agents in a video scene. Our method is based on space-varying sparse motion fields, which simultaneously characterize diverse motion patterns in the scene and implicitly learn contextual cues about the static environment, namely obstacles and semantic constraints. The sparse motion fields are applied to the task of long-term trajectory prediction using a probabilistic generative approach. Several benchmark data sets are used to demonstrate the potential of the proposed approach and show that our method achieves competitive state-of-the-art performances.

We present new achievements on the use of deep convolutional neural networks (CNN) in the problem of pedestrian detection (PD). In this paper, we aim to address the following questions: (i) Given non-deep state-of-the-art pedestrian detectors (e.g. ACF, LDCF), is it possible to improve their top performances?; (ii) is it possible to apply a pre-trained deep model to these detectors to boost their performances in the PD context? In this paper, we address the aforementioned questions by cascading CNN models (pre-trained on Imagenet) with state-of-the-art non-deep pedestrian detectors. Furthermore, we also show that this strategy is extensible to different segmentation maps (e.g. RGB, gradient, LUV) computed from the same pedestrian bounding box (i.e. the proposal). We demonstrate that the proposed approach is able to boost the detection performance of state-of-the-art non-deep pedestrian detectors. We apply the proposed methodology to address the pedestrian detection problem on the publicly available datasets INRIA and Caltech.

The vast quantity of visual data generated by the rapid expansion of large scale distributed multicamera networks, makes automated person detection and reidentification (RE-ID) essential components of modern surveillance systems. However, the integration of automated person detection and RE-ID algorithms is not without problems, and the errors arising in this integration must be measured (e.g., detection failures that may hamper the RE-ID performance). In this paper, we present a window-based classifier based on a recently proposed architecture for the integration of pedestrian detectors and RE-ID algorithms, that takes the output of any bounding-box RE-ID classifier and exploits the temporal continuity of persons in video streams. We evaluate our contributions on a recently proposed dataset featuring 13 high-definition cameras and over 80 people, acquired during 30 min at rush hour in an office space scenario. We expect our contributions to drive research in integrated pedestrian detection and RE-ID systems, bringing them closer to practical applications.

Deep-learning-based solutions have experienced a massive interest from the medical image analysis community. The main reasons behind this attention lie in the ability of such solutions to process h...

We present a detection model that is capable of accelerating the inference time of lesion detection from breast dynamically contrast-enhanced magnetic resonance images (DCE-MRI) at state-of-the-art accuracy. In contrast to previous methods based on computationally expensive exhaustive search strategies, our method reduces the inference time with a search approach that gradually focuses on lesions by progressively transforming a bounding volume until the lesion is detected. Such detection model is trained with reinforcement learning and is modeled by a deep Q-network (DQN) that iteratively outputs the next transformation to the current bounding volume. We evaluate our proposed approach in a breast MRI data set containing the T1-weighted and the first DCE-MRI subtraction volume from 117 patients and a total of 142 lesions. Results show that our proposed reinforcement learning based detection model reaches a true positive rate (TPR) of 0.8 at around three false positive detections and a speedup of at least 1.78 times compared to baselines methods.

Obtaining the segmentation of an object in a sequence of images is usually achieved using a tracking methodology. However, in some applications, the whole sequence is available beforehand. This means that the segmentations can be determined simultaneously for all the frames in the sequence and taking into account the motion of the object. This paper proposes a new framework to incorporate motion information in the segmentation of image sequences using an active shape model (ASM). The motion of the object is modeled using a vector field, which is learned and refined online as the segmentation algorithm proceeds. The vector field is determined from the trajectories described by ASM points throughout the sequence. The vector field, in turn, influences the estimation of the ASM parameters by acting as a regularizer, ensuring that the segmentations are in agreement with the expected motion. The results show that coupling these models during the segmentation leads to an increase in performance, in particular by guarantee more consistent segmentations and by avoid gross errors in more challenging frames.

The training and the evaluation of learning algorithms depend critically on the quality of data samples. We denote as pure the samples that identify clearly and without any ambiguity the class of objects of interest. For instance, in pedestrian detection algorithms, we consider as pure samples the ones containing persons who are fully visible and are imaged at a good resolution (larger than the detector window in size). The exclusive use of pure samples entails two kinds of problems. In training, it biases the detector to neglect slightly occluded and small sized samples (which we denote as impure), thus reducing its detection rate in a real world application. In testing, it leads to the unfair evaluation and comparison of different detectors since slightly impure samples, when detected, can be accounted for as false positives. In this paper we study how a sensible use of impure samples can benefit both the training and the evaluation of pedestrian detection algorithms. We improve the labelling of one of the most widely used pedestrian data sets (INRIA) taking into account the degree of sample impurity. We observe that including partially occluded pedestrians in the training improves performance, not only on partially visible examples, but also on the fully visible ones. Furthermore, we found that including pedestrians imaged at low resolutions is beneficial for detecting pedestrians in the same range of heights, leaving the performance on pure samples unchanged. However, including samples with too high a grade of impurity degrades the performance, thus a careful balance must be found. The proposed labelling will allow further studies on the role of impure samples in training pedestrian detectors and on devising fairer comparison metrics between different algorithms.

Object segmentation is still an active topic that is highly visited in image processing and computer vision communities. This task is challenging due not only to difficult image conditions (e.g., poor resolution or contrast), but also to objects whose appearance vary significantly. This paper visits the Active Shape Model (ASM) that has become a widely used deformable model for object segmentation in images. Since the success of this model depends on its ability to locate the object, many detectors have been proposed. Here, we propose a new methodology in which the ASM search takes the form of local rectangular regions sampled around each landmark point. These regions are then correlated to variable or fixed texture templates learned over a training set. We compare the performance of the proposed approach against other detectors based on: (i) the classical ASM edge detection; (ii) the Histogram of Oriented Gradients (HOG); and (iii) the Scale-Invariant Feature Transform (SIFT). The evaluation is performed in two different applications: facial fitting and segmentation of the left ventricle (LV) in cardiac magnetic resonance (CMR) images, showing that the proposed method leads to a significant increase in accuracy and outperforms the other approaches.

Modeling the trajectories of pedestrians is a key task in video surveillance. However, finding a suitable model to describe the trajectories is challenging, mainly because several of the models tend to have a large number of parameters to be estimated. This paper addresses this issue and provides insights on how to tackle this problem. We model the trajectories using a mixture of vector fields with probabilistic switching mechanism that allows to efficiently change the trajectory motion. Depending on the probabilistic formulation, the motions fields can have a dense or sparse representation, which we believe influences the performance of the model. Moreover, the model has a large set of parameters that need to be estimated using the initialization-dependent EM-algorithm. To overcome the previous issues, an extensive study of the parameters estimation is conducted, namely: (i) initialization, and (ii) priors distribution that controls the sparsity of the solution. The various models are evaluated in the trajectory prediction task, using a newly proposed method. Experimental results in both synthetic and real examples provide new insights and valuable information how the parameters play an important in the proposed framework.

The segmentation of the left ventricle (LV) is an important task to assess the cardiac function in ultrasound images of the heart. This paper presents a novel methodology for the segmentation of the LV in three-dimensional (3-D) echocardiographic images based on the probabilistic data association filter (PDAF). The proposed methodology begins by initializing a 3-D deformable model either semiautomatically, with user input, or automatically, and it comprises the following feature hierarchical approach: 1) edge detection in the vicinity of the surface (low-level features); 2) edge grouping to obtain potential LV surface patches (mid-level features); and 3) patch filtering using a shape-PDAF framework (high-level features). This method provides good performance accuracy in 20 echocardiographic volumes, and compares favorably with the state-of-the-art segmentation methodologies proposed in the recent literature.

This paper proposes a new approach for the segmentation of the endocardium of the left ventricle using short axis magnetic resonance (MR) images. The proposed method comprises two main stages. First, each image is converted to polar coordinates, and an edge map is computed from the transformed image. Then, the contour of the left ventricle (LV) is estimated by computing the optimal path along the edge map, using a dynamic programming approach. The system is evaluated on a public database comprising 660 magnetic resonance volumes and the results testify its usefulness both in terms of running time and accuracy. The proposed methodology is able to segment a whole volume in 1.5 seconds achieving an average Dice similarity coefficient of 85.9% (8.3%), which compares favorably with related state-of-the-art methods.

This paper proposes a novel methodology to describe the trajectories performed by pedestrians in long-range surveillance scenarios. The proposed approach describes the trajectories by using sparse motion/vector fields together with a space-varying switching mechanism embedded in a Hidden Markov Model framework. Despite the diversity of motion patterns that may occur in a given scenario, the observed trajectories do not lie in the entire surveilled area. Instead, they are constrained to patterns corresponding to typical motions. To achieve a compact representation, we propose a sparse model estimated using the l 1 norm applied to the log prior distribution of the vector fields. Experimental evaluation is conducted in real scenarios, and testify the usefulness of the proposed approach in modeling typical trajectories that occur in a far-field surveillance setup.

In this work, we present a context-aware ensemble fusion framework based on soft-biometric features, for long term person re-identification (Re-ID) in wild surveillance scenarios. The characteristics of a person that best correlate to its identity depend strongly on the view point. For instance, a person with a short stride gait is better perceived from a lateral view, whereas a person with a large chest is more distinct from a frontal view. Thus we associate context to the viewing direction of walking people in a surveillance scenario and choose the best features for each case. Using the MS KinectTM sensor v.2, we collect data from walking subjects and extract associated anthropometric and gait features. Each context is analysed with a Feature selection technique (Sequential Forward Selection) so that only the most relevant features for the context are retained. Then, individual context-specific classifiers are trained leveraging those selected features. Finally, we propose a contextaware ensemble fusion strategy, which we term as 'Contextspecific score-level fusion', based on the adaptive weighted sum of the results of individual classifiers. The proposed contextaware Re-ID framework demonstrate significant performance improvement both in terms of speed (up to 4.5 times faster) and accuracy (up to 17% rank-1 Re-ID rate) compared to the Context-unaware systems. From the study, we show that gait features are better for lateral views and anthropometric features are better for frontal views, confirming the results of previous studies.

Estimation of velocity fields from a video sequence is an important step towards activity classification in a surveillance system. It has been recently shown that multiple motion fields estimated from trajectories are an efficient tool to describe the movement of objects, allowing an automatic classification of activities in the scene. However, the trajectory detection in noisy environments is difficult, usually requiring some sort manual editing to complete or correct them. This paper proposes two novel contributions. First, an automatic method for building pedestrian trajectories in far-field surveillance scenarios is presented not requiring user intervention. This basically comprises the detection of multiple moving objects in a video sequence through the detection of the active regions, followed by the estimation of the velocity fields that is accomplished by performing region matching of the above regions at consecutive time instants. This leads to a sequence of centroids and corresponding velocity vectors, describing the local motions presented in the image. A motion correspondence algorithm is then applied to group the centroids in a contiguous sequence of frames into trajectories corresponding to each moving object. The second contribution is a method for automatically finding the trajectories from a library of previously computed ones. Experiments on extensive video sequences from university campuses show that motion fields can be reliably estimated from these automatically detected trajectories, leading to a fully automatic procedure for the estimation of multiple motion fields.

The analysis of moving objects in image sequences (video) has been one of the major themes in computer vision. In this paper, we focus on video-surveillance tasks; more specifically, we consider pedestrian trajectories and propose modeling them through a small set of motion/vector fields together with a space-varying switching mechanism. Despite the diversity of motion patterns that can occur in a given scene, we show that it is often possible to find a relatively small number of typical behaviors, and model each of these behaviors by a "simple" motion field. We increase the expressiveness of the formulation by allowing the trajectories to switch from one motion field to another, in a space-dependent manner. We present an expectation-maximization algorithm to learn all the parameters of the model, and apply it to trajectory classification tasks. Experiments with both synthetic and real data support the claims about the performance of the proposed approach.

We present a novel methodology for the automated detection of breast lesions from dynamic contrast-enhanced magnetic resonance volumes (DCE-MRI). Our method, based on deep reinforcement learning, significantly reduces the inference time for lesion detection compared to an exhaustive search, while retaining state-of-art accuracy. This speed-up is achieved via an attention mechanism that progressively focuses the search for a lesion (or lesions) on the appropriate region(s) of the input volume. The attention mechanism is implemented by training an artificial agent to learn a search policy, which is then exploited during inference. Specifically, we extend the deep Q-network approach, previously demonstrated on simpler problems such as anatomical landmark detection, in order to detect lesions that have a significant variation in shape, appearance, location and size. We demonstrate our results on a dataset containing 117 DCE-MRI volumes, validating run-time and accuracy of lesion detection.

In this paper, we propose an ensemble classification approach to the Pedestrian Detection (PD) problem, resorting to distinct input channels and Convolutional Neural Networks (CNN). This methodology comprises two stages: (i) the proposals extraction, and (ii) the ensemble classification. In order to obtain the proposals, we apply several detectors specifically developed for the PD task. Afterwards, these proposals are converted into different input channels (e.g. gradient magnitude, LUV or RGB), and classified by each CNN. Finally, several ensemble methods are used to combine the output probabilities of each CNN model. By correctly selecting the best combination strategy, we achieve improvements, comparatively to the single CNN models predictions.

Background and Objective The segmentation of the left ventricle (LV) in cardiac magnetic resonance imaging is a necessary step for the analysis and diagnosis of cardiac function. In most clinical setups, this step is still manually performed by cardiologists, which is time-consuming and laborious. This paper proposes a fast system for the segmentation of the LV that significantly reduces human intervention. Methods A dynamic programming approach is used to obtain the border of the LV. Using very simple assumptions about the expected shape and location of the segmentation, this system is able to deal with many of the challenges associated with this problem. The system was evaluated on two public datasets: one with 33 patients, comprising a total of 660 magnetic resonance volumes and another with 45 patients, comprising a total of 90 volumes. Quantitative evaluation of the segmentation accuracy and computational complexity was performed. Results The proposed system is able to segment a whole volume in 1.5 seconds and achieves an average Dice similarity coefficient of 86.0% and an average perpendicular distance of 2.4 mm, which compares favorably with other state-of-the-art methods. Conclusions A system for the segmentation of the left ventricle in cardiac magnetic resonance imaging is proposed. It is a fast framework that significantly reduces the amount of time and work required of cardiologists.

A real-time Deep Learning based method for Pedestrian Detection (PD) is applied to the Human-Aware robot navigation problem. The pedestrian detector combines the Aggregate Channel Features (ACF) detector with a deep Convolutional Neural Network (CNN) in order to obtain fast and accurate performance. Our solution is firstly evaluated using a set of real images taken from onboard and offboard cameras and, then, it is validated in a typical robot navigation environment with pedestrians (two distinct experiments are conducted). The results on both tests show that our pedestrian detector is robust and fast enough to be used on robot navigation applications.

We propose new methods for the prediction of 5-year mortality in elderly individuals using chest computed tomography (CT). The methods consist of a classifier that performs this prediction using a set of features extracted from the CT image and segmentation maps of multiple anatomic structures. We explore two approaches: 1) a unified framework based on deep learning, where features and classifier are automatically learned in a single optimisation process; and 2) a multi-stage framework based on the design and selection/extraction of hand-crafted radiomics features, followed by the classifier learning process. Experimental results, based on a dataset of 48 annotated chest CTs, show that the deep learning model produces a mean 5-year mortality prediction accuracy of 68.5%, while radiomics produces a mean accuracy that varies between 56% to 66% (depending on the feature selection/extraction method and classifier). The successful development of the proposed models has the potential to make a profound impact in preventive and personalised healthcare., 9 pages

We present a new supervised learning model designed for the automatic segmentation of the left ventricle (LV) of the heart in ultrasound images. We address the following problems inherent to supervised learning models: 1) the need of a large set of training images; 2) robustness to imaging conditions not present in the training data; and 3) complex search process. The innovations of our approach reside in a formulation that decouples the rigid and nonrigid detections, deep learning methods that model the appearance of the LV, and efficient derivative-based search algorithms. The functionality of our approach is evaluated using a data set of diseased cases containing 400 annotated images (from 12 sequences) and another data set of normal cases comprising 80 annotated images (from two sequences), where both sets present long axis views of the LV. Using several error measures to compute the degree of similarity between the manual and automatic segmentations, we show that our method not only has high sensitivity and specificity but also presents variations with respect to a gold standard (computed from the manual annotations of two experts) within interuser variability on a subset of the diseased cases. We also compare the segmentations produced by our approach and by two state-of-the-art LV segmentation models on the data set of normal cases, and the results show that our approach produces segmentations that are comparable to these two approaches using only 20 training images and increasing the training set to 400 images causes our approach to be generally more accurate. Finally, we show that efficient search methods reduce up to tenfold the complexity of the method while still producing competitive segmentations. In the future, we plan to include a dynamical model to improve the performance of the algorithm, to use semisupervised learning methods to reduce even more the dependence on rich and large training sets, and to design a shape model less dependent on the training set.

We present a color and shape based 3D tracking system suited to a large class of vision sensors. The method is applicable, in principle, to any known calibrated projection model. The tracking architecture is based on particle filtering methods where each particle represents the 3D state of the object, rather than its state in the image, therefore overcoming the nonlinearity caused by the projection model. This allows the use of realistic 3D motion models and easy incorporation of self-motion measurements. All nonlinearities are concentrated in the observation model so that each particle projects a few tens of special points onto the image, on (and around) the 3D object's surface. The likelihood of each state is then evaluated by comparing the color distributions inside and outside the object's occluding contour. Since only pixel access operations are required, the method does not require the use of image processing routines like edge/feature extraction, color segmentation or 3D reconstruction, which can be sensitive to motion blur and optical distortions typical in applications of omnidirectional sensors to robotics. We show tracking applications considering different objects (balls, boxes), several projection models (catadioptric, dioptric, perspective) and several challenging scenarios (clutter, occlusion, illumination changes, motion and optical blur). We compare our methodology against a state-of-the-art alternative, both in realistic tracking sequences and with ground truth generated data.

This paper proposes an approach for recognizing human activities (more specifically, pedestrian trajectories) in video sequences, in a surveillance context. A system for automatic processing of video information for surveillance purposes should be capable of detecting, recognizing, and collecting statistics of human activity, reducing human intervention as much as possible. In the method described in this paper, human trajectories are modeled as a concatenation of segments produced by a set of low level dynamical models. These low level models are estimated in an unsupervised fashion, based on a finite mixture formulation, using the expectation-maximization (EM) algorithm; the number of models is automatically obtained using a minimum message length (MML) criterion. This leads to a parsimonious set of models tuned to the complexity of the scene. We describe the switching among the low-level dynamic models by a hidden Markov chain; thus, the complete model is termed a switched dynamical hidden Markov model (SD-HMM). The performance of the proposed method is illustrated with real data from two different scenarios: a shopping center and a university campus. A set of human activities in both scenarios is successfully recognized by the proposed system. These experiments show the ability of our approach to properly describe trajectories with sudden changes.

This paper describes an algorithm for classifying human motion patterns (trajectories) observed in video sequences. We address this task in a hierarchical way: high-level activities are described as sequences of low-level motion patterns (dynamic models). These low-level dynamic models are simply independent increment processes, each describing a specific motion regime (e.g., ''moving left''). Classifying a trajectory thus consists in segmenting it into the sequence its low-level components; each sequence of low-level components corresponds to a high-level activity. To perform the segmentation, we introduce a penalized maximum-likelihood criterion which is able to select the number of segments via a novel MDL-type penalty. Experiments with synthetic and real data illustrate the effectiveness of the proposed approach.

In this work, we propose a novel methodology of re-identifying people in frontal video sequences, based on a spatio-temporal representation of the gait based on optic flow features, which we call Histogram Of Flow Energy Image (HOFEI). Optic Flow based methods do not require the silhouette computation thus avoiding image segmentation issues and enabling online re-identification (Re-ID) tasks. Not many works addressed Re-ID with optic flow features in frontal gait. Here, we conduct an extensive study on CASIA dataset, as well as its application in a realistic surveillance scenario- HDA Person dataset. Results show, for the first time, the feasibility of gait re-identification in frontal sequences, without the need for image segmentation.

Measuring airways in chest computed tomography (CT) images is important for characterizing diseases such as cystic fibrosis, yet very time-consuming to perform manually. Machine learning algorithms offer an alternative, but need large sets of annotated data to perform well. We investigate whether crowdsourcing can be used to gather airway annotations which can serve directly for measuring the airways, or as training data for the algorithms. We generate image slices at known locations of airways and request untrained crowd workers to outline the airway lumen and airway wall. Our results show that the workers are able to interpret the images, but that the instructions are too complex, leading to many unusable annotations. After excluding unusable annotations, quantitative results show medium to high correlations with expert measurements of the airways. Based on this positive experience, we describe a number of further research directions and provide insight into the challenges of crowdsourcing in medical images from the perspective of first-time users.

Extraction of trajectories of moving objects from video is an important step towards activity classification in a surveillance system. It has been recently shown that multiple motion fields (MMF) estimated from trajectories can efficiently describe the movement of objects, and allow an automatic classification of activities in the scene. However, the object segmentation and tracking may introduce gaps in trajectories due to the sensor failures (i.e. miss detections) or pedestrian occlusions. This may hamper the performance of activity classification from the observed trajectories. In this paper, we propose a predictive motion model based on MMF to describe the trajectory of the object. Precomputed motion fields are used to predict the possible position of an object based on its location and trajectory up to that point and therefore, the method can deal with the case of a missed detection in one or more frames, thus able to perform trajectory completion. Experiments on real data show that the proposed method is remarkable in dealing with up to around 70% of the positions missing in the pedestrian trajectories.

The problem of non-rigid object segmentation is formulated in a two-stage approach in Machine Learning based methodologies. In the first stage, the automatic initialization problem is solved by the estimation of a rigid shape of the object. In the second stage, the non-rigid segmentation is performed. The rational behind this strategy, is that the rigid detection can be performed at lower dimensional space than the original contour space. In this paper, we explore this idea and propose the use of manifolds to reduce even more the dimensionality of the rigid transformation space (first stage) of current state-of-the-art top-down segmentation methodologies. Also, we propose the use of deep belief networks to allow for a training process capable to produce robust appearance models. Experiments in lips segmentation from frontal face images are conducted to testify the performance of the proposed algorithm.

In this paper, we propose novel methods to evaluate the performance of object detection algorithms in video sequences. This procedure allows us to highlight characteristics (e.g., region splitting or merging) which are specific of the method being used. The proposed framework compares the output of the algorithm with the ground truth and measures the differences according to objective metrics. In this way it is possible to perform a fair comparison among different methods, evaluating their strengths and weaknesses and allowing the user to perform a reliable choice of the best method for a specific application. We apply this methodology to segmentation algorithms recently proposed and describe their performance. These methods were evaluated in order to assess how well they can detect moving regions in an outdoor scene in fixed-camera situations

Shape tracking with low level features (e.g., edge points) often fails in complex environments (e.g., in the presence of clutter, inner edges, or multiple objects). Two alternative methods are discussed in this paper. Both methods use middle level features (data centroids, strokes) which are more informative and reliable than edge transitions used in most tracking algorithms. Furthermore, it is assumed in this paper that each feature can be either a valid measurement or an outlier. A confidence degree is assigned to each feature or to a given interpretation of all visual features. Features/ interpretations with high degrees of confidence have large influence on the shape estimates while features/interpretations with low degrees of confidence have negligible influence on the shape estimates. It is shown in this paper that both items (middle level features and confidence degrees) lead to a significant improvement of the tracker robustness and performance in the presence of clutter and abrupt shape and motion changes.

3D Active shape models use a set of annotated volumes to learn a shape model. The shape model is defined by a fixed number of landmarks at specific locations and takes shape constraints into account in the segmentation process. A relevant problem in which these models can be used is the segmentation of the left ventricle in 3D MRI volumes. In this problem, the annotations correspond to a set of contours that define the LV border at each volume slice. However, each volume has a different number of slices (i.e., a different number of landmarks), which makes model learning difficult. Furthermore, motion artifacts and the large distance between slices make interpolation of voxel intensities a bad choice when applying the learned model to a test volume. These two problems raise the following questions: (1) how can we learn a shape model from volumes with a variable number of slices? and (2) how can we segment a test volume without interpolating voxel intensities between slices? This paper provides an answer to these questions and proposes a 3D active shape model that can be used to segment the left ventricle in cardiac MRI.