Search

Thermal infrared imaging is an emerging modality that has gained increasing interest in recent years, mostly due to technical advances resulting in the availability of affordable microbolometer-based IR imaging sensors. However, while sensors are widely available, algorithms for thermal image processing still lack robustness and accuracy when compared to their RGB counterparts. Current methods developed for RGB data make use of machine learning algorithms that require large amounts of labeled images which are currently not available for the thermal domain. In this paper, we address the question whether providing a large number of labeled images would allow the application of current image processing methods on the example of solving challenging face analysis tasks. We introduce a high-resolution thermal facial image database with extensive manual annotations and explore how it can be used to adapt methods from the visual domain for infrared images. In addition, we extend existing approaches for infrared landmark detection with a head pose estimation for improved robustness and analyze the performance of a deep learning method on this task. An evaluation of algorithm performance shows that learning algorithms either outperform available solutions or allow completely new applications that could previously not be addressed. As a conclusion, we prove that investing the effort into acquiring appropriate training data and adapting competitive algorithms is not only a viable approach in analysing thermal infrared images but can also allow outperforming specific task-designed solutions. The database is freely available for academic use at https://github.com/marcinkopaczka/thermalfaceproject .

Recent advances in deep learning have shown the capability to accurately segment cardiac structures in magnetic resonance images. However, while these models provide a good segmentation performance for the specified datasets, their generalization with respect to unseen data across different MRI scanners, vendors or clinics is still under investigation. Previous work that aims to increase the generalization performance provides proof that emphasizing the model design on a uniform preprocessing step may be more beneficial than searching for a better neural architecture. In this paper we build upon this idea and show that a carefully designed preprocessing pipeline plays an important role in enabling the neural network to generalize to the large variety in MRI images. We evaluate our model in the context of the Multi-Centre, Multi-Vendor & Multi-Disease Cardiac Image (M&Ms) Segmentation Challenge.

Face alignment is commonly solved by utilizing a shape representation which is either implicitly learned as in current deep architectures or explicitly given as in active shape models and their derivatives. In this work, we present a method for integrating explicit statistical shape priors into a deep convolutional network by introducing a novel layer which allows using statistical PCA-based models directly in the network, resulting in a highly efficient deep face alignment architecture capable of performing landmark detection with several hundreds of frames per second on consumer hardware. In addition to introducing the core algorithm, we also introduce and discuss extensions for improving fitting precision. We validate the performance of all methods quantitatively on the 300W dataset, where we achieve state-of-the-art precision while requiring only a fraction of the runtime of most competing methods.

Identifying image features that are robust with respect to segmentation variability and domain shift is a tough challenge in radiomics. So far, this problem has mainly been tackled in test-retest analyses. In this work we analyze radiomics feature stability based on probabilistic segmentations. Based on a public lung cancer dataset, we generate an arbitrary number of plausible segmentations using a Probabilistic U-Net. From these segmentations, we extract a high number of plausible feature vectors for each lung tumor and analyze feature variance with respect to the segmentations. Our results suggest that there are groups of radiomic features that are more (e.g. statistics features) and less (e.g. gray-level size zone matrix features) robust against segmentation variability. Finally, we demonstrate that segmentation variance impacts the performance of a prognostic lung cancer survival model and propose a new and potentially more robust radiomics feature selection workflow., Comment: accepted at ISBI 2020

Analysis of laboratory animal behavior allows assessment of animal wellbeing. We present a method for the classification of different activities of laboratory mice by analyzing video clips using three deep learning methods. Animals placed in observation cages are filmed and short video clips are labelled as belonging to one of five defined behaviors. Subsequently, three different methods based on convolutional neural networks (CNNS) are applied to classify the clips. The best performing method - a two-stream network that analyzes individual frames as well as the video’s optical flow - achieves an accuracy of 86.4%, including detection of important behavioral patterns such as self-grooming. These results show that the presented analysis protocol allows automated assessment of animal behavior by algorithmic analysis of videos of mice on observation boxes.

We present a multistage method for deep semantic segmentation of bone structures based on a landmark-based shape regression and subsequent local segmentation of relevant areas. Our solution covers the entire pipeline from 2D-based pre-segmentation, a method for fast deep 3D shape regression and subsequent patch-based 3D semantic segmentation for final segmentation. Since we perform landmark regression using a statistical shape model, our method is able to fit an arbitrary number of landmarks without increase in model complexity. The algorithm is evaluated on the OAI-ZIB dataset, for which we use the binary masks to generate sets of corresponding landmarks and build a deep statistical shape model. By employing our proposed deep shape fitting, our method achieves the performance of existing high-precision approaches in terms of segmentation accuracy while at the same time drastically reducing computational complexity and improving runtime by a large margin.

We introduce a domain switch for deep neural networks that enables to re-weight convolutional kernels for an input of a known domain. This technique is designed to address re-occurring tasks across multiple domains that are known at runtime and to incorporate them into a single, domain-spanning network. We evaluate this approach in three distinct tasks, namely combined cell nuclei analysis across different stains and fluorescence images, facial landmark detection in grayscale and thermal infrared images, and the BraTS challenge where we treat different recording institutions as domains. We found that conventional U-nets trained on multiple domains perform similar to domain-specific U-nets. Our method improves the results in facial landmark detection significantly, but no change is measured in the other two experiments compared to multi-domain U-nets.

Thermal infrared imaging is an emerging imaging modality allowing capturing heat radiation not detectable in the visible spectrum. In recent years, numerous applications of thermal infrared imaging for the processing of face images have been published. Many of these approaches only allow minimal head movement due to the lack of sufficiently robust face tracking and interpretation algorithms for thermal infrared images. To address this issue, we present a suite of interconnected algorithms for a number of typical facial image processing tasks face detection, head pose estimation, detection and tracking of facial landmarks and facial expression analysis. The modules can be used independently or in combination with each other. For combined use and as a demonstration of the versatility of our solution, we present a multiprocess solution based on a networking middleware that allows using all proposed algorithms to perform real-time face tracking and emotion recognition in thermal infrared images. The full code is made freely available on gitub under GNU license to allow incorporating our solutions into own projects.

A lightweight framework for fast prototyping and training of deep neural networks for medical image analysis.