Search

Scientific reports 11(1), 23244 (2021). doi:10.1038/s41598-021-02708-y, Published by Macmillan Publishers Limited, part of Springer Nature, [London]

Thermal infrared imaging is an emerging modality that has gained increasing interest in recent years, mostly due to technical advances resulting in the availability of affordable microbolometer-based IR imaging sensors. However, while sensors are widely available, algorithms for thermal image processing still lack robustness and accuracy when compared to their RGB counterparts. Current methods developed for RGB data make use of machine learning algorithms that require large amounts of labeled images which are currently not available for the thermal domain. In this paper, we address the question whether providing a large number of labeled images would allow the application of current image processing methods on the example of solving challenging face analysis tasks. We introduce a high-resolution thermal facial image database with extensive manual annotations and explore how it can be used to adapt methods from the visual domain for infrared images. In addition, we extend existing approaches for infrared landmark detection with a head pose estimation for improved robustness and analyze the performance of a deep learning method on this task. An evaluation of algorithm performance shows that learning algorithms either outperform available solutions or allow completely new applications that could previously not be addressed. As a conclusion, we prove that investing the effort into acquiring appropriate training data and adapting competitive algorithms is not only a viable approach in analysing thermal infrared images but can also allow outperforming specific task-designed solutions. The database is freely available for academic use at https://github.com/marcinkopaczka/thermalfaceproject .

Background and Objective Segmentation on carpus provides essential information for clinical applications including pathological evaluations, therapy planning, wrist biomechanical analysis, etc. Along with the acquisition procedure of magnetic resonance (MR) technique, poor quality of wrist images (e.g., occlusion, low signal-to-noise ratio, and contrast) often causes segmentation failure. Methods In this work, to address such problems, a shape prior enhanced level set model was proposed. By transferring a shape contour in Cartesian Coordinate System (COS) into a curve in Polar Coordinate System (POS), parameters describing conventional shape invariance, i.e., translations, rotation, and scale were simplified into a single parameter for phase shift, which strongly improved algorithm efficiency. Given a training set in COS, a confidence interval representing the corresponding curves in POS was utilized as the shape prior set term in the model. Integrated with an edge detector, a local intensity descriptor, and a regularization term, the proposed method further possessed abilities against noise, intensity inhomogeneity as well as re-initialization problem. Images from 15 in-vivo acquired MR-datasets of the human wrist were used for validation. The performance of the proposed method has been compared with three state-of-the-art methods. Results We reported a Dice Similarity Coefficient of 96.88±1.20%, a Relative Volume Difference of -1.53±3.01%, a Volume Overlap Error of 6.03±2.23%, a 95% Hausdorff Distance of 1.43±0.66 mm, an Average Symmetric Surface Distance of 0.50±0.17 mm, and a Root Mean Square Distance of 0.71±0.25 mm for the proposed method. The time consumption was 36.03±19.98 s. Conclusions Experimental results indicated that, compared with three other methods, the proposed method achieved significant improvement in terms of accuracy and efficiency.

Monthly notices of the Royal Astronomical Society 503(1), 574-585 (2021). doi:10.1093/mnras/stab427, Published by Oxford Univ. Press, Oxford

Recent advances in deep learning have shown the capability to accurately segment cardiac structures in magnetic resonance images. However, while these models provide a good segmentation performance for the specified datasets, their generalization with respect to unseen data across different MRI scanners, vendors or clinics is still under investigation. Previous work that aims to increase the generalization performance provides proof that emphasizing the model design on a uniform preprocessing step may be more beneficial than searching for a better neural architecture. In this paper we build upon this idea and show that a carefully designed preprocessing pipeline plays an important role in enabling the neural network to generalize to the large variety in MRI images. We evaluate our model in the context of the Multi-Centre, Multi-Vendor & Multi-Disease Cardiac Image (M&Ms) Segmentation Challenge.

Identifying image features that are robust with respect to segmentation variability and domain shift is a tough challenge in radiomics. So far, this problem has mainly been tackled in test-retest analyses. In this work we analyze radiomics feature stability based on probabilistic segmentations. Based on a public lung cancer dataset, we generate an arbitrary number of plausible segmentations using a Probabilistic U-Net. From these segmentations, we extract a high number of plausible feature vectors for each lung tumor and analyze feature variance with respect to the segmentations. Our results suggest that there are groups of radiomic features that are more (e.g. statistics features) and less (e.g. gray-level size zone matrix features) robust against segmentation variability. Finally, we demonstrate that segmentation variance impacts the performance of a prognostic lung cancer survival model and propose a new and potentially more robust radiomics feature selection workflow., Comment: accepted at ISBI 2020

Analysis of laboratory animal behavior allows assessment of animal wellbeing. We present a method for the classification of different activities of laboratory mice by analyzing video clips using three deep learning methods. Animals placed in observation cages are filmed and short video clips are labelled as belonging to one of five defined behaviors. Subsequently, three different methods based on convolutional neural networks (CNNS) are applied to classify the clips. The best performing method - a two-stream network that analyzes individual frames as well as the video’s optical flow - achieves an accuracy of 86.4%, including detection of important behavioral patterns such as self-grooming. These results show that the presented analysis protocol allows automated assessment of animal behavior by algorithmic analysis of videos of mice on observation boxes.

We present a multistage method for deep semantic segmentation of bone structures based on a landmark-based shape regression and subsequent local segmentation of relevant areas. Our solution covers the entire pipeline from 2D-based pre-segmentation, a method for fast deep 3D shape regression and subsequent patch-based 3D semantic segmentation for final segmentation. Since we perform landmark regression using a statistical shape model, our method is able to fit an arbitrary number of landmarks without increase in model complexity. The algorithm is evaluated on the OAI-ZIB dataset, for which we use the binary masks to generate sets of corresponding landmarks and build a deep statistical shape model. By employing our proposed deep shape fitting, our method achieves the performance of existing high-precision approaches in terms of segmentation accuracy while at the same time drastically reducing computational complexity and improving runtime by a large margin.

Next generation interferometers, such as the Square Kilometre Array, are set to obtain vast quantities of information about the kinematics of cold gas in galaxies. Given the volume of data produced by such facilities astronomers will need fast, reliable, tools to informatively filter and classify incoming data in real time. In this paper, we use machine learning techniques with a hydrodynamical simulation training set to predict the kinematic behaviour of cold gas in galaxies and test these models on both simulated and real interferometric data. Using the power of a convolutional autoencoder we embed kinematic features, unattainable by the human eye or standard tools, into a three-dimensional space and discriminate between disturbed and regularly rotating cold gas structures. Our simple binary classifier predicts the circularity of noiseless, simulated, galaxies with a recall of $85\%$ and performs as expected on observational CO and HI velocity maps, with a heuristic accuracy of $95\%$. The model output exhibits predictable behaviour when varying the level of noise added to the input data and we are able to explain the roles of all dimensions of our mapped space. Our models also allow fast predictions of input galaxies' position angles with a $1\sigma$ uncertainty range of $\pm17^{\circ}$ to $\pm23^{\circ}$ (for galaxies with inclinations of $82.5^{\circ}$ to $32.5^{\circ}$, respectively), which may be useful for initial parameterisation in kinematic modelling samplers. Machine learning models, such as the one outlined in this paper, may be adapted for SKA science usage in the near future., Comment: 15 pages, 11 figures

We introduce a domain switch for deep neural networks that enables to re-weight convolutional kernels for an input of a known domain. This technique is designed to address re-occurring tasks across multiple domains that are known at runtime and to incorporate them into a single, domain-spanning network. We evaluate this approach in three distinct tasks, namely combined cell nuclei analysis across different stains and fluorescence images, facial landmark detection in grayscale and thermal infrared images, and the BraTS challenge where we treat different recording institutions as domains. We found that conventional U-nets trained on multiple domains perform similar to domain-specific U-nets. Our method improves the results in facial landmark detection significantly, but no change is measured in the other two experiments compared to multi-domain U-nets.

Thermal infrared imaging is an emerging imaging modality allowing capturing heat radiation not detectable in the visible spectrum. In recent years, numerous applications of thermal infrared imaging for the processing of face images have been published. Many of these approaches only allow minimal head movement due to the lack of sufficiently robust face tracking and interpretation algorithms for thermal infrared images. To address this issue, we present a suite of interconnected algorithms for a number of typical facial image processing tasks face detection, head pose estimation, detection and tracking of facial landmarks and facial expression analysis. The modules can be used independently or in combination with each other. For combined use and as a demonstration of the versatility of our solution, we present a multiprocess solution based on a networking middleware that allows using all proposed algorithms to perform real-time face tracking and emotion recognition in thermal infrared images. The full code is made freely available on gitub under GNU license to allow incorporating our solutions into own projects.

A lightweight framework for fast prototyping and training of deep neural networks for medical image analysis.