Search

Machine learning algorithms commonly have one or more parameters that alter the learning behaviour of the algorithm. Examples are the number of trees in a random forest, or the kernel of support vector machines. Hyperparameter optimisation is the process of identifying a set of hyperparameters that yields a generalisable model. There are various approaches to hyperparameter optimisation, for example grid search, random search, and model-based or Bayesian optimisation. These approaches share three characteristics: 1) they explore the hyperparameter space, and 2) identify suitable hyperparameter sets by optimising an objective function, and 3) do so using subsampling techniques where the data are randomly divided into a training and validation set. The training set is used for training a machine learning model with a specific set of hyperparameters, and the validation set is subsequently used to assess performance of the model. The performance on the validation set, or its average across validation sets of multiple random subsamples, is often used as the objective. Anecdotal evidence suggests that for inherently difficult problems—problems where the features contain little information pertaining to the outcome—using model performance on the validation set as objective function leads to selecting hyperparameter sets that do not yield generalisable models. We hypothesize that this due to a variance-bias trade-off during hyperparameter optimisation. By selecting hyperparameter sets based on minimising bias, we might be trading minimal improvements in bias with large increases in variance. Here, we will assess different objective functions for hyperparameter optimisation to understand how the choice of objective function relates to generalisability of the resulting model.

The Image Biomarker Standardisation Initiative (IBSI) aims to improve reproducibility of radiomics studies by standardising the computational process of extracting image biomarkers (features) from images. We have previously established reference values for 169 commonly used features, created a standard radiomics image processing scheme, and developed reporting guidelines for radiomic studies. However, several aspects are not standardised. Here we present a complete version of a reference manual on the use of convolutional filters in radiomics and quantitative image analysis. Filters, such as wavelets or Laplacian of Gaussian filters, play an important part in emphasising specific image characteristics such as edges and blobs. Features derived from filter response maps were found to be poorly reproducible. This reference manual provides definitions for convolutional filters, parameters that should be reported, reference feature values, and tests to verify software compliance with the reference standard., 87 pages. For additional information see https://theibsi.github.io/

For treatment individualisation of patients with locally advanced head and neck squamous cell carcinoma (HNSCC) treated with primary radiochemotherapy, we explored the capabilities of different deep learning approaches for predicting loco-regional tumour control (LRC) from treatment-planning computed tomography images. Based on multicentre cohorts for exploration (206 patients) and independent validation (85 patients), multiple deep learning strategies including training of 3D- and 2D-convolutional neural networks (CNN) from scratch, transfer learning and extraction of deep autoencoder features were assessed and compared to a clinical model. Analyses were based on Cox proportional hazards regression and model performances were assessed by the concordance index (C-index) and the model’s ability to stratify patients based on predicted hazards of LRC. Among all models, an ensemble of 3D-CNNs achieved the best performance (C-index 0.31) with a significant association to LRC on the independent validation cohort. It performed better than the clinical model including the tumour volume (C-index 0.39). Significant differences in LRC were observed between patient groups at low or high risk of tumour recurrence as predicted by the model (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p=0.001$$\end{document}p=0.001). This 3D-CNN ensemble will be further evaluated in a currently ongoing prospective validation study once follow-up is complete.

The image biomarker standardisation initiative (IBSI) is an independent international collaboration which works towards standardising the extraction of image biomarkers from acquired imaging for the purpose of high-throughput quantitative image analysis (radiomics). Lack of reproducibility and validation of high-throughput quantitative image analysis studies is considered to be a major challenge for the field. Part of this challenge lies in the scantiness of consensus-based guidelines and definitions for the process of translating acquired imaging into high-throughput image biomarkers. The IBSI therefore seeks to provide image biomarker nomenclature and definitions, benchmark data sets, and benchmark values to verify image processing and image biomarker calculations, as well as reporting guidelines, for high-throughput image analysis., Added figures 2.5, 2.6. Replaced figure 2.7. Added missing section header for the normalised dependence count non-uniformity feature. Fixed layout issues with small font sizes that appeared in the last half of the document