Search

This manuscript is the third in a five-part series related to statistical assessment methodology for technical performance of multi-parametric quantitative imaging biomarkers (mp-QIBs). We outline approaches and statistical methodologies for developing and evaluating a phenotype classification model from a set of multiparametric QIBs. We then describe validation studies of the classifier for precision, diagnostic accuracy, and interchangeability with a comparator classifier. We follow with an end-to-end real-world example of development and validation of a classifier for atherosclerotic plaque phenotypes. We consider diagnostic accuracy and interchangeability to be clinically meaningful claims for a phenotype classification model informed by mp-QIB inputs, aiming to provide tools to demonstrate agreement between imaging-derived characteristics and clinically established phenotypes. Understanding that we are working in an evolving field, we close our manuscript with an acknowledgement of existing challenges and a discussion of where additional work is needed. In particular, we discuss the challenges involved with technical performance and analytical validation of mp-QIBs. We intend for this manuscript to further advance the robust and promising science of multiparametric biomarker development.

This paper is the fifth in a five-part series on statistical methodology for performance assessment of multi-parametric quantitative imaging biomarkers (mpQIBs) for radiomic analysis. Radiomics is the process of extracting visually imperceptible features from radiographic medical images using data-driven algorithms. We refer to the radiomic features as data-driven imaging markers (DIMs), which are quantitative measures discovered under a data-driven framework from images beyond visual recognition but evident as patterns of disease processes irrespective of whether or not ground truth exists for the true value of the DIM. This paper aims to set guidelines on how to build machine learning models using DIMs in radiomics and to apply and report them appropriately. We provide a list of recommendations, named RANDAM (an abbreviation of "Radiomic ANalysis and DAta Modeling"), for analysis, modeling, and reporting in a radiomic study to make machine learning analyses in radiomics more reproducible. RANDAM contains five main components to use in reporting radiomics studies: design, data preparation, data analysis and modeling, reporting, and material availability. Real case studies in lung cancer research are presented along with simulation studies to compare different feature selection methods and several validation strategies.

Psoriasis is a systemic inflammatory disease with an increased risk of atherosclerotic events and premature cardiovascular disease. S100A7, A8/A9, and A12 are protein complexes that are produced by activated neutrophils, monocytes, and keratinocytes in psoriasis. Lipid-rich necrotic core (LRNC) is a high-risk coronary plaque feature previously found to be associated with cardiovascular risk factors and psoriasis severity. LRNC can decrease with biologic therapy, but how this occurs remains unknown. We investigated the relationship between S100 proteins, LRNC, and biologic therapy in psoriasis. S100A8/A9 associated with LRNC in fully adjusted models (β = 0.27, P = 0.009; n = 125 patients with psoriasis with available coronary computed tomography angiography scans; LRNC analyses; and serum S100A7, S100A8, S100A9, S100A12, and S100A8/A9 levels). At 1 year, in patients receiving biologic therapy (36 of 73 patients had 1-year coronary computed tomography angiography scans available), a 79% reduction in S100A8/A9 levels (‒172 [‒291.7 to 26.4] vs. ‒29.9 [‒137.9 to 50.5]; P = 0.04) and a 0.6 mm

Rationale and Objectives A critical performance metric for any quantitative imaging biomarker is its ability to reliably generate similar values on repeat testing. This is known as the repeatability of the biomarker, and it is used to determine the minimum detectable change needed in order to show that a change over time is real change and not just due to measurement error. Test-retest studies are the classic approach for estimating repeatability; however, these studies can be infeasible when the imaging is expensive, time-consuming, invasive, or requires contrast agents. The objective of this study was to develop and test a method for estimating repeatability without a test-retest study. Materials and Methods We present a statistical method for estimating repeatability and testing whether an imaging method meets a specified criterion for repeatability in the absence of a test-retest study. The new method is applicable for the particular situation where a reference standard is available. A Monte Carlo simulation study was conducted to evaluate the performance of the new method. Results The proposed estimator is unbiased, and hypothesis tests with the new estimator have nominal type I error rate and power similar to a test-retest study. We considered the situation where the reference standard provides the true value, as well as when the reference standard itself has various magnitudes of measurement error. An example from CT imaging biomarkers of atherosclerosis illustrates the new method. Conclusion Precision of a QIB can be measured without a test-retest study in the situation where a reference standard is available.

Objective Ischaemic strokes can be caused by unstable carotid atherosclerosis, but methods for identification of high risk lesions are lacking. Carotid plaque morphology imaging using software for visualisation of plaque components in computed tomography angiography (CTA) may improve assessment of plaque phenotype and stroke risk, but it is unknown if such analyses also reflect the biological processes related to lesion stability. Here, we investigated how carotid plaque morphology by image analysis of CTA is associated with biological processes assessed by transcriptomic analyses of corresponding carotid endarterectomies (CEAs). Methods Carotid plaque morphology was assessed in patients undergoing CEA for symptomatic or asymptomatic carotid stenosis consecutively enrolled between 2006 and 2015. Computer based analyses of pre-operative CTA was performed to define calcification, lipid rich necrotic core (LRNC), intraplaque haemorrhage (IPH), matrix (MATX), and plaque burden. Plaque morphology was correlated with molecular profiles obtained from microarrays of corresponding CEAs and models were built to assess the ability of plaque morphology to predict symptomatology. Results Carotid plaques (n = 93) from symptomatic patients (n = 61) had significantly higher plaque burden and LRNC compared with plaques from asymptomatic patients (n = 32). Lesions selected from the transcriptomic cohort (n = 40) with high LRNC, IPH, MATX, or plaque burden were characterised by molecular signatures coupled with inflammation and extracellular matrix degradation, typically linked with instability. In contrast, highly calcified plaques had a molecular signature signifying stability with enrichment of profibrotic pathways and repressed inflammation. In a cross validated prediction model for symptoms, plaque morphology by CTA alone was superior to the degree of stenosis. Conclusion The study demonstrates that CTA image analysis for evaluation of carotid plaque morphology, also reflects prevalent biological processes relevant for assessment of plaque phenotype. The results support the use of CTA image analysis of plaque morphology for risk stratification and management of patients with carotid stenosis.

The application of machine learning to assess plaque risk phenotypes on cardiovascular CT angiography (CTA) is an area of active investigation. Studies using accepted histologic definitions of plaque risk as ground truth for machine learning models are uncommon. The aim was to evaluate the accuracy of a machine-learning software for determining plaque risk phenotype as compared to expert pathologists (histologic ground truth).Sections of atherosclerotic plaques paired with CTA were prospectively collected from patients undergoing carotid endarterectomy at two centers. Specimens were annotated for lipid-rich necrotic core, calcification, matrix, and intraplaque hemorrhage at 2 mm spacing and classified as minimal disease, stable plaque, or unstable plaque according to a modified American Heart Association histological definition. Phenotype is determined in two steps: plaque morphology is delineated according to histological tissue definitions, followed by a machine learning classifier. The performance in derivation and validation cohorts for plaque risk categorization and stenosis was compared to histologic ground truth at each matched cross-section.A total of 496 and 408 vessel cross-sections in the derivation and validation cohorts (from 30 and 23 patients, respectively). The software demonstrated excellent agreement in the validation cohort with histological ground truth plaque risk phenotypes with weighted kappa of 0.82 [0.78-0.86] and area under the receiver operating curve for correct identification of plaque type was 0.97 [0.96, 0.98], 0.95 [0.94, 0.97], 0.99 [0.99, 1.0] for unstable plaque, stable plaque, and minimal disease, respectively. Diameter stenosis correlated poorly to histologically defined plaque type; weighted kappa 0.25 in the validation cohort.A machine-learning software trained on histological ground-truth tissue inputs demonstrated high accuracy for identifying plaque stability phenotypes as compared to expert pathologists.

Multiparametric quantitative imaging biomarkers (QIBs) offer distinct advantages over single, univariate descriptors because they provide a more complete measure of complex, multidimensional biological systems. In disease, where structural and functional disturbances occur across a multitude of subsystems, multivariate QIBs are needed to measure the extent of system malfunction. This paper, the first Use Case in a series of articles on multiparameter imaging biomarkers, considers multiple QIBs as a multidimensional vector to represent all relevant disease constructs more completely. The approach proposed offers several advantages over QIBs as multiple endpoints and avoids combining them into a single composite that obscures the medical meaning of the individual measurements. We focus on establishing statistically rigorous methods to create a single, simultaneous measure from multiple QIBs that preserves the sensitivity of each univariate QIB while incorporating the correlation among QIBs. Details are provided for metrological methods to quantify the technical performance. Methods to reduce the set of QIBs, test the superiority of the mp-QIB model to any univariate QIB model, and design study strategies for generating precision and validity claims are also provided. QIBs of Alzheimer's Disease from the ADNI merge data set are used as a case study to illustrate the methods described.

Evaluation of coronary artery disease (CAD) using coronary computed tomography angiography (CCTA) has seen a paradigm shift in the last decade. Evidence increasingly supports the clinical utility of CCTA across various stages of CAD, from the detection of early subclinical disease to the assessment of acute chest pain. Additionally, CCTA can be used to noninvasively quantify plaque burden and identify high-risk plaque, aiding in diagnosis, prognosis, and treatment. This is especially important in the evaluation of CAD in immune-driven conditions with increased cardiovascular disease prevalence. Emerging applications of CCTA based on hemodynamic indices and plaque characterization may provide personalized risk assessment, affect disease detection, and further guide therapy. This review provides an update on the evidence, clinical applications, and emerging technologies surrounding CCTA as highlighted at the 2019 National Heart, Lung and Blood Institute CCTA Summit.

Despite advances in therapy, reduction in myocardial infarction or death remains elusive. Whereas computed tomography angiography (CTA) is increasingly appreciated, the analyses are often subjective or qualitative. Methods for specific tissue characterization using histopathologic correlates have recently been reported. We extend this here to demonstrate accurate discrimination between, and quantitation of, lipid-rich necrotic core (LRNC), intraplaque hemorrhage (IPH), and fibrotic tissues.NCT02143102 collected 576 tissue samples with paired CTA. Cardiovascular pathologists annotated LRNC, IPH, and dense calcification (CALC) regions as a reference standard. Blinded to histology, CTA was analyzed using ElucidVivo (Elucid Bioimaging Inc., Boston, MA USA). Structure and tissue characteristics of atherosclerotic plaque from CTA, accounting for both the imaging acquisition process and the biology, accounting for differences in density distributions that result from the different cellular and molecular level milieu of the relevant tissue types.LRNC was tested across a true range of 0-10 mmLRNC, IPH, CALC, and MATX may be objectively quantified using histopathologic correlates automatically from CTA for use singly or in combination to optimize patient care. The availability of objectively validated quantitative markers that may be followed longitudinally may extend the clinical utility of CTA. Additionally, these measures contribute efficacy variables for developing novel drugs and clinical decision support tools for tailored therapeutics.

Current guidelines for primary and secondary prevention of stroke in patients with carotid atherosclerosis are based on the quantification of the degree of stenosis and symptom status. Recent publications have demonstrated that plaque morphology and composition, independent of the degree of stenosis, are important in the risk stratification of carotid atherosclerotic disease. This finding raises the question as to whether current guidelines are adequate or if they should be updated with new evidence, including imaging for plaque phenotyping, risk stratification, and clinical decision-making in addition to the degree of stenosis. To further this discussion, this roadmap consensus article defines the limits of luminal imaging and highlights the current evidence supporting the role of plaque imaging. Furthermore, we identify gaps in current knowledge and suggest steps to generate high-quality evidence, to add relevant information to guidelines currently based on the quantification of stenosis.

RATIONALE AND OBJECTIVES To evaluate a new approach to establish compliance of segmentation tools with the computed tomography volumetry profile of the Quantitative Imaging Biomarker Alliance (QIBA); and determine the statistical exchangeability between real and simulated lesions through an international challenge. MATERIALS AND METHODS The study used an anthropomorphic phantom with 16 embedded physical lesions and 30 patient cases from the Reference Image Database to Evaluate Therapy Response with pathologically confirmed malignancies. Hybrid datasets were generated by virtually inserting simulated lesions corresponding to physical lesions into the phantom datasets using one projection-domain-based method (Method 1), two image-domain insertion methods (Methods 2 and 3), and simulated lesions corresponding to real lesions into the Reference Image Database to Evaluate Therapy Response dataset (using Method 2). The volumes of the real and simulated lesions were compared based on bias (measured mean volume differences between physical and virtually inserted lesions in phantoms as quantified by segmentation algorithms), repeatability, reproducibility, equivalence (phantom phase), and overall QIBA compliance (phantom and clinical phase). RESULTS For phantom phase, three of eight groups were fully QIBA compliant, and one was marginally compliant. For compliant groups, the estimated biases were -1.8 ± 1.4%, -2.5 ± 1.1%, -3 ± 1%, -1.8 ± 1.5% (±95% confidence interval). No virtual insertion method showed statistical equivalence to physical insertion in bias equivalence testing using Schuirmann's two one-sided test (±5% equivalence margin). Differences in repeatability and reproducibility across physical and simulated lesions were largely comparable (0.1%-16% and 7%-18% differences, respectively). For clinical phase, 7 of 16 groups were QIBA compliant. CONCLUSION Hybrid datasets yielded conclusions similar to real computed tomography datasets where phantom QIBA compliant was also compliant for hybrid datasets. Some groups deemed compliant for simulated methods, not for physical lesion measurements. The magnitude of this difference was small (

Objective: The purpose of this study is to assess the value of an automated model-based plaque characterization tool for the prediction of major adverse cardiac events (MACE).Methods: We retrospectively included 45 patients with suspected coronary artery disease of which 16 (33%) experienced MACE within 12 months. Commercially available plaque quantification software was used to automatically extract quantitative plaque morphology: lumen area, wall area, stenosis percentage, wall thickness, plaque burden, remodeling ratio, calcified area, lipid rich necrotic core (LRNC) area and matrix area. The measurements were performed at all cross sections, spaced at 0.5 mm, based on fully 3D segmentations of lumen, wall, and each tissue type. Discriminatory power of these markers and traditional risk factors for predicting MACE were assessed.Results: Regression analysis using clinical risk factors only resulted in a prognostic accuracy of 63% with a corresponding area under the curve (AUC) of 0.587. Based on our plaque morphology analysis, minimal cap thickness, lesion length, LRNC volume, maximal wall area/thickness, the remodeling ratio, and the calcium volume were included into our prognostic model as parameters. The use of morphologic features alone resulted in an increased accuracy of 77% with an AUC of 0.94. Combining both clinical risk factors and morphological features in a multivariate logistic regression analysis increased the accuracy to 87% with a similar AUC of 0.924.Conclusion: An automated model based algorithm to evaluate CCTA-derived plaque features and quantify morphological features of atherosclerotic plaque increases the ability for MACE prognostication significantly compared to the use of clinical risk factors alone.

Rupture of unstable atherosclerotic plaques with a large lipid-rich necrotic core and a thin fibrous cap cause myocardial infarction and stroke. Yet it has not been possible to assess this for individual patients. Clinical guidelines still rely on use of luminal narrowing, a poor indicator but one that persists for lack of effective means to do better. We present a case study demonstrating the assessment of biomechanical indices pertaining to plaque rupture risk non-invasively for individual patients enabled by histologically validated tissue characterization.Routinely acquired clinical images of plaques were analyzed to characterize vascular wall tissues using software validated by histology (ElucidVivo, Elucid Bioimaging Inc.). Based on the tissue distribution, wall stress and strain were then calculated at spatial locations with varied fibrous cap thicknesses at diastolic, mean and systolic blood pressures.The von Mises stress of 152 [131, 172] kPa and the equivalent strain of 0.10 [0.08, 0.12] were calculated where the fibrous cap thickness was smallest (560 μm) (95% CI in brackets). The stress at this location was at a level predictive of plaque failure. Stress and strain at locations with larger cap thicknesses were calculated to be lower, demonstrating a clinically relevant range of risk levels.Patient specific tissue characterization can identify distributions of stress and strain in a clinically relevant range. This capability may be used to identify high-risk lesions and personalize treatment decisions for individual patients with cardiovascular disease and improve prevention of myocardial infarction and stroke.

The current risk assessment for patients with carotid atherosclerosis relies primarily on measuring the degree of stenosis. More reliable risk stratification could improve patient selection for targeted treatment. We have developed and validated a model to predict for major adverse neurologic events (MANE; stroke, transient ischemic attack, amaurosis fugax) that incorporates a combination of plaque morphology, patient demographics, and patient clinical information.We enrolled 221 patients with asymptomatic carotid stenosis of any severity who had undergone computed tomography angiography at baseline and ≥6 months later. The images were analyzed for carotid plaque morphology (plaque geometry and tissue composition). The data were partitioned into training and validation cohorts. Of the 221 patients, 190 had complete records available and were included in the present analysis. The training cohort was used to develop the best model for predicting MANE, incorporating the patient and plaque features. First, single-variable correlation and unsupervised clustering were performed. Next, several multivariable models were implemented for the response variable of MANE. The best model was selected by optimizing the area under the receiver operating characteristic curve (AUC) and Cohen's kappa statistic. The model was validated using the sequestered data to demonstrate generalizability.A total of 62 patients had experienced a MANE during follow-up. Unsupervised clustering of the patient and plaque features identified single-variable predictors of MANE. Multivariable predictive modeling showed that a combination of the plaque features at baseline (matrix, intraplaque hemorrhage [IPH], wall thickness, plaque burden) with the clinical features (age, body mass index, lipid levels) best predicted for MANE (AUC, 0.79), In contrast, the percent diameter stenosis performed the worst (AUC, 0.55). The strongest single variable for discriminating between patients with and without MANE was IPH, and the most predictive model was produced when IPH was considered with wall remodeling. The selected model also performed well for the validation dataset (AUC, 0.64) and maintained superiority compared with percent diameter stenosis (AUC, 0.49).A composite of plaque geometry, plaque tissue composition, patient demographics, and clinical information predicted for MANE better than did the traditionally used degree of stenosis alone for those with carotid atherosclerosis. Implementing this predictive model in the clinical setting could help identify patients at high risk of MANE.

Background To evaluate the feasibility of non-invasive fractional flow reserve (FFR) estimation using histologically-validated assessment of plaque morphology on coronary CTA (CCTA) as inputs to a predictive model further validated against invasive FFR. Methods Patients (n = 113, 59 ± 8.9 years, 77% male) with suspected coronary artery disease (CAD) who had undergone CCTA and invasive FFR between August 2013 and May 2018 were included. Commercially available software was used to extract quantitative plaque morphology inclusive of both vessel structure and composition. The extracted plaque morphology was then fed as inputs to an optimized artificial neural network to predict lesion-specific ischemia/hemodynamically significant CAD with performance validated by invasive FFR. Results A total of 122 lesions were considered, 59 (48%) had low FFR values. Plaque morphology-based FFR assessment achieved an area under the curve, sensitivity and specificity of 0.94, 0.90 and 0.81, respectively, versus 0.71, 0.71, and 0.50, respectively, for an optimized threshold applied to degree of stenosis. The optimized ridge regression model for continuous value estimation of FFR achieved a cross-correlation coefficient of 0.56 and regression slope of 0.59 using cross validation, versus 0.18 and 0.10 for an optimized threshold applied to degree of stenosis. Conclusions Our results show that non-invasive plaque morphology-based FFR assessment may be used to predict lesion-specific ischemia resulting in hemodynamically significant CAD. This substantially outperforms degree of stenosis interpretation and has a comparable level of sensitivity and specificity relative to publicly reported results from computational fluid dynamics-based approaches.

Background: Residual cardiovascular (CV) risk persists despite statin therapy. In REDUCE-IT, icosapent ethyl (IPE) reduced total CV events by 30%, but the mechanisms of benefit are not fully understood. The EVAPORATE trial evaluated the effects of IPE as adjunct to statins on adverse atherosclerotic plaque characteristics by CCTA. Here we use a novel software validated using histology to evaluate the effect of IPE on vulnerable plaque features. Methods: The EVAPORATE trial randomized statin-treated patients, with high TG (135-499 mg/dL), and known atherosclerosis to IPE 4 g/d or placebo. Plaque characteristics including lipid rich necrotic core (LRNC), fibrous cap thickness, and intraplaque hemorrhage (IPH) were assessed using vascuCAP ® (Elucid Bioimaging Inc., Boston, MA). Per-patient multivariable models robust with respect to physiological variation were used to evaluate plaque progression. Results: A total of 60 patients had interpretable images. Relative to placebo, patients on IPE demonstrated decreased wall volume (-7.2 vs. +28.4 mm 3 ), LRNC (-1.4 vs. +9.7 mm 3 ), and IPH (-0.02 vs. +0.3 mm 3 ), as well as increased cap thickness (+100 vs. -290 microns), indicating a migration to more stable phenotypes (p>0.05).Statistical significance was achieved when incorporated into an optimized neural network model of lipid-rich phenotype (p = 0.04), AUROC=0.7[0.63,0.77], sensitivity=0.66[0.59,0.74], specificity=0.64 [0.56,0.72], and Cohen’s kappa=0.3 [0.19,0.41]. Operating points when used as a per-patient measure of response are presented in Figure 1. Conclusions: This study demonstrated that IPE, when added to statin therapy, is associated with reduction in vulnerable plaque, moving patients to a more stable plaque phenotype.

Background: Lipid-rich necrotic core (LRNC), a high-risk coronary plaque feature assessed by coronary computed tomography angiography, is associated with increased risk of future cardiovascular events in patients with subclinical, nonobstructive coronary artery disease. Psoriasis is a chronic inflammatory condition that is associated with increased prevalence of high-risk coronary plaque and risk of cardiovascular events. This study characterized LRNC in psoriasis and how LRNC modulates in response to biologic therapy. Methods: Consecutive biologic naïve psoriasis patients (n=209) underwent coronary computed tomography angiography at baseline and 1-year to assess changes in LRNC using a novel histopathologically validated software (vascuCAP Elucid Bioimaging, Boston, MA) before and after biologic therapy over 1 year. Results: Study participants were middle-aged, predominantly male with similar cardiometabolic and psoriasis status between treatment groups. In all participants at baseline, LRNC was associated with Framingham risk score (β [standardized β]=0.12 [95% CI, 0.00–0.15]; P =0.045), and psoriasis severity (β=0.13 [95% CI, 0.01–0.26]; P =0.029). At 1-year, participants receiving biologic therapy had a reduction in LRNC (mm 2 ; 3.12 [1.99–4.66] versus 2.97 [1.84–4.35]; P =0.028), while those who did not receive biologic therapy over 1 year demonstrated no significant change with nominally higher LRNC (3.12 [1.82–4.60] versus 3.34 [2.04–4.74]; P =0.06). The change in LRNC was significant compared with that of the nonbiologic treated group (ΔLRNC, −0.22 mm 2 versus 0.14 mm 2 , P =0.004) and remained significant after adjusting for cardiovascular risk factors and psoriasis severity (β=−0.09 [95% CI, −0.01 to −0.18]; P =0.033). Conclusions: LRNC was associated with psoriasis severity and cardiovascular risk factors in psoriasis. Additionally, there was favorable modification of LRNC in those on biologic therapy. This study provides evidence of potential reduction in LRNC with treatment of systemic inflammation. Larger, longer follow-up prospective studies should be conducted to understand how changes in LRNC may translate into a reduction in future cardiovascular events in psoriasis.

Coronary artery disease (CAD) is one of the leading causes of death world-wide. Because of its great clinical importance, technological and diagnostic advances occur to combat it at a similarly high rate. With the number of clinical trials that would be necessary, this becomes infeasible and virtual clinical trials (VCT) are necessary. These require virtual patients and virtual pathologies. A generative adversarial network (GAN) was used to create a library of coronary plaques which were physiologically validated with finite element analysis. The resulting plaque library consists of a large number of realistic, variable virtual pathologies for use in VCTs.

Evaluation of coronary artery disease (CAD) using coronary computed tomography angiography (CCTA) has seen a paradigm shift in the last decade. Evidence increasingly supports the clinical utility of CCTA across various stages of CAD, from the detection of early subclinical disease to the assessment of acute chest pain. Additionally, CCTA can be used to noninvasively quantify plaque burden and identify high-risk plaque, aiding in diagnosis, prognosis, and treatment. This is especially important in the evaluation of CAD in immune-driven conditions with increased cardiovascular disease prevalence. Emerging applications of CCTA based on hemodynamic indices and plaque characterization may provide personalized risk assessment, affect disease detection, and further guide therapy. This review provides an update on the evidence, clinical applications, and emerging technologies surrounding CCTA as highlighted at the 2019 National Heart, Lung and Blood Institute CCTA Summit.

The prevention and management of atherosclerosis poses a tough challenge to public health organizations worldwide. Together with myocardial infarction, stroke represents its main manifestation, with up to 25% of all ischemic strokes being caused by thromboembolism arising from the carotid arteries. Therefore, a vast number of publications have focused on the characterization of the culprit lesion, the atherosclerotic plaque. A paradigm shift appears to be taking place at the current state of research, as the attention is gradually moving from the classically defined degree of stenosis to the identification of features of plaque vulnerability, which appear to be more reliable predictors of recurrent cerebrovascular events. The present review will offer a perspective on the present state of research in the field of carotid atherosclerotic disease, focusing on the imaging modalities currently used in the study of the carotid plaque and the impact that such diagnostic means are having in the clinical setting.

As imaging technologies and treatment options continue to advance, imaging outcome measures are becoming increasingly utilized as the basis of making major decisions in new drug development and clinical practice. Quantitative imaging biomarkers (QIBs) are now commonly used for subject selection, response assessment, and safety monitoring. Although quantitative measurements can have many advantages compared with subjective, qualitative endpoints, it is important to recognize that QIBs are measured with error. This study uses Monte Carlo simulation to examine the impact of measurement error on a variety of clinical trial designs as well as to test proposed adjustments for measurement error. The focus is on some of the QIBs currently being studied by the Quantitative Imaging Biomarkers Alliance. The results show that the ability of QIBs to discriminate between health states and predict patient outcome is attenuated by measurement error; however, the known technical performance characteristics of QIBs can be used to adjust study sample size, control the misinterpretation rate of imaging findings, and establish statistically valid decision thresholds. We conclude that estimates of the precision and bias of a QIB are important for properly designing clinical trials and establishing the level of imaging standardization required.

Rationale and Objectives This study aimed to review the current understanding and capabilities regarding use of imaging for noninvasive lesion characterization and its relationship to lung cancer screening and treatment. Materials and Methods Our review of the state of the art was broken down into questions about the different lung cancer image phenotypes being characterized, the role of imaging and requirements for increasing its value with respect to increasing diagnostic confidence and quantitative assessment, and a review of the current capabilities with respect to those needs. Results The preponderance of the literature has so far been focused on the measurement of lesion size, with increasing contributions being made to determine the formal performance of scanners, measurement tools, and human operators in terms of bias and variability. Concurrently, an increasing number of investigators are reporting utility and predictive value of measures other than size, and sensitivity and specificity is being reported. Relatively little has been documented on quantitative measurement of non-size features with corresponding estimation of measurement performance and reproducibility. Conclusions The weight of the evidence suggests characterization of pulmonary lesions built on quantitative measures adds value to the screening for, and treatment of, lung cancer. Advanced image analysis techniques may identify patterns or biomarkers not readily assessed by eye and may also facilitate management of multidimensional imaging data in such a way as to efficiently integrate it into the clinical workflow.

© 2015 The Authors. PURPOSE: To determine the variability of lesion size measurements in computed tomography data sets of patients imaged under a “no change” (“coffee break”) condition and to determine the impact of two reading paradigms on measurement variability. METHOD AND MATERIALS: Using data sets from 32 non-small cell lung cancer patients scanned twice within 15 minutes (“no change”), measurements were performed by five radiologists in two phases: (1) independent reading of each computed tomography dataset (timepoint): (2) a locked, sequential reading of datasets. Readers performed measurements using several sizing methods, including one-dimensional (1D) longest in-slice dimension and 3D semi-automated segmented volume. Change in size was estimated by comparing measurements performed on both timepoints for the same lesion, for each reader and each measurement method. For each reading paradigm, results were pooled across lesions, across readers, and across both readers and lesions, for each measurement method. RESULTS: The mean percent difference (±SD) when pooled across both readers and lesions for 1D and 3D measurements extracted from contours was 2.8 ± 22.2% and 23.4 ± 105.0%, respectively, for the independent reads. For the locked, sequential reads, the mean percent differences (±SD) reduced to 2.52 ± 14.2% and 7.4 ± 44.2% for the 1D and 3D measurements, respectively. CONCLUSION: Even under a “no change” condition between scans, there is variation in lesion size measurements due to repeat scans and variations in reader, lesion, and measurement method. This variation is reduced when using a locked, sequential reading paradigm compared to an independent reading paradigm.

Purpose To (a) evaluate whether plaque tissue characteristics determined with conventional computed tomographic (CT) angiography could be quantitated at higher levels of accuracy by using image processing algorithms that take characteristics of the image formation process coupled with biologic insights on tissue distributions into account by comparing in vivo results and ex vivo histologic findings and (b) assess reader variability. Materials and Methods Thirty-one consecutive patients aged 43-85 years (average age, 64 years) known to have or suspected of having atherosclerosis who underwent CT angiography and were referred for endarterectomy were enrolled. Surgical specimens were evaluated with histopathologic examination to serve as standard of reference. Two readers used lumen boundary to determine scanner blur and then optimized component densities and subvoxel boundaries to best fit the observed image by using semiautomatic software. The accuracy of the resulting in vivo quantitation of calcification, lipid-rich necrotic core (LRNC), and matrix was assessed with statistical estimates of bias and linearity relative to ex vivo histologic findings. Reader variability was assessed with statistical estimates of repeatability and reproducibility. Results A total of 239 cross sections obtained with CT angiography and histologic examination were matched. Performance on held-out data showed low levels of bias and high Pearson correlation coefficients for calcification (-0.096 mm

Quantitative imaging biomarkers are being used increasingly in medicine to diagnose and monitor patients’ disease. The computer algorithms that measure quantitative imaging biomarkers have different technical performance characteristics. In this paper we illustrate the appropriate statistical methods for assessing and comparing the bias, precision, and agreement of computer algorithms. We use data from three studies of pulmonary nodules. The first study is a small phantom study used to illustrate metrics for assessing repeatability. The second study is a large phantom study allowing assessment of four algorithms’ bias and reproducibility for measuring tumor volume and the change in tumor volume. The third study is a small clinical study of patients whose tumors were measured on two occasions. This study allows a direct assessment of six algorithms’ performance for measuring tumor change. With these three examples we compare and contrast study designs and performance metrics, and we illustrate the advantages and limitations of various common statistical methods for quantitative imaging biomarker studies.

Quantitative biomarkers from medical images are becoming important tools for clinical diagnosis, staging, monitoring, treatment planning, and development of new therapies. While there is a rich history of the development of quantitative imaging biomarker (QIB) techniques, little attention has been paid to the validation and comparison of the computer algorithms that implement the QIB measurements. In this paper we provide a framework for QIB algorithm comparisons. We first review and compare various study designs, including designs with the true value (e.g. phantoms, digital reference images, and zero-change studies), designs with a reference standard (e.g. studies testing equivalence with a reference standard), and designs without a reference standard (e.g. agreement studies and studies of algorithm precision). The statistical methods for comparing QIB algorithms are then presented for various study types using both aggregate and disaggregate approaches. We propose a series of steps for establishing the performance of a QIB algorithm, identify limitations in the current statistical literature, and suggest future directions for research.

Medical imaging serves many roles in patient care and the drug approval process, including assessing treatment response and guiding treatment decisions. These roles often involve a quantitative imaging biomarker, an objectively measured characteristic of the underlying anatomic structure or biochemical process derived from medical images. Before a quantitative imaging biomarker is accepted for use in such roles, the imaging procedure to acquire it must undergo evaluation of its technical performance, which entails assessment of performance metrics such as repeatability and reproducibility of the quantitative imaging biomarker. Ideally, this evaluation will involve quantitative summaries of results from multiple studies to overcome limitations due to the typically small sample sizes of technical performance studies and/or to include a broader range of clinical settings and patient populations. This paper is a review of meta-analysis procedures for such an evaluation, including identification of suitable studies, statistical methodology to evaluate and summarize the performance metrics, and complete and transparent reporting of the results. This review addresses challenges typical of meta-analyses of technical performance, particularly small study sizes, which often causes violations of assumptions underlying standard meta-analysis techniques. Alternative approaches to address these difficulties are also presented; simulation studies indicate that they outperform standard techniques when some studies are small. The meta-analysis procedures presented are also applied to actual [18F]-fluorodeoxyglucose positron emission tomography (FDG-PET) test–retest repeatability data for illustrative purposes.

Quantitative imaging biomarkers are of particular interest in drug development for their potential to accelerate the drug development pipeline. The lack of consensus methods and carefully characterized performance hampers the widespread availability of these quantitative measures. A framework to support collaborative work on quantitative imaging biomarkers would entail advanced statistical techniques, the development of controlled vocabularies, and a service-oriented architecture for processing large image archives. Until now, this framework has not been developed. With the availability of tools for automatic ontology-based annotation of datasets, coupled with image archives, and a means for batch selection and processing of image and clinical data, imaging will go through a similar increase in capability analogous to what advanced genetic profiling techniques have brought to molecular biology. We report on our current progress on developing an informatics infrastructure to store, query, and retrieve imaging biomarker data across a wide range of resources in a semantically meaningful way that facilitates the collaborative development and validation of potential imaging biomarkers by many stakeholders. Specifically, we describe the semantic components of our system, QI-Bench, that are used to specify and support experimental activities for statistical validation in quantitative imaging.

A major initiative of the Quantitative Imaging Biomarker Alliance (QIBA) is to develop standards-based documents called “Profiles”, which describe one or more technical performance claims for a given imaging modality. The term “actor” denotes any entity (device, software, person) whose performance must meet certain specifications in order for the claim to be met. The objective of this paper is to present the statistical issues in testing actors’ conformance with the specifications. In particular, we present the general rationale and interpretation of the claims, the minimum requirements for testing whether an actor achieves the performance requirements, the study designs used for testing conformity, and the statistical analysis plan. We use three examples to illustrate the process: apparent diffusion coefficient (ADC) in solid tumors measured by MRI, change in Perc 15 as a biomarker for the progression of emphysema, and percent change in solid tumor volume by CT as a biomarker for lung cancer progression.

Rationale and Objectives Quantifying changes in lung tumor volume is important for diagnosis, therapy planning, and evaluation of response to therapy. The aim of this study was to assess the performance of multiple algorithms on a reference data set. The study was organized by the Quantitative Imaging Biomarker Alliance (QIBA). Materials and Methods The study was organized as a public challenge. Computed tomography scans of synthetic lung tumors in an anthropomorphic phantom were acquired by the Food and Drug Administration. Tumors varied in size, shape, and radiodensity. Participants applied their own semi-automated volume estimation algorithms that either did not allow or allowed post-segmentation correction (type 1 or 2, respectively). Statistical analysis of accuracy (percent bias) and precision (repeatability and reproducibility) was conducted across algorithms, as well as across nodule characteristics, slice thickness, and algorithm type. Results Eighty-four percent of volume measurements of QIBA-compliant tumors were within 15% of the true volume, ranging from 66% to 93% across algorithms, compared to 61% of volume measurements for all tumors (ranging from 37% to 84%). Algorithm type did not affect bias substantially; however, it was an important factor in measurement precision. Algorithm precision was notably better as tumor size increased, worse for irregularly shaped tumors, and on the average better for type 1 algorithms. Over all nodules meeting the QIBA Profile, precision, as measured by the repeatability coefficient, was 9.0% compared to 18.4% overall. Conclusion The results achieved in this study, using a heterogeneous set of measurement algorithms, support QIBA quantitative performance claims in terms of volume measurement repeatability for nodules meeting the QIBA Profile criteria.

OBJECTIVE: This study was designed to characterize the reproducibility of measurement for tumor volumes and their longest tumor diameters (LDs) and estimate the potential impact of using changes in tumor volumes instead of LDs as the basis for response assessments. METHODS: We studied patients with advanced lung cancer who have been observed longitudinally with x-ray computed tomography in a multinational trial. A total of 71 time points from 10 patients with 13 morphologically complex target lesions were analyzed. A total of 6461 volume measurements and their corresponding LDs were made by seven independent teams using their own work flows and image analysis tools. Interteam agreement and overall interrater concurrence were characterized. RESULTS: Interteam agreement between volume measurements was better than between LD measurements (i = 0.945 vs 0.734, P = .005). The variability in determining the nadir was lower for volumes than for LDs ( P = .005). Use of standard thresholds for the RECIST-based method and use of experimentally determined cutoffs for categorizing responses showed that volume measurements had a significantly greater sensitivity for detecting partial responses and disease progression. Earlier detection of progression would have led to earlier changes in patient management in most cases. CONCLUSIONS: Our findings indicate that measurement of changes in tumor volumes is adequately reproducible. Using tumor volumes as the basis for response assessments could have a positive impact on both patient management and clinical trials. More authoritative work to qualify or discard changes in volume as the basis for response assessments should proceed.

Quantitative imaging biomarkers could speed the development of new treatments for unmet medical needs and improve routine clinical care. However, it is not clear how the various regulatory and nonregulatory (eg, reimbursement) processes (often referred to as pathways) relate, nor is it clear which data need to be collected to support these different pathways most efficiently, given the time- and cost-intensive nature of doing so. The purpose of this article is to describe current thinking regarding these pathways emerging from diverse stakeholders interested and active in the definition, validation, and qualification of quantitative imaging biomarkers and to propose processes to facilitate the development and use of quantitative imaging biomarkers. A flexible framework is described that may be adapted for each imaging application, providing mechanisms that can be used to develop, assess, and evaluate relevant biomarkers. From this framework, processes can be mapped that would be applicable to both imaging product development and to quantitative imaging biomarker development aimed at increasing the effectiveness and availability of quantitative imaging.http://radiology.rsna.org/lookup/suppl/doi:10.1148/radiol.10100800/-/DC1.RSNA, 2011

Rationale and Objectives New ways to understand biology as well as increasing interest in personalized treatments requires new capabilities for the assessment of therapy response. The lack of consensus methods and qualification evidence needed for large-scale multicenter trials, and in turn the standardization that allows them, are widely acknowledged to be the limiting factor in the deployment of qualified imaging biomarkers. Materials and Methods The Quantitative Imaging Biomarker Alliance is organized to establish a methodology whereby multiple stakeholders collaborate. It has charged the Volumetric Computed Tomography (CT) Technical Subcommittee with investigating the technical feasibility and clinical value of quantifying changes over time in either volume or other parameters as biomarkers. The group selected solid tumors of the chest in subjects with lung cancer as its first case in point. Success is defined as sufficiently rigorous improvements in CT-based outcome measures to allow individual patients in clinical settings to switch treatments sooner if they are no longer responding to their current regimens, and reduce the costs of evaluating investigational new drugs to treat lung cancer. Results The team has completed a systems engineering analysis, has begun a roadmap of experimental groundwork, documented profile claims and protocols, and documented a process for imaging biomarker qualification as a general paradigm for qualifying other imaging biomarkers as well. Conclusion This report addresses a procedural template for the qualification of quantitative imaging biomarkers. This mechanism is cost-effective for stakeholders while simultaneously advancing the public health by promoting the use of measures that prove effective.