Search

Health Technology Assessment is one of the standard instruments in support of the decision-making to define the public health services both internationally and in the German health care system. Besides systematic reviews, benefit-harm-analyses, health economic evaluations, and, decision-analytic modelling, especially epidemiological and biometrical questions and methods play a key role. From this perspective discussions on increased European cooperation including calls for wider harmonization are attracting greater interests.The overall aim is, to present at the GMDS workshop relevant information on this emerging field of harmonization in Europe across similarities and differences in the HTA process. Current developments around the composition of the EU-HTA Network are provided. This network accepts the challenge to define and establish a “Joint Work” across Europe. Special emphasis was placed on the discussion on “Harmonization of HTA: is it a threat or does it mean support?”. Furthermore, methodological discussions and questions are being addressed: “Are Real World Data and Surrogates possible parameters for decision-making or HTA?”

Background: Progression-free survival (PFS) is often used instead of the patient-relevant endpoint overall survival (OS) in cancer clinical trials. In order for PFS to be accepted as a patient-relevant outcome within the benefit assessment of pharmaceuticals in accordance with the German Social Code, Book Five (SGB V), section 35a, it has to be validated as a surrogate endpoint for OS in the relevant indication. As part of a rapid report the Institute for Quality and Efficiency in Health Care (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen – IQWiG) presented methods for surrogate endpoints validation and recommendations for correlation-based procedures. These methods include the evaluation of the certainty of conclusion of study results and the correlation between estimates of surrogate outcome and patient-relevant outcome on trial-level. The correlation is estimated by sample Pearson correlation coefficient or coefficient of determination and respective confidence interval (CI). Requirements for surrogate validation are a high correlation and a high certainty of conclusion of the study results. In case of medium correlation IQWiG methods propose applying the concept of surrogate threshold effect (STE) to determine thresholds for the estimate of the surrogate endpoint.Methods: In simulation studies we investigate the requirements for a successful surrogate validation when applying a correlation-based approach. Simulation parameters are the estimates of the surrogate and the patient-relevant outcome, the correlation between them, the number of patients and the number of studies. We analyzed different scenarios in order to figure out parameters contributing to high correlation. Furthermore, we investigate requirements of the STE method, allowing conclusions on patient-relevant endpoints by means of surrogate endpoints. Finally, in consideration of IQWiG methods we analyze the challenges of surrogate validation in practical use.Results: Both, simulations of the surrogate validation using correlation-based procedure as well as an analytical derivation show low statistical power despite a medium-sized number of studies and a high true correlation. The power for =5 studies and correlation =0.9 is below 6%. A very high true correlation of =0.95 in at least =25 studies would be required in order to preserve a power of 80%, however this scenario is considered implausible in practice. Further simulations investigating the power of the method of STE showed that only one fifth of the considered scenarios have power above 80%. However, these scenarios included parameter constellations with impractical values regarding number of studies, number of patients and effect estimate of OS. The correlation parameter as well as the parameter of the estimate of PFS barely have an impact on the power of the STE procedure.Conclusion: Our simulations show that in practical use it is quite unlikely to fulfill the condition of high correlation as defined in the rapid report of IQWiG, proposing the lower limit of confidence interval to be crucial. Despite setting the true correlation in the model to a high value, statistical power will be quite small as long as the number of studies remains low or medium which is a realistic assumption in validation of surrogate endpoints within the framework of early benefit assessment. Besides, recommendation to involve certainty of studies in the analysis remains problematic. On closer inspection of the density function of sample correlation coefficient and assuming a given true correlation we can conclude that sample correlation does not depend on the variance of the single estimates but only on sample size (representing the number of studies in the model). Therefore, patient number does not have an impact on the confidence interval of the correlation whether using weight vectors for studies or not. Application of the STE concept according to the requirements described in the rapid report appears to be rather complicated as well. We propose an alternative solution of comparing the value of STE with point estimate of the surrogate endpoint instead of its lower level of confidence interval showing low α-errors in realistic scenarios.