Search

Additional file 1: Supplementary Methods. Table S1. DIC scores of competing seasonality components. Table S2. ICD-9-CM-coded respiratory hospitalizations in the entire population and RSV hospitalizations among children

To quantify the potential impact of rotavirus vaccines and identify strategies to improve vaccine performance in Bangladesh, a better understanding of the drivers of pre-vaccination rotavirus patterns is required. We developed and fit mathematical models to 23 years (1990–2012) of weekly rotavirus surveillance data from Dhaka with and without incorporating long-term and seasonal variation in the birth rate and meteorological factors. We performed external model validation using data between 2013 and 2019 from the regions of Dhaka and Matlab. The models showed good agreement with the observed age distribution of rotavirus cases and captured the observed shift in seasonal patterns of rotavirus hospitalizations from biannual to annual peaks. The declining long-term trend in the birth rate in Bangladesh was the key driver of the observed shift from biannual to annual winter rotavirus patterns. Meteorological indices were also important: a 1 °C, 1% and 1 mm increase in diurnal temperature range, surface water presence and degree of wetness were associated with a 19%, 3.9% and 0.6% increase in the transmission rate, respectively. The model demonstrated reasonable predictions for both Dhaka and Matlab, and can be used to evaluate the impact of rotavirus vaccination in Bangladesh against changing patterns of disease incidence.

Funder: Public Health Research Programme; Id: http://dx.doi.org/10.13039/501100001921, Decisions about typhoid fever prevention and control are based on estimates of typhoid incidence and their uncertainty. Lack of specific clinical diagnostic criteria, poorly sensitive diagnostic tests, and scarcity of accurate and complete datasets contribute to difficulties in calculating age-specific population-level typhoid incidence. Using data from the Strategic Typhoid Alliance across Africa and Asia program, we integrated demographic censuses, healthcare utilization surveys, facility-based surveillance, and serological surveillance from Malawi, Nepal, and Bangladesh to account for under-detection of cases. We developed a Bayesian approach that adjusts the count of reported blood-culture-positive cases for blood culture detection, blood culture collection, and healthcare seeking-and how these factors vary by age-while combining information from prior published studies. We validated the model using simulated data. The ratio of observed to adjusted incidence rates was 7.7 (95% credible interval [CrI]: 6.0-12.4) in Malawi, 14.4 (95% CrI: 9.3-24.9) in Nepal, and 7.0 (95% CrI: 5.6-9.2) in Bangladesh. The probability of blood culture collection led to the largest adjustment in Malawi, while the probability of seeking healthcare contributed the most in Nepal and Bangladesh; adjustment factors varied by age. Adjusted incidence rates were within or below the seroincidence rate limits of typhoid infection. Estimates of blood-culture-confirmed typhoid fever without these adjustments results in considerable underestimation of the true incidence of typhoid fever. Our approach allows each phase of the reporting process to be synthesized to estimate the adjusted incidence of typhoid fever while correctly characterizing uncertainty, which can inform decision-making for typhoid prevention and control.

Individual-based models (IBMs) informing public health policy should be calibrated to data and provide estimates of uncertainty. Two main components of model-calibration methods are the parameter-search strategy and the goodness-of-fit (GOF) measure; many options exist for each of these. This review provides an overview of calibration methods used in IBMs modelling infectious disease spread. We identified articles on PubMed employing simulation-based methods to calibrate IBMs informing public health policy in HIV, tuberculosis, and malaria epidemiology published between 1 January 2013 and 31 December 2018. Articles were included if models stored individual-specific information, and calibration involved comparing model output to population-level targets. We extracted information on parameter-search strategies, GOF measures, and model validation. The PubMed search identified 653 candidate articles, of which 84 met the review criteria. Of the included articles, 40 (48%) combined a quantitative GOF measure with an algorithmic parameter-search strategy-either an optimisation algorithm (14/40) or a sampling algorithm (26/40). These 40 articles varied widely in their choices of parameter-search strategies and GOF measures. For the remaining 44 (52%) articles, the parameter-search strategy could either not be identified (32/44) or was described as an informal, non-reproducible method (12/44). Of these 44 articles, the majority (25/44) were unclear about the GOF measure used; of the rest, only five quantitatively evaluated GOF. Only a minority of the included articles, 14 (17%) provided a rationale for their choice of model-calibration method. Model validation was reported in 31 (37%) articles. Reporting on calibration methods is far from optimal in epidemiological modelling studies of HIV, malaria and TB transmission dynamics. The adoption of better documented, algorithmic calibration methods could improve both reproducibility and the quality of inference in model-based epidemiology. There is a need for research comparing the performance of calibration methods to inform decisions about the parameter-search strategies and GOF measures. Author summary Calibration-that is, "fitting" the model to data-is a crucial part of using mathematical models to better forecast and control the population-level spread of infectious diseases. Evidence that the mathematical model is well-calibrated improves confidence that the model provides a realistic picture of the consequences of health policy decisions. To make informed decisions, Policymakers need information about uncertainty: i.e., what is the range of likely outcomes (rather than just a single prediction). Thus, modellers should also strive to provide accurate measurements of uncertainty, both for their model parameters and for their predictions. This systematic review provides an overview of the methods used to calibrate individual-based models (IBMs) of the spread of HIV, malaria, and tuberculosis. We found that less than half of the reviewed articles used reproducible, non-subjective calibration methods. For the remaining articles, the method could either not be identified or was described as an informal, non-reproducible method. Only one-third of the articles obtained estimates of parameter uncertainty. We conclude that the adoption of better-documented, algorithmic calibration methods could improve both reproducibility and the quality of inference in model-based epidemiology. WD was supported by grant 12L5816N from the Research Foundation - Flanders (FWO). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Hazelbag, CM (corresponding author), Stellenbosch Univ, South African DSI NRF Ctr Excellence Epidemiol Mo, Stellenbosch, South Africa. marijnhazelbag@sun.ac.za

For practical reasons, many forecasts of case, hospitalization, and death counts in the context of the current Coronavirus Disease 2019 (COVID-19) pandemic are issued in the form of central predictive intervals at various levels. This is also the case for the forecasts collected in the COVID-19 Forecast Hub (https://covid19forecasthub.org/). Forecast evaluation metrics like the logarithmic score, which has been applied in several infectious disease forecasting challenges, are then not available as they require full predictive distributions. This article provides an overview of how established methods for the evaluation of quantile and interval forecasts can be applied to epidemic forecasts in this format. Specifically, we discuss the computation and interpretation of the weighted interval score, which is a proper score that approximates the continuous ranked probability score. It can be interpreted as a generalization of the absolute error to probabilistic forecasts and allows for a decomposition into a measure of sharpness and penalties for over- and underprediction.