Search

The ability to produce timely and accurate flu forecasts in the United States can significantly impact public health. Augmenting forecasts with internet data has shown promise for improving forecast accuracy and timeliness in controlled settings, but results in practice are less convincing, as models augmented with internet data have not consistently outperformed models without internet data. In this paper, we perform a controlled experiment, taking into account data backfill, to improve clarity on the benefits and limitations of augmenting an already good flu forecasting model with internet-based nowcasts. Our results show that a good flu forecasting model can benefit from the augmentation of internet-based nowcasts in practice for all considered public health-relevant forecasting targets. The degree of forecast improvement due to nowcasting, however, is uneven across forecasting targets, with short-term forecasting targets seeing the largest improvements and seasonal targets such as the peak timing and intensity seeing relatively marginal improvements. The uneven forecasting improvements across targets hold even when “perfect” nowcasts are used. These findings suggest that further improvements to flu forecasting, particularly seasonal targets, will need to derive from other, non-nowcasting approaches. [ABSTRACT FROM AUTHOR]

Human networks of sexual contacts are dynamic by nature, with partnerships forming and breaking continuously over time. Sexual behaviours are also highly heterogeneous, so that the number of partners reported by individuals over a given period of time is typically distributed as a power-law. Both the dynamism and heterogeneity of sexual partnerships are likely to have an effect in the patterns of spread of sexually transmitted diseases. To represent these two fundamental properties of sexual networks, we developed a stochastic process of dynamic partnership formation and dissolution, which results in power-law numbers of partners over time. Model parameters can be set to produce realistic conditions in terms of the exponent of the power-law distribution, of the number of individuals without relationships and of the average duration of relationships. Using an outbreak of antibiotic resistant gonorrhoea amongst men have sex with men as a case study, we show that our realistic dynamic network exhibits different properties compared to the frequently used static networks or homogeneous mixing models. We also consider an approximation to our dynamic network model in terms of a much simpler branching process. We estimate the parameters of the generation time distribution and offspring distribution which can be used for example in the context of outbreak reconstruction based on genomic data. Finally, we investigate the impact of a range of interventions against gonorrhoea, including increased condom use, more frequent screening and immunisation, concluding that the latter shows great promise to reduce the burden of gonorrhoea, even if the vaccine was only partially effective or applied to only a random subset of the population. [ABSTRACT FROM AUTHOR]

Real-time forecasts based on mathematical models can inform critical decision-making during infectious disease outbreaks. Yet, epidemic forecasts are rarely evaluated during or after the event, and there is little guidance on the best metrics for assessment. Here, we propose an evaluation approach that disentangles different components of forecasting ability using metrics that separately assess the calibration, sharpness and bias of forecasts. This makes it possible to assess not just how close a forecast was to reality but also how well uncertainty has been quantified. We used this approach to analyse the performance of weekly forecasts we generated in real time for Western Area, Sierra Leone, during the 2013–16 Ebola epidemic in West Africa. We investigated a range of forecast model variants based on the model fits generated at the time with a semi-mechanistic model, and found that good probabilistic calibration was achievable at short time horizons of one or two weeks ahead but model predictions were increasingly unreliable at longer forecasting horizons. This suggests that forecasts may have been of good enough quality to inform decision making based on predictions a few weeks ahead of time but not longer, reflecting the high level of uncertainty in the processes driving the trajectory of the epidemic. Comparing forecasts based on the semi-mechanistic model to simpler null models showed that the best semi-mechanistic model variant performed better than the null models with respect to probabilistic calibration, and that this would have been identified from the earliest stages of the outbreak. As forecasts become a routine part of the toolkit in public health, standards for evaluation of performance will be important for assessing quality and improving credibility of mathematical models, and for elucidating difficulties and trade-offs when aiming to make the most useful and reliable forecasts. [ABSTRACT FROM AUTHOR]