Search

Invited discussion paper for "A Review of Some Sampling and Aggregation Strategies for Basic Statistical Process Monitoring" by Zwetsloot and Woodall.

Display Omitted Used data from12 national notifiable diseases and simulated alerting signals.Compared 4 detection methods by background alert rate, sensitivity, and timeliness.Long-term trend adjustment improved detection performance.The modified Historical Limits Method (HLM) outperformed the traditional HLM.The Farrington-like method provided the best detection performance overall. To compare the performance of the standard Historical Limits Method (HLM), with a modified HLM (MHLM), the Farrington-like Method (FLM), and the Serfling-like Method (SLM) in detecting simulated outbreak signals. We used weekly time series data from 12 infectious diseases from the U.S. Centers for Disease Control and Preventions National Notifiable Diseases Surveillance System (NNDSS). Data from 2006 to 2010 were used as baseline and from 2011 to 2014 were used to test the four detection methods. MHLM outperformed HLM in terms of background alert rate, sensitivity, and alerting delay. On average, SLM and FLM had higher sensitivity than MHLM. Among the four methods, the FLM had the highest sensitivity and lowest background alert rate and alerting delay. Revising or replacing the standard HLM may improve the performance of aberration detection for NNDSS standard weekly reports.

Objective Our project goal is to enhance the capability of automating health surveillance[MOU1] by US Department of Defense (DoD) epidemiologists. We employ software tools that build and train Bayesian networks (BNs) to facilitate the development of analytic fusion of multiple, disparate data sources comprising both syndromic and diagnostic data streams for rapid estimation of overall levels of concern for potential disease outbreaks. Working with previously developed heuristic BNs, we evaluate the ability of machine learning algorithms to detect outbreaks with greater accuracy. We use historical training data on the ability to detect outbreaks of influenza-like illness (ILI). Introduction The motivation for this project is to provide greater situational awareness to DoD epidemiologists monitoring the health of military personnel and their dependents. An increasing number of data sources of varying clinical specificity and timeliness are available to the staff. The challenge is to integrate all the information for a coherent, up-to-date view of population health. Developers at the Johns Hopkins Applied Physics Laboratory, in collaboration with medical epidemiologists at the Armed Forces Health Surveillance Branch, previously designed a multivariate decision support tool to add to the DoD implementation of the Electronic Surveillance System for Early Notification of Community-Based Epidemics (ESSENCE). Data sources included clinical encounter records including free-text chief complaints, filled prescription records, and laboratory test orders and results. Filtered data streams were derived from these sources for daily monitoring, and alerting algorithms were customized and applied to the resulting time series. We built BNs to derive overall levels of concern from the collection of data streams and algorithm outputs to derive, in the form of daily fusion alerts, the overall level of various outbreak concerns. Visualizations made apparent which data features accounted for these concerns, including drill-down to the level of patient record details. Advantages of the BN approach are this transparency and the capacity for assessments using incomplete data and incorporating novel and report-based data streams. The need for such fusion was nearly unanimous in a global survey of public health epidemiologists [1]. Our proof-of-concept system based on commercial BN software was well received by a cross-section of DoD health monitors. The new software tools we apply in this project use freely available R packages which provide more comprehensive tools for BN training and development. These results will allow us to improve the analytic fusion abilities of DoD ESSENCE, as well as in civilian surveillance systems Our testing procedures and results are presented below. Methods We employ a 3.75-year dataset (2006-2010) with information from 502 US medical treatment facilities including 289 hospitals. Our data include time series of daily counts and alerting algorithm outputs from each facility for syndrome groups based on a) chief complaints and diagnosis codes from clinic visits, b) groups of laboratory test orders and influenza test results, and c) selected groups of filled prescriptions. For each facility group, the challenge is to combine these data streams into a daily assessment of levels of concern for an ILI outbreak. The software developed in this project facilitates the formation, training, and testing of BNs for outbreak alerting based on the datasets above. Underlying each BN is a directed acyclic graph whose leaf nodes represent discrete states of concern for each data stream, ranging from general streams such as ILI-related chief complaints to specific ones such as positive flu test results. The states are derived from both daily stream counts and alerting algorithm outputs. Internal nodes represent mid-level combinations of indicators, such as ILI concern based only on clinic data, and parent nodes which represent the calculated level of concern based on all data sources. The connectivity between the nodes, and the orientation of the edges of the graph are determined by the heuristic relationships adopted in previous projects [2]. Each node is associated with a conditional probability table (CPT). Training a BN requires state assignments for every node in the underlying graph, which must be obtained from a trusted source. These assignments can generate CPTs at each node of the BN such that, when given only a set of evidence nodes, the levels of concern can be propagated up the network to provide the desired levels of concern at the decision nodes. Truth data for ILI outbreaks comes from two sources: 33 documented outbreaks with dates supplied by DoD surveillance reports or media articles and, because many modest-sized outbreaks are unreported, 81 unconfirmed data-derived events with algorithmic alerts across multiple data sources. We use data from the more numerous unconfirmed events to train the BNs. To avoid commercial software constraints, we use the free R package gRain [3] to create, train, and test the BNs. We test multiple BNs for multivariate ILI outbreak detection, all based on the same nodal structure with 18 parent, intermediate, and leaf nodes. Candidate BNs are created and trained using either a) CPTs determined with a multivariate stochastic search in the previous project, augmented with ground truth data or b) a heuristic lookup table of state combinations. Results Table 1 shows the high odds ratios calculated for candidate BNs. These statistics are calculated for decision node outputs for event vs non-event dates in the truth data with the constraint that every event is detected for at least one date. The machine learning advantage from the training data is evident from comparing the two rows. To show the advantage of fusing data sources, Table 2 gives analogous odds ratios based on single-stream alerting algorithms. Aside from the lower detection statistics, single streams offer no corroboration of statistical alerts. Conclusions Analytic fusion is essential for the efficient, timely use of a growing collection of complex, streaming information by a limited workforce of human health monitors. This project builds upon previous fusion capability for automated health surveillance by expediting new development and facilitating software implementation through open source tools. The detection results indicate a significant advantage in both sensitivity and alert rates of automated systems achievable with machine learning. Moreover, if basic challenges of multivariate data acquisition and determination of truth datasets for supervised learning can be met, further improvements are likely using BN structure discovery methods as well as other machine learning approaches. References 1. Hopkins RS, Tong CC, Burkom HS, et al. A Practitioner-Driven Research Agenda for Syndromic Surveillance. Public Health Rep. 2017;132(1_suppl):116s-126s. 2. Burkom H.S., Elbert Y., Ramac-Thomas L., Cuellar C., Hung V., Refinement of a Population-Based Bayesian Network for Fusion of Health Surveillance Data, Online Journal of Public Health Informatics, Vol 5, No 1 (2013), DOI: https://doi.org/10.5210/ojphi.v5i1.4413. 3. Hojsgaard S., Graphical Independence Networks with the gRain Package for R, Journal of Statistical Software, (2012) 46(10), DOI: 10.18637/jss.v046.i10.

Driven by the growing importance of situational awareness of bioterrorism threats, the Republic of Korea (ROK) and the United States have constructed a joint military capability, called the Biosurveillance Portal (BSP), to enhance biosecurity. As one component of the BSP, we developed the Military Active Real-time Syndromic Surveillance (MARSS) system to detect and track natural and deliberate disease outbreaks. This article describes the ROK military health data infrastructure and explains how syndromic data are derived and made available to epidemiologists. Queries corresponding to 8 syndromes, based on published clinical effects of weaponized pathogens, were used to classify military hospital patient records to form aggregated daily syndromic counts. A set of ICD-10 codes for each syndrome was defined through literature review and expert panel discussion. A study set of time series of national daily counts for each syndrome was extracted from the Defense Medical Statistical Information System between January 1, 2011, and May 31, 2014. A stratified, adjusted cumulative summation algorithm was implemented for each syndrome group to signal alerts prompting investigation. The algorithm was developed by calculating sensitivity to sets of 1,000 artificial outbreak signals randomly injected in the dataset, with each signal injected in a separate trial. Queries and visualizations were adapted from the Suite for Automated Global bioSurveillance. Findings indicated that early warning of outbreaks affecting fewer than 50 patients will require analysis at subnational levels, especially for common syndrome groups. Developing MARSS to improve sensitivity will require modification of underlying syndromic diagnosis codes, engineering to coordinate alerts among subdivisions, and enhanced algorithms. The bioterrorist threat in the Korean peninsula mandates these efforts.

Objective Our objective was to compare the effectiveness of applying the historical limits method (HLM) to poison center (PC) call volumes with vs without stratifying by exposure type. Introduction The Centers for Disease Control and Prevention (CDC) uses the National Poison Data System (NPDS) to conduct surveillance of calls to United States PCs. PCs provide triage and treatment advice for hazardous exposures through a free national hotline. Information on demographics, health effects, implicated substance(s), medical outcome of the patient, and other variables are collected. CDC uses automated algorithms to identify anomalies in both pure call volume and specific clinical effect volume, and to identify calls reporting exposure to high priority agents. Pure and clinical effect volume anomalies are identified when an hourly call count exceeds a threshold based on historical data using HLM. 1 Clinical toxicologists and epidemiologists at the American Association of Poison Control Centers and CDC apply standardized criteria to determine if the anomaly identifies a potential incident of public health significance (IPHS) and to notify the respective health departments and local PCs as needed. Discussions with NPDS users and analysis of IPHS showed that alerting based on pure call volume yielded excessive false positives. A study using a 5-year NPDS call dataset assessed the positive predictive value (PPV) of the call volume-based approach. This study showed that less than 4% of anomalies were IPHS. 2 A low PPV can cause unnecessary waste of staff time and resources analyzing false positive anomalies. As an alternative to pure call volume-based detection where all calls to each PC are aggregated for anomaly detection, we considered separating calls by toxicologically-relevant exposure categories for more targeted anomaly detection. We hypothesized that this stratified approach would reduce the number of false positives. Methods We derived our exposure categories based on the criteria that the categories must: 1) relate to hazardous exposures of public health importance, 2) reflect categories based on clinical effects and treatment modalities, 3) avoid high priority exposures that may be triggered by single calls, 4) be compatible with exposure substance identification codes currently used by PCs and NPDS, and 5) include enough calls for meaningful tracking. We queried all calls reporting exposures to the proposed categories between January 1, 2009 and July 31, 2015 for ten PCs. We applied the HLM method after stratifying by exposure category and tabulated the number of alerts triggered for each category during the study period. We then applied the HLM method for the ten PCs on all combined exposure calls to represent the traditional non-stratified approach. We compared the combined alert burden generated by stratifying by exposure category with the alert burden for the non-stratified approach for varying time windows (1-, 2-, 4-, 8- and 24-hours). We conducted analysis in R. Results We derived a total of 20 exposure categories, including chemicals (n=4), drugs of abuse (n=6), pesticides (n=3), gas/fume/vapors (n=2), contaminated food/water (n=1), and others (n=4). Call counts during 2015 for these categories ranged from approximately 5,000 to 90,000. Table 1 shows the total number of alerts triggered for each method by time windows. There was a marked reduction of alert burden when first stratifying by exposure category for time windows shorter than eight hours compared to the alert burden for the non-stratified approach. Conclusions Stratification of call volume by exposure category and time window suggests potential improvement over traditional non- stratified approach by having a lower alert burden. Further work should focus on refining the exposure categories, refining the time window for surveillance, and assessing other detection performance metrics, such as sensitivity. Table 1: Alert burden comparison for the non-stratified vs stratified approach

Analysis of available, potentially large data streams could have value in assisting risk identification efforts. We explored data from the American Society for the Prevention of Cruelty to Animals (ASPCA) Animal Poison Control Center hotline to identify weekly increases in: 1) call volume, 2) calls regarding certain intoxication products, 3) calls regarding certain species, and 4) calls with clinical signs associated with specific syndromes. Seasonal trends were observed in calls regarding pesticide intoxications. Human-centric product intoxication trends were observed in swine. These data support the value of passive data streams in risk identification, biosurveillance, and animal health monitoring.

Algorithms for identifying public health threats or disease outbreaks are vulnerable to false alarms arising from sudden shifts in health-care utilization or data participation. This paper describes a method of reducing false alerts in automated public health surveillance algorithms, and in particular, automated syndromic surveillance algorithms, that rely on health-care utilization data. The technique is based on monitoring syndromic counts with reference to a suitable background, or reference, series of counts. The suitability of the background time series in decreasing the false-alarm rate will be shown to be related mathematically to the so-called mutual information that exists between the random variables representing the syndromic and background time series of counts. The method can be understood as a noise cancellation filter technique in which one noisy (reference) channel is used to cancel the background noise of the monitored (measured) channel. The issues discussed here may also be relevant to the appropriate use of rates in epidemiology and biostatistics. Copyright © 2011 John Wiley & Sons, Ltd.

This paper discusses further advances in making robust predictions with the Holt-Winters forecasts for a variety of syndromic time series behaviors and introduces a control-chart detection approach based on these forecasts. Using three collections of time series data, we compare biosurveillance alerting methods with quantified measures of forecast agreement, signal sensitivity, and time-to-detect. The study presents practical rules for initialization and parameterization of biosurveillance time series. Several outbreak scenarios are used for detection comparison. We derive an alerting algorithm from forecasts using Holt-Winters-generalized smoothing for prospective application to daily syndromic time series. The derived algorithm is compared with simple control-chart adaptations and to more computationally intensive regression modeling methods. The comparisons are conducted on background data from both authentic and simulated data streams. Both types of background data include time series that vary widely by both mean value and cyclic or seasonal behavior. Plausible, simulated signals are added to the background data for detection performance testing at signal strengths calculated to be neither too easy nor too hard to separate the compared methods. Results show that both the sensitivity and the timeliness of the Holt-Winters-based algorithm proved to be comparable or superior to that of the more traditional prediction methods used for syndromic surveillance.

As a part of the Korea-US joint Biosurveillance Portal project, we have developed the military syndromic surveillance system with electronic medical records from military hospitals for early identification of bio-terror related events among Armed Forces in Korea. Respiratory, Gastrointestinal, Botulism, Dermatologic, Neurologic, Hemorrhagic and Fever syndromes were defined by different ICD-10 codes and their alert thresholds were developed based on the characteristics of time series derived using daily counts of ICD-10 codes for each syndrome.

The presented method is a practical, understandable way to monitor single care facilities for chief complaint clusters of concern based on unusually high occurrence of rare or common terms that need not be related to syndromes. Routine implementation requires a human monitor to inspect the relevant CCs make follow-up decisions. Using 7 years of patient records from 15 hospitals, our approach pools CCs into contiguous time blocks and uses a statistical hypothesis test to seek current terms that are anomalous relative to their occurrence in a large sliding baseline. Sets of anomalous terms are then presented for further investigation.

Researchers working on the Department of Defense Global Emerging Infections System (DoD-GEIS) pilot system, the Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE), have applied scan statistics for early outbreak detection using both traditional and nontraditional data sources. These sources include medical data indexed by International Classification of Disease, 9th Revision (ICD-9) diagnosis codes, as well as less-specific, but potentially timelier, indicators such as records of over-the-counter remedy sales and of school absenteeism. Early efforts employed the Kulldorff scan statistic as implemented in the SaTScan software of the National Cancer Institute. A key obstacle to this application is that the input data streams are typically based on time-varying factors, such as consumer behavior, rather than simply on the populations of the component subregions. We have used both modeling and recent historical data distributions to obtain background spatial distributions. Data analyses have provided guidance on how to condition and model input data to avoid excessive clustering. We have used this methodology in combining data sources for both retrospective studies of known outbreaks and surveillance of high-profile events of concern to local public health authorities. We have integrated the scan statistic capability into a Microsoft Access-based system in which we may include or exclude data sources, vary time windows separately for different data sources, censor data from subsets of individual providers or subregions, adjust the background computation method, and run retrospective or simulated studies.

The on-site beta test of system upgrade to the Early Notification of community-based Epidemics (ESSENCE) allowed user feedback to the development team to aid in the evaluation of an analytic combination of weighted clinical and syndromic evidence, known as fusion, and direct the completion of the upgrade prior to launch. The analytic fusion capability was improved particularly in the area of severity-related indicator data, improving algorithms, and user interfaces.

To manage an increasingly complex data environment, a fusion module based on Bayesian networks (BN) was developed for the Dept. of Defense (DoD) Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE). Subsequent efforts have produced a full fusion-enabled version of ESSENCE for beta testing and further upgrades. The current presentation describes advances to formalize the network training, calibrate the component alerting algorithms and decision nodes together, and implement a validation strategy. A cross-validation strategy produced consistent threshold combinations yielding 88% sensitivity from reported events, a 10-15% improvement over the original demonstration module.

Objective We examined the utility of combining surveillance data from the Departments of Defense (DoD) and Veterans Affairs (VA) for spatial cluster detection. Introduction The Joint VA/DoD BioSurveillance System for Emerging Biological Threats project seeks to improve situational awareness of the health of VA/DoD populations by combining their respective data. Each system uses a version of the Electronic Surveillance System for Early Notification of Community-Based Epidemics (ESSENCE); a combined version is being tested. The current effort investigated combining the datasets for disease cluster detection. We compared results of retrospective cluster detection studies using both separate and joined data. — Does combining datasets worsen the rate of background cluster determination? — Does combining mask clusters detected on the separate datasets? — Does combining find clusters that the separate datasets alone would miss? Methods Cluster determination runs were done with a spatial scan statistics implementation previously verified [1] by comparison with SaTScan software [2] using DoD data from the Biosense system. Input data files were extracted from a repository of outpatient records from both DoD and VA facilities covering 4 years beginning Jan. 1, 2007. This repository includes over 37 million DoD records and over 86 million VA records. Input files were matrices of daily Influenza-like-Illness (ILI) or gastrointestinal (GI) visit counts. Matrix rows were consecutive days, columns were patient residence zip codes, and entry (i, j) was the number of visits on day i from with zip code j. These files were made for DoD data, VA data, and combined data. For assessing the alerting burden from combining datasets, three sets of runs were executed using data from three regions, Baltimore/Washington D.C. (dominated by DoD data), Los Angeles (mainly VA data), and Tampa (representation of both). For each region, sets of 1672 daily runs were executed for ILI and GI syndrome data. Lastly, focused runs were done to investigate known outbreaks in New York (GI, Jan–Mar 2010), San Diego (ILI, Dec 2007–Apr 2008 and Fall 2009), and New Jersey (GI, Jan–Mar 2010). Results Combining the data sources increased the rate of significant cluster alerting by a manageable 1–10% across run sets. Some clusters found only when the data were combined persisted over several days and may have indicated small events not reported in either system; however, we were unable to validate minor events that may have occurred in past years. Retrospective looks at known outbreaks were successful in that clustering evidence found in separate DoD and VA runs persisted when data sets were combined. For the New York run, a West Point outbreak was seen in repeated clusters of combined data, beginning days before the event report. However, clustering did not consistently produce alerts before outbreak report dates. In the New Jersey DoD runs, repeated clusters indicated a 10-week GI outbreak at Fort Dix; adding VA data that dominated the record counts gave the same clusters with no added cases, so the DoD event was probably self-contained. The San Diego runs were aimed at detecting unusually severe influenza epidemics in February 2008 and in the fall of 2009, and numerous clusters were found but did not enhance regional disease tracking. Conclusions From the analysis, combining DoD and VA data enhances cluster detection capability without loss of sensitivity to events isolated in either population and with manageable effect on the customary alert rate. For cluster detection, there may be many geographic regions where a health monitor in one of the systems would benefit from combined data. More detailed outbreak information is needed to quantify the timeliness/sensitivity advantages of combining datasets. In events examined, clustering itself yielded an occasional but not consistent timeliness advantage.

Objective For a multi-source decision support application, we sought to match univariate alerting algorithms to surveillance data types to optimize detection performance. Introduction Temporal alerting algorithms commonly used in syndromic surveillance systems are often adjusted for data features such as cyclic behavior but are subject to overfitting or misspecification errors when applied indiscriminately. In a project for the Armed Forces Health Surveillance Center to enable multivariate decision support, we obtained 4.5 years of out-patient, prescription and laboratory test records from all US military treatment facilities. A proof-of-concept project phase produced 16 events with multiple evidence corroboration for comparison of alerting algorithms for detection performance. We used the representative streams from each data source to compare sensitivity of 6 algorithms to injected spikes, and we used all data streams from 16 known events to compare them for detection timeliness. Methods The six methods compared were: Holt-Winters generalized exponential smoothing method (1)automated choice between daily methods, regression and an exponential weighted moving average (2)adaptive daily Shewhart-type chartadaptive one-sided daily CUSUMEWMA applied to 7-day means with a trend correction; and7-day temporal scan statistic Sensitivity testing: We conducted comparative sensitivity testing for categories of time series with similar scales and seasonal behavior. We added multiples of the standard deviation of each time series as single-day injects in separate algorithm runs. For each candidate method, we then used as a sensitivity measure the proportion of these runs for which the output of each algorithm was below alerting thresholds estimated empirically for each algorithm using simulated data streams. We identified the algorithm(s) whose sensitivity was most consistently high for each data category. For each syndromic query applied to each data source (outpatient, lab test orders, and prescriptions), 502 authentic time series were derived, one for each reporting treatment facility. Data categories were selected in order to group time series with similar expected algorithm performance: Median > 100 < Median ≤ 10Median = 0Lag 7 Autocorrelation Coefficient ≥ 0.2Lag 7 Autocorrelation Coefficient < 0.2 Timeliness testing: For the timeliness testing, we avoided artificiality of simulated signals by measuring alerting detection delays in the 16 corroborated outbreaks. The multiple time series from these events gave a total of 141 time series with outbreak intervals for timeliness testing. The following measures were computed to quantify timeliness of detection: Median Detection Delay – median number of days to detect the outbreak.Penalized Mean Detection Delay –mean number of days to detect the outbreak with outbreak misses penalized as 1 day plus the maximum detection time. Results Based on the injection results, the Holt-Winters algorithm was most sensitive among time series with positive medians. The adaptive CUSUM and the Shewhart methods were most sensitive for data streams with median zero. Table 1 provides timeliness results using the 141 outbreak-associated streams on sparse (Median=0) and non-sparse data categories. [Insert table #1 here] Data median Detection Delay, days Holt-winters Regression EWMA Adaptive Shewhart Adaptive CUSUM 7-day Trend-adj. EWMA 7-day Temporal Scan Median 0 Median 3 2 4 2 4.5 2 Penalized Mean 7.2 7 6.6 6.2 7.3 7.6 Median >0 Median 2 2 2.5 2 6 4 Penalized Mean 6.1 7 7.2 7.1 7.7 6.6 The gray shading in the table 1 indicates methods with shortest detection delays for sparse and non-sparse data streams. The Holt-Winters method was again superior for non-sparse data. For data with median=0, the adaptive CUSUM was superior for a daily false alarm probability of 0.01, but the Shewhart method was timelier for more liberal thresholds. Conclusions Both kinds of detection performance analysis showed the method based on Holt-Winters exponential smoothing superior on non-sparse time series with day-of-week effects. The adaptive CUSUM and She-whart methods proved optimal on sparse data and data without weekly patterns.

Objective The project involves analytic combination of multiple evidence sources to monitor health at hundreds of care facilities. A demonstration module featuring a population-based Bayes Network [1] was refined and expanded for application in the Department of Defense Electronic Surveillance System for Community-Based Epidemics (ESSENCE). Introduction The ESSENCE demonstration module was built to help DoD health monitors make routine decisions based on disparate evidence sources such as daily counts of ILI-related chief complaints, ratios of positive lab tests for influenza, patient age distribution, and counts of antiviral prescriptions [1]. The module was a population-based (rather than individual-based) Bayesian network (PBN) in that inputs were algorithmic results from these multiple aggregate data streams, and output was the degree of belief that the combined evidence required investigation. The module reduced total alerts substantially and retained sensitivity to the majority of documented outbreaks while clarifying underlying sources of evidence. The current effort was to advance the prototype to production by refining components of the fusion methodology to improve sensitivity while retaining the reduced alert rate. Methods The multi-level approach to sensitivity improvement included expanded syndromic queries, more data-sensitive algorithm selection, improved transformation of algorithm outputs to alert states, and hierarchical training of Bayesian networks. Components were tested individually, and the net result was iteratively refined with performance using documented outbreaks. We examined time series of classes of prescribed drugs and laboratory tests during known events and discussed outbreak-associated elements with domain experts to liberalize data queries. Algorithms were matched to data streams with injection testing applied to 4.5 years of data from 502 outpatient clinics. A hierarchical approach was applied for improved training and verification of PBNs for events related to categories of Influenza-like Illness, Gastrointestinal, Fever, Neurological, and Rash, chosen both for public health importance and for availability of multiple supporting data types. Hierarchical, modular training was applied to common subnetworks, such as a severity indicator PBN depending on case disposition, acute case indicators, complex evaluation/management codes, and patient bounce-backs, depicted in Figure 1. Conversion of individual algorithm outputs to belief states (e.g. “at least two red alerts/past 7 days”) was broadened using analysis of lags between data sources. With data from the known events, we calculated decision support thresholds for the parent-level PBN decision nodes with a stochastic optimization technique maximizing the ratio of alert rates during outbreak to non-outbreak periods. Results The expanded data queries, more stream-specific algorithm selection, generalized state transformation, and hierarchical PBN training detected 22 of an expanded collection of 24 documented outbreaks, with incremental improvement ongoing. The mean alert rate drop achieved by the Bayes Net was 87% (minimum of 85%) compared to the combined alerts of all component algorithms across syndromes and facilities. Conclusions Expansion and further technical validation upheld the PBN approach as a user-friendly means of analytic decision support given multiple, variably weighted evidence sources. The PBN affords not only sharply reduced alerting, but also transparent indication of evidence underlying each alert. The older algorithm approach remains available as backup. Beta testing of the resulting production system will drive further modification.

This study introduces new information fusion algorithms to enhance disease surveillance systems with Bayesian decision support capabilities. A detection system was built and tested using chief complaints from emergency department visits, International Classification of Diseases Revision 9 (ICD-9) codes from records of outpatient visits to civilian and military facilities, and influenza surveillance data from health departments in the National Capital Region (NCR). Data anomalies were identified and distribution of time offsets between events in the multiple data streams were established. The Bayesian Network was built to fuse data from multiple sources and identify influenza-like epidemiologically relevant events. Results showed increased specificity compared with the alerts generated by temporal anomaly detection algorithms currently deployed by NCR health departments. Further research should be done to investigate correlations between data sources for efficient fusion of the collected data.

Objective: Broadly, this research aims to improve the outbreak detection performance and, therefore, the cost effectiveness of automated syndromic surveillance systems by building novel, recombinant temporal aberration detection algorithms from components of previously developed detectors. Methods: This study decomposes existing temporal aberration detection algorithms into two sequential stages and investigates the individual impact of each stage on outbreak detection performance. The data forecasting stage (Stage 1) generates predictions of time series values a certain number of time steps in the future based on historical data. The anomaly measure stage (Stage 2) compares features of this prediction to corresponding features of the actual time series to compute a statistical anomaly measure. A Monte Carlo simulation procedure is then used to examine the recombinant algorithms' ability to detect synthetic aberrations injected into authentic syndromic time series. Results: New methods obtained with procedural components of published, sometimes widely used, algorithms were compared to the known methods using authentic datasets with plausible stochastic injected signals. Performance improvements were found for some of the recombinant methods, and these improvements were consistent over a range of data types, outbreak types, and outbreak sizes. For gradual outbreaks, the WEWD MovAvg7+WEWD Z-Score recombinant algorithm performed best; for sudden outbreaks, the HW+WEWD Z-Score performed best. Conclusion: This decomposition was found not only to yield valuable insight into the effects of the aberration detection algorithms but also to produce novel combinations of data forecasters and anomaly measures with enhanced detection performance.

This study decomposes existing temporal aberration detection algorithms into two, sequential stages and investigates the individual impact of each stage on outbreak detection performance. The data forecasting stage (stage 1) generates a prediction of the value of the time series a certain number of time steps in the future based on historical data. The anomaly measure stage (stage 2) compares one or more features of this prediction to the actual time series to compute a measure of the potential anomaly. This decomposition was found not only to yield valuable insight into the effects of the aberration detection algorithms but also to produce novel combinations of data forecasters and anomaly measures with enhanced detection performance.

This study presents and applies a methodology for selecting anomaly detection algorithms for biosurveillance time series data. The study employs both an authentic dataset and a simulated dataset which are freely available for replication of the results presented and for extended analysis. Using this approach, a public health monitor may choose algorithms that will be suited to the scale and behavior of the data of interest based on the calculation of simple discriminants from a limited sample. The tabular classification of typical time series behaviors using these discriminants is achieved using the ROC approach of detection theory, with realistic, stochastic, simulated signals injected into the data. The study catalogues the detection performance of 6 algorithms across data types and shows that for practical alert rates, sensitivity gains of 20% and higher may be achieved by appropriate algorithm selection.

Publisher Summary This chapter discusses hospital syndromic surveillance and reviews the aggregation of diagnosis-coded and the chief-complaint data into syndrome group categories to increase the sensitivity to an unknown data signal. Syndromic surveillance is a new and rapidly evolving approach to detecting important changes in the community health status as early as possible. This surveillance depends upon the identification of opportunities to use data in new ways as well as on the identification of novel data sources and statistical methods, and it is intended to enhance the traditional surveillance systems already in place, such as the traditional notifiable disease surveillance and the anecdotal surveillance carried out by the astute clinician. Such surveillance is valuable at the hospital level, but it is more useful at the community level. Therefore, it is critical to share this type of information with officials in public health to meet the mutually held goals and to develop agreements to determine how the information will be shared and utilized.

Background Surveillance of univariate syndromic data as a means of potential indicator of developing public health conditions has been used extensively. This paper aims to improve the performance of detecting outbreaks by using a background forecasting algorithm based on the adaptive recursive least squares method combined with a novel treatment of the Day of the Week effect. Methods Previous work by the first author has suggested that univariate recursive least squares analysis of syndromic data can be used to characterize the background upon which a prediction and detection component of a biosurvellance system may be built. An adaptive implementation is used to deal with data non-stationarity. In this paper we develop and implement the RLS method for background estimation of univariate data. The distinctly dissimilar distribution of data for different days of the week, however, can affect filter implementations adversely, and so a novel procedure based on linear transformations of the sorted values of the daily counts is introduced. Seven-days ahead daily predicted counts are used as background estimates. A signal injection procedure is used to examine the integrated algorithm's ability to detect synthetic anomalies in real syndromic time series. We compare the method to a baseline CDC forecasting algorithm known as the W2 method. Results We present detection results in the form of Receiver Operating Characteristic curve values for four different injected signal to noise ratios using 16 sets of syndromic data. We find improvements in the false alarm probabilities when compared to the baseline W2 background forecasts. Conclusion The current paper introduces a prediction approach for city-level biosurveillance data streams such as time series of outpatient clinic visits and sales of over-the-counter remedies. This approach uses RLS filters modified by a correction for the weekly patterns often seen in these data series, and a threshold detection algorithm from the residuals of the RLS forecasts. We compare the detection performance of this algorithm to the W2 method recently implemented at CDC. The modified RLS method gives consistently better sensitivity at multiple background alert rates, and we recommend that it should be considered for routine application in bio-surveillance systems.

Background The Centers for Disease Control and Prevention's (CDC's) BioSense system provides near-real time situational awareness for public health monitoring through analysis of electronic health data. Determination of anomalous spatial and temporal disease clusters is a crucial part of the daily disease monitoring task. Our study focused on finding useful anomalies at manageable alert rates according to available BioSense data history. Methods The study dataset included more than 3 years of daily counts of military outpatient clinic visits for respiratory and rash syndrome groupings. We applied four spatial estimation methods in implementations of space-time scan statistics cross-checked in Matlab and C. We compared the utility of these methods according to the resultant background cluster rate (a false alarm surrogate) and sensitivity to injected cluster signals. The comparison runs used a spatial resolution based on the facility zip code in the patient record and a finer resolution based on the residence zip code. Results Simple estimation methods that account for day-of-week (DOW) data patterns yielded a clear advantage both in background cluster rate and in signal sensitivity. A 28-day baseline gave the most robust results for this estimation; the preferred baseline is long enough to remove daily fluctuations but short enough to reflect recent disease trends and data representation. Background cluster rates were lower for the rash syndrome counts than for the respiratory counts, likely because of seasonality and the large scale of the respiratory counts. Conclusion The spatial estimation method should be chosen according to characteristics of the selected data streams. In this dataset with strong day-of-week effects, the overall best detection performance was achieved using subregion averages over a 28-day baseline stratified by weekday or weekend/holiday behavior. Changing the estimation method for particular scenarios involving different spatial resolution or other syndromes can yield further improvement.