Search

Background Heart failure and sleep-disordered breathing have been increasingly recognized as co-occurring conditions. Their bidirectional relationship warrants investigation into whether heart failure therapy improves sleep and sleep-disordered breathing. We sought to explore the effect of treatment with sacubitril/valsartan on sleep-related endpoints from the AWAKE-HF study. Methods and Results AWAKE-HF was a randomized, double-blind study conducted in 23 centers in the United States. Study participants with heart failure with reduced rejection fraction and New York Heart Association class II or III symptoms were randomly assigned to receive treatment with either sacubitril/valsartan or enalapril. All endpoints were assessed at baseline and after 8 weeks of treatment. Portable sleep-monitoring equipment was used to measure the apnea-hypopnea index, including obstructive and central events. Total sleep time, wake after sleep onset and sleep efficiency were exploratory measures assessed using wrist actigraphy. The results were as follows 140 patients received treatment in the double-blind phase (sacubitril/valsartan, n = 70; enalapril, n = 70). At baseline, 39% and 40% of patients randomly assigned to receive sacubitril/valsartan or enalapril, respectively, presented with undiagnosed, untreated, moderate-to-severe sleep-disordered breathing (≥ 15 events/h), and nearly all had obstructive sleep apnea. After 8 weeks of treatment, the mean 4% apnea-hypopnea index changed minimally from 16.3/h to 15.2/h in the sacubitril/valsartan group and from 16.8/h to 17.6/h in the enalapril group. Mean total sleep time was long at baseline and decreased only slightly in both treatment groups at week 8 (–14 and –11 minutes for sacubitril/valsartan and enalapril, respectively), with small changes in wake after sleep onset and sleep efficiency in both groups. Conclusions In a cohort of patients with heart failure with reduced rejection fraction who met prescribing guidelines for sacubitril/valsartan, one-third had undiagnosed moderate-to-severe obstructive sleep apnea. The addition of sacubitril/valsartan therapy did not significantly improve sleep-disordered breathing or sleep duration or efficiency. Patients who meet indications for treatment with sacubitril/valsartan should be evaluated for sleep-disordered breathing.

Context: There is an ongoing established need to develop engaging pain assessment strategies to provide more effective individualized care to pediatric patients with serious illnesses. This study explores the acceptability of wireless devices as one option. Objective: To evaluate the ability of wrist-wearable technology to collect physiological data from children with serious illnesses. Methods: Single-site prospective observational study conducted between September 2017 and September 2018 at Rady Children's Hospital, San Diego, California, inpatient wards. Pediatric patients with diagnoses of cancer and sickle cell disease admitted to the hospital for acute-on-chronic pain and taking opioid pain medications were asked to complete two 24-hour continuous monitoring periods with the Empatica E4 wristband. Results: Data collected from the device correlated with manually obtained vital signs. Children responded favorably to wearing the device. Participants with reported subjective pain versus no pain had average heart rate increased by 16.4 bpm, skin temperature decreased by 3.5°C, and electrodermal activity decreased by 0.27. Conclusions: This study shows the possibility of collecting continuous biophysical data in a nonobtrusive manner in seriously ill children experiencing acute-on-chronic pain using wearable devices. It provides the framework for larger studies to explore the utility of such data in relation to metrics of pain and suffering in this patient population.

The use of information and communication technology (ICT) in medical and healthcare services goes beyond everyday life. Ex pectations of a new medical environment, not previously experienced by ICT, exist in the near future. In particular, chronic meta bolic diseases such as diabetes and obesity, have a high prevalence and high social and economic burden. In addition, the continu ous evaluation and monitoring of daily life is important for effective treatment and management. Therefore, the wide use of ICT-based digital health systems is required for the treatment and management of these diseases. In this article, we compiled a variety of digital health technologies introduced to date in the field of diabetes and metabolic diseases.

Background Screening for asymptomatic, undiagnosed atrial fibrillation (AF) has the potential to allow earlier treatment, possibly resulting in prevention of strokes, but also to increase medical resource utilization. Objective To compare healthcare utilization rates during the year following initiation of screening among participants screened for AF by electrocardiogram (ECG) sensor patch compared with a matched observational control group. Methods A total of 1718 participants recruited from a health care plan based on age and comorbidities who were screened with an ECG patch (actively monitored group) as part of a prospective, pragmatic research trial were matched by age, sex, and CHA2DS2-VASc score with 3371 members from the same health plan (observational control group). Healthcare utilization, including visits, prescriptions, procedures, and diagnoses, during the 1 year following screening was compared between the groups using health plan claims data. Results Overall, the actively monitored group had significantly higher rates of cardiology visits (adjusted incidence rate ratio [aIRR] [95% confidence interval (CI)]: 1.43 [1.27, 1.60]), no difference in primary care provider visits (aIRR [95% CI]: 1.0 [0.95, 1.05]), but lower rates of emergency department (ED) visits and hospitalizations (aIRR [95% CI]: 0.80 [0.69, 0.92]) compared with controls. Among those with newly diagnosed AF, the reduction in ED visits and hospitalizations was even greater (aIRR [95% CI]: 0.27 [0.17, 0.43]). Conclusion AF screening in an asymptomatic, moderate-risk population with an ECG patch was associated with an increase in cardiology outpatient visits but also significantly lower rates of ED visits and hospitalizations over the 1 year following screening.

Aims Screening for asymptomatic atrial fibrillation (AF) could prevent strokes and save lives, but the AF burden of those detected can impact prognosis. New technologies enable continuous monitoring or intermittent electrocardiogram (ECG) snapshots, however, the relationship between AF detection rates and the burden of AF found with intermittent strategies is unknown. We simulated the likelihood of detecting AF using real-world 2-week continuous ECG recordings and developed a generalizable model for AF detection strategies. Methods and results From 1738 asymptomatic screened individuals, ECG data of 69 individuals (mean age 76.3, median burden 1.9%) with new AF found during 14 days continuous monitoring were used to simulate 30 seconds ECG snapshots one to four times daily for 14 days. Based on this simulation, 35–66% of individuals with AF would be detected using intermittent screening. Twice-daily snapshots for 2 weeks missed 48% of those detected by continuous monitoring, but mean burden was 0.68% vs. 4% in those detected (P Conclusion Using twice-daily ECG snapshots over 2 weeks would detect only half of individuals discovered to have AF by continuous recordings, but AF burden of those missed was low. A model predicting AF detection, validated using real-world data, could assist development of optimized AF screening programmes.

AWAKE-HF evaluated the effect of the initiation of sacubitril/valsartan versus enalapril on activity and sleep using actigraphy in patients who have heart failure with reduced ejection fraction (HFrEF). In this randomized, double-blind study, patients with HFrEF (n = 140) were randomly assigned to sacubitril/valsartan or enalapril for 8 weeks, followed by an 8-week open-label phase with sacubitril/valsartan. Primary endpoint was change from baseline in mean activity counts during the most active 30 min/day at week 8. The key secondary endpoint was change in mean nightly activity counts/minute from baseline to week 8. Kansas City Cardiomyopathy Questionnaire-23 (KCCQ-23) was an exploratory endpoint. There were no detectable differences between groups in geometric mean ratio of activity counts during the most active 30 min/day at week 8 compared with baseline (0.9456 [sacubitril/valsartan:enalapril]; 95% confidence interval [CI] 0.8863–1.0088; P = 0.0895) or in mean change from baseline in activity during sleep (difference: 2.038 counts/min; 95% CI − 0.062 to 4.138; P = 0.0570). Change from baseline to week 8 in KCCQ-23 was 2.89 for sacubitril/valsartan and 4.19 for enalapril, both nonsignificant. In AWAKE-HF, no detectable differences in activity and sleep were observed when comparing sacubitril/valsartan with enalapril in patients with HFrEF using a wearable biosensor. ClinicalTrials.gov, NCT02970669.

Background Diabetes is an independent risk factor for atrial fibrillation (AF), which is associated with increases in mortality and morbidity, as well as a diminished quality of life. Renal involvement in diabetes is common, and since chronic kidney disease (CKD) shares several of the same putative mechanisms as AF, it may contribute to its increased risk in individuals with diabetes. The objective of this study is to identify the relationship between CKD and the rates of newly-diagnosed AF in individuals with diabetes taking part in a screening program using a self-applied wearable electrocardiogram (ECG) patch. Materials and methods The study included 608 individuals with a diagnosis of diabetes among 1738 total actively monitored participants in the prospective mHealth Screening to Prevent Strokes (mSToPS) trial. Participants, without a prior diagnosis of AF, wore an ECG patch for 2 weeks, twice, over a 4-months period and followed clinically through claims data for 1 year. Definitions of CKD included ICD-9 or ICD-10 chronic renal failure diagnostic codes, and the Health Profile Database algorithm. Individuals requiring dialysis were excluded from trial enrollment. Results Ninety-six (15.8%) of study participants with diabetes also had a diagnosis of CKD. Over 12 months of follow-up, 19 new cases of AF were detected among the 608 participants. AF was newly diagnosed in 7.3% of participants with CKD and 2.3% in those without (P Conclusion Among individuals with diabetes, CKD significantly increases the risk of incident AF. Identification of AF prior to clinical symptoms through active ECG screening could help to improve the clinical outcomes in individuals with CKD and diabetes.

Background:Pregnant women living in rural locations in the USA have higher rates of maternal and infant mortality compared to their urban counterparts. One factor contributing to this disparity may be lack of representation of rural women in traditional clinical research studies of pregnancy. Barriers to participation often include transportation to research facilities, which are typically located in urban centers, childcare, and inability to participate during nonwork hours.Methods:POWERMOM is a digital research app which allows participants to share both survey and sensor data during their pregnancy. Through non-targeted, national outreach a study population of 3612 participants (591 from rural zip codes and 3021 from urban zip codes) have been enrolled so far in the study, beginning on March 16, 2017, through September 20, 2019.Results:On average rural participants in our study were younger, had higher pre-pregnancy weights, were less racially diverse, and were more likely to plan a home birth compared to the urban participants. Both groups showed similar engagement in terms of week of pregnancy when they joined, percentage of surveys completed, and completion of the outcome survey after they delivered their baby. However, rural participants shared less HealthKit or sensor data compared to urban participants.Discussion:Our study demonstrated the feasibility and effectiveness of enrolling pregnant women living in rural zip codes using a digital research study embedded within a popular pregnancy app. Future efforts to conduct remote digital research studies could help fill representation and knowledge gaps related to pregnant women.

Individual smartwatch or fitness band sensor data in the setting of COVID-19 has shown promise to identify symptomatic and pre-symptomatic infection or the need for hospitalization, correlations between peripheral temperature and self-reported fever, and an association between changes in heart-rate-variability and infection. In our study, a total of 38,911 individuals (61% female, 15% over 65) have been enrolled between March 25, 2020 and April 3, 2021, with 1,118 reported testing positive and 7,032 negative for COVID-19 by nasopharyngeal PCR swab test. We propose an explainable gradient boosting prediction model based on decision trees for the detection of COVID-19 infection that can adapt to the absence of self-reported symptoms and to the available sensor data, and that can explain the importance of each feature and the post-test-behavior for the individuals. We tested it in a cohort of symptomatic individuals who exhibited an AUC of 0.83 [0.81-0.85], or AUC=0.78 [0.75-0.80] when considering only data before the test date, outperforming state-of-the-art algorithm in these conditions. The analysis of all individuals (including asymptomatic and pre-symptomatic) when self-reported symptoms were excluded provided an AUC of 0.78 [0.76-0.79], or AUC of 0.70 [0.69-0.72] when considering only data before the test date. Extending the use of predictive algorithms for detection of COVID-19 infection based only on passively monitored data from any device, we showed that it is possible to scale up this platform and apply the algorithm in other settings where self-reported symptoms can not be collected.

The COVID-19 pandemic has accelerated the adoption of innovative healthcare methods, including remote patient monitoring. In the setting of limited healthcare resources, outpatient management of individuals newly diagnosed with COVID-19 was commonly implemented, some taking advantage of various personal health technologies, but only rarely using a multi-parameter chest-patch for continuous monitoring. Here we describe the development and validation of a COVID-19 decompensation index (CDI) model based on chest patch-derived continuous sensor data to predict COVID-19 hospitalizations in outpatient-managed COVID-19 positive individuals, achieving an overall AUC of the ROC Curve of 0.84 on 308 event negative participants, and 22 event positive participants, out of an overall study cohort of 400 participants. We retrospectively compare the performance of CDI to standard of care modalities, finding that the machine learning model outperforms the standard of care modalities in terms of both numbers of events identified and with a lower false alarm rate. While only a pilot phase study, the CDI represents a promising application of machine learning within a continuous remote patient monitoring system.

BackgroundEarly diagnosis of structural heart disease improves patient outcomes, yet many remain underdiagnosed. While population screening with echocardiography is impractical, electrocardiogram (ECG)-based prediction models can help target high-risk patients. We developed a novel ECG-based machine learning approach to predict multiple structural heart conditions, hypothesizing that a composite model would yield higher prevalence and positive predictive values (PPVs) to facilitate meaningful recommendations for echocardiography.MethodsUsing 2,232,130 ECGs linked to electronic health records and echocardiography reports from 484,765 adults between 1984-2021, we trained machine learning models to predict the presence of any of seven echocardiography-confirmed diseases within one year. This composite label included: moderate or severe valvular disease (aortic/mitral stenosis or regurgitation, tricuspid regurgitation), reduced ejection fraction 15mm. We tested various combinations of input features (demographics, labs, structured ECG data, ECG traces) and evaluated model performance using 5-fold cross-validation, multi-site validation trained on one clinical site and tested on 11 other independent sites, and simulated retrospective deployment trained on pre-2010 data and deployed in 2010.FindingsOur composite “rECHOmmend” model using age, sex and ECG traces had an area under the receiver operating characteristic curve (AUROC) of 0.91 and a PPV of 42% at 90% sensitivity at a prevalence of 17.9% for our composite label. Individual disease models had AUROCs ranging from 0.86-0.93 and lower PPVs from 1%-31%. The AUROC for models using different input features ranged from 0.80-0.93, increasing with additional features. Multi-site validation showed similar results to the cross-validation, with an aggregate AUROC of 0.91 across our independent test set of 11 clinical sites after training on a separate site. Our simulated retrospective deployment showed that for ECGs acquired in patients without pre-existing known structural heart disease in a single year, 2010, 11% were classified as high-risk, of which 41% developed true, echocardiography-confirmed disease within one year.InterpretationAn ECG-based machine learning model using a composite endpoint can predict previously undiagnosed, clinically significant structural heart disease while outperforming single disease models and improving practical utility with higher PPVs. This approach can facilitate targeted screening with echocardiography to improve under-diagnosis of structural heart disease.

Background Atrial fibrillation (AF) is common, often without symptoms, and is an independent risk factor for mortality, stroke and heart failure. It is unknown if screening asymptomatic individuals for AF can improve clinical outcomes. Methods mSToPS was a pragmatic, direct-to-participant trial that randomized individuals from a single US-wide health plan to either immediate or delayed screening using a continuous-recording ECG patch to be worn for two weeks and 2 occasions, ~3 months apart, to potentially detect undiagnosed AF. The 3-year outcomes component of the trial was designed to compare clinical outcomes in the combined cohort of 1718 individuals who underwent monitoring and 3371 matched observational controls. The prespecified primary outcome was the time to first event of the combined endpoint of death, stroke, systemic embolism, or myocardial infarction among individuals with a new AF diagnosis, which was hypothesized to be the same in the two cohorts but was not realized. Results Over the 3 years following the initiation of screening (mean follow-up 29 months), AF was newly diagnosed in 11.4% (n = 196) of screened participants versus 7.7% (n = 261) of observational controls (p Conclusions At 3 years, screening for AF was associated with a lower rate of clinical events and improved outcomes relative to a matched cohort, although the influence of earlier diagnosis of AF via screening on this finding is unclear. These observational data, including the high event rate surrounding a new clinical diagnosis of AF, support the need for randomized trials to determine whether screening for AF will yield a meaningful protection from strokes and other clinical events. Trail registration The mHealth Screening To Prevent Strokes (mSToPS) Trial is registered on ClinicalTrials.gov with the identifier NCT02506244.

Background: The availability of a wide range of innovative wearable sensor technologies today allows for the ability to capture and collect potentially important health-related data in ways not previously possible. These sensors can be adopted in digitalized clinical trials, i.e., clinical trials conducted outside the clinic to capture data about study participants in their day-to-day life. However, having participants activate, charge, and wear the digital sensors for long hours may prove to be a significant obstacle to the success of these trials. Objective: This study explores a broad question of wrist-wearable sensor effectiveness in terms of data collection as well as data that are analyzable per individual. The individuals who had already consented to be part of an asymptomatic atrial fibrillation screening trial were directly sent a wrist-wearable activity and heart rate tracker device to be activated and used in a home-based setting. Methods: A total of 230 participants with a median age of 71 years were asked to wear the wristband as frequently as possible, night and day, for at least a 4-month monitoring period, especially to track heart rhythm during sleep. Results: Of the individuals who received the device, 43% never transmitted any data. Those who used the device wore it a median of ∼15 weeks (IQR 2–24) and for 5.3 days (IQR 3.2–6.5) per week. For rhythm detection purposes, only 5.6% of all recorded data from individuals were analyzable (with beat-to-beat intervals reported). Conclusions: This study provides some important learnings. It showed that in an older population, despite initial enthusiasm to receive a consumer-quality wrist-based fitness device, a large proportion of individuals never activated the device. However, it also found that for a majority of participants it was possible to successfully collect wearable sensor data without clinical oversight inside a home environment, and that once used, ongoing wear time was high. This suggests that a critical barrier to overcome when incorporating a wearable device into clinical research is making its initiation of use as easy as possible for the participant.

Aims Limited data are available regarding the ability of sacubitril/valsartan to provide clinically meaningful health‐related quality of life (HRQoL) improvements among individuals with heart failure (HF). Objective measurement of physical activity and sleep using actigraphy can provide insight into daily functioning and HRQoL. Methods and results We designed an 18 week, multicenter, randomized, double‐blind, double‐dummy, parallel‐group study to objectively assess changes in function and HRQoL directly after initiating sacubitril/valsartan vs. enalapril in participants with HF in their home environments. A total of 136 outpatient, ambulatory participants with New York Heart Association Class II or III HF with reduced ejection fraction (HFrEF) will be included in the study. Patients will undergo a 2 week baseline observational phase (continuing current HF treatment); data from the second week of this phase will be the baseline value for comparison with those of subsequent periods. Patients will then enter an 8 week blinded‐treatment phase (randomly assigned 1:1 to sacubitril/valsartan or enalapril), followed by an 8 week open‐label extension phase (treatment with only sacubitril/valsartan). The primary efficacy endpoint is the change in mean activity counts during the most active 30 min of the participant's day between baseline and the final randomized treatment phase measurement. Secondary endpoints include the change in mean sleep activity during the randomized and open‐label phases; questionnaires will also assess HRQoL measures. Rather than analysing pooled actigraphy data, the researchers are considering each participant to be acting as his or her own control. Conclusions This will be the first study to assess the effects of sacubitril/valsartan on objective measures of sleep and activity in individuals with HFrEF within the context of their daily lives. Wearable accelerometer devices will be used to gain insight into how the medication affects physical activity and sleep.

Digital medicine has the capacity to affect all aspects of medicine, including disease prediction, prevention, diagnosis, treatment, and post-treatment management. In the field of thyroidology, researchers are also investigating potential applications of digital technology for the thyroid disease. Recent studies using artificial intelligence (AI)/machine learning (ML) have reported reasonable performance for the classification of thyroid nodules based on ultrasonographic (US) images. AI/ML-based methods have also shown good diagnostic accuracy for distinguishing between benign and malignant thyroid lesions based on cytopathologic findings. Assistance from AI/ML methods could overcome the limitations of conventional thyroid US and fine-needle aspiration cytology. A web-based database has been developed for thyroid cancer care. In addition to its role as a nationwide registry of thyroid cancer, it is expected to serve as a clinical platform to facilitate better thyroid cancer care and as a research platform providing comprehensive disease-specific big data. Evidence has been found that biosignal monitoring with wearable devices may predict thyroid dysfunction. This real-world thyroid function monitoring could aid in the management and early detection of thyroid dysfunction. In the thyroidology field, research involving the range of digital medicine technologies and their clinical applications is expected to be even more active in the future.

Introduction: Deep learning (DL) has proved effective for automatic identification of atrial fibrillation (AF) using single-lead ECG. Adoption and trust of DL by clinicians is limited by its black box nature. Hypothesis: Post hoc explanations can elucidate what part of ECG signal is used by the black box DL algorithm, quantifying the importance of clinically relevant features in the classification decision. Making DL decision process transparent will help its integration into clinical practice. Methods: 8,528 single-lead ECG recordings collected using AliveCor devices (PhysioNet) were used. Each signal was labeled as normal sinus rhythm, AF, other arrhythmia or noise. DL automatic classification involves a lightweight convolutional neural network architecture - MobileNet - whose performance is analyzed with an explanation method for DL. Results: Each RR interval is divided into 8 equal segments, where segment 1 follows each R peak, 4 and 5 correspond to the isoelectric baseline, and 7 to the P wave. The explanation method substitutes one of these segments with a straight line, and the corresponding change in sensitivity highlights its importance for the DL algorithm decision. MobileNet achieved a sensitivity of 92.5% to identify AF (9.4% of ECGs were in AF). Sensitivity increases by 2.5% when Segment 7 is removed, indicating that the absence of P wave leads the network to classify more frequently samples as AF.(Figure) When Segments 4 and 5 are removed, the sensitivity decreases by 2.5% and 5.0%, and by 26.7% when removed together. When all RR intervals are normalized to the same value (RR in the Figure), sensitivity for AF drops by 78.3%, showing that RR intervals are key for AF detection by DL algorithm. Conclusions: Post hoc explanations for AF detection by DL from single-lead ECG show the importance of common morphological features used for classifying AF. These methods can be used to understand the decision-making process of DL and motivate its clinical adoption.

Traditional screening for COVID-19 typically includes survey questions about symptoms, travel history, and sometimes temperature measurements. We explored whether longitudinal, personal sensor data can help identify subtle changes which may indicate an infection, such as COVID-19. To do this we developed an app that collects smartwatch and activity tracker data, as well as self-reported symptoms and diagnostic testing results from participants living in the US. We assessed whether symptoms and sensor data could differentiate COVID-19 positive versus negative cases in symptomatic individuals. Between March 25 and June 7, 2020, we enrolled 30,529 participants, of whom 3,811 reported symptoms, 54 reported testing positive for COVID-19, and 279 negative. We found that a combination of symptom and sensor data resulted in an AUC=0.80 [0.73 – 0.86] which was significantly better (p < 0.01) than a model which just considered symptoms alone (AUC=0.71 [0.63 – 0.79]) in the discrimination between symptomatic individuals positive or negative for COVID-19. Such orthogonal, continuous, passively captured data may be complementary to virus testing that is generally a one-off, or infrequent, sampling assay.

Traditional screening for COVID-19 typically includes survey questions about symptoms and travel history, as well as temperature measurements. Here, we explore whether personal sensor data collected over time may help identify subtle changes indicating an infection, such as in patients with COVID-19. We have developed a smartphone app that collects smartwatch and activity tracker data, as well as self-reported symptoms and diagnostic testing results, from individuals in the United States, and have assessed whether symptom and sensor data can differentiate COVID-19 positive versus negative cases in symptomatic individuals. We enrolled 30,529 participants between 25 March and 7 June 2020, of whom 3,811 reported symptoms. Of these symptomatic individuals, 54 reported testing positive and 279 negative for COVID-19. We found that a combination of symptom and sensor data resulted in an area under the curve (AUC) of 0.80 (interquartile range (IQR): 0.73-0.86) for discriminating between symptomatic individuals who were positive or negative for COVID-19, a performance that is significantly better (P < 0.01) than a model1 that considers symptoms alone (AUC = 0.71; IQR: 0.63-0.79). Such continuous, passively captured data may be complementary to virus testing, which is generally a one-off or infrequent sampling assay.

Hypertension is one of the greatest contributors to premature morbidity and mortality worldwide. It has been demonstrated that lowering blood pressure (BP) by just a few mmHg can bring substantial clinical benefits, reducing the risk of stroke and ischemic heart disease. Properly managing high BP is one of the most pressing global health issues, but accurate methods to continuously monitoring BP at home are still under discussion. Indeed, the BP for any given individual can fluctuate significantly during intervals as short as a few minutes. In clinical settings, the guidelines suggest to wait for 5 or 10 minutes in seated rest before taking the measure, in order to alleviate the effect of the stress induced by the clinical environment. Alternatively, BP measured in the home environment is thought to provide a more accurate measure free of the stress of a clinical environment, but there is currently a lack of extensive studies on the trajectory of serial BP measurements over minutes in the home setting. In this paper, we aim at filling this gap by analyzing a large dataset of more than 16 million BP measurements taken at home with commercial BP monitoring devices. In particular, we propose new techniques to analyze this dataset, taking into account the limitations due to the uncontrolled data collection, and we study the characteristics of the BP trajectory for consecutive measures over several minutes. We show that the BP values significantly decrease after 10 minutes minutes from the initial measurement (4.1 and 6.6 mmHg for the diastolic and systolic BP, respectively), and continue to decrease for about 25 minutes. We also describe statistically the clinical relevance of this change, observing more than 50% misclassifications for measurements in the hypertension region. We then propose a model to study the inter-subject variability, showing significant variations in the expected decrease in systolic BP. These results may provide the initial evidence for future large clinical studies using participant-monitored BP.

Background: Although maternal morbidity and mortality in the United States is among the worst of developed countries, pregnant women have been under-represented in research studies, resulting in deficiencies in evidence-based guidance for treatment. There are over two billion smartphone users worldwide, enabling researchers to easily and cheaply conduct extremely large-scale research studies through smartphone apps, especially among pregnant women in whom app use is exceptionally high, predominantly as an information conduit. Objective: We aim to improve our understanding of factors that promote a healthy pregnancy for both the mother and developing fetus through the large-scale and simplified collection of participant-generated data. Study Design: We developed the first pregnancy research app that is embedded within an existing, popular pregnancy app for self-management and education of expectant mothers. Our research app is used to collect a combination of survey and sensor generated data from pregnant women living in the U.S. Results: From the study9s launch on March 16, 2017 through December 17, 2017, we have enrolled 2,058 participants from all 50 states. Our study population is diverse geographically and demographically, and fairly representative of U.S. population averages. We have collected 14,045 individual surveys and 11,669 days of sleep, activity, blood pressure and heart rate measurements during this time. On average, women stayed engaged in the study for 64 days and 48 percent who reached their due date filled out the final outcome survey. Conclusion: During the first nine months of the deployment of this unique program of ongoing research in pregnancy, we demonstrated the potential for a smartphone-based research platform to capture an ever-expanding array of longitudinal, objective and subjective participant-generated data from a continuously growing and diverse population of pregnant women. Clinical Trial Registration ID #: NCT03085875

As we enter the information age of health care, digital health technologies offer significant opportunities to optimize both clinical care delivery and clinical research. Despite their potential, the use of such information technologies in clinical care and research faces major data quality, privacy, and regulatory concerns. In hopes of addressing both the promise and challenges facing digital health technologies in the transformation of health care, we convened a think tank meeting with academic, industry, and regulatory representatives in December 2016 in Washington, DC. In this paper, we summarize the proceedings of the think tank meeting and aim to delineate a framework for appropriately using digital health technologies in healthcare delivery and research.

Multi-omics longitudinal profiling of individuals can detect subtle changes in health status at the earliest possible time point, allowing preemptive initiation of mechanism-appropriate disease-prevention strategies.

Administrative data are frequently used to identify venous thromboembolism (VTE) for research and quality reporting. However, the validity of these codes, particularly in outpatients, has not been well-established. To determine how well International Classification of Diseases, Ninth Revision (ICD-9) codes for VTE predict chart-confirmed acute VTE in inpatient and outpatients. We selected 4642 adults with an incident ICD-9 diagnosis of VTE between years 2004 and 2010 from the Cardiovascular Research Network Venous Thromboembolism cohort study. Medical charts were reviewed to determine validity of events. Positive predictive values (PPVs) of ICD-9 codes were calculated as the number of chart-validated VTE events divided by the number with specific VTE codes. Analyses were stratified by VTE type [pulmonary embolism (PE), deep venous thrombosis (DVT)], code position (primary, secondary), and setting [hospital/emergency department (ED), outpatient]. The PPV for any diagnosis of VTE was 64.6% for hospital/ED patients and 30.9% for outpatients. Primary diagnosis codes from hospital/ED patients were more likely to represent acute VTE than secondary diagnosis codes (78.9% vs. 44.4%, P

Summary Background Acute infections can cause an individual to have an elevated resting heart rate (RHR) and change their routine daily activities due to the physiological response to the inflammatory insult. Consequently, we aimed to evaluate if population trends of seasonal respiratory infections, such as influenza, could be identified through wearable sensors that collect RHR and sleep data. Methods We obtained de-identified sensor data from 200 000 individuals who used a Fitbit wearable device from March 1, 2016, to March 1, 2018, in the USA. We included users who wore a Fitbit for at least 60 days and used the same wearable throughout the entire period, and focused on the top five states with the most Fitbit users in the dataset: California, Texas, New York, Illinois, and Pennsylvania. Inclusion criteria included having a self-reported birth year between 1930 and 2004, height greater than 1 m, and weight greater than 20 kg. We excluded daily measurements with missing RHR, missing wear time, and wear time less than 1000 min per day. We compared sensor data with weekly estimates of influenza-like illness (ILI) rates at the state level, as reported by the US Centers for Disease Control and Prevention (CDC), by identifying weeks in which Fitbit users displayed elevated RHRs and increased sleep levels. For each state, we modelled ILI case counts with a negative binomial model that included 3-week lagged CDC ILI rate data (null model) and the proportion of weekly Fitbit users with elevated RHR and increased sleep duration above a specified threshold (full model). We also evaluated weekly change in ILI rate by linear regression using change in proportion of elevated Fitbit data. Pearson correlation was used to compare predicted versus CDC reported ILI rates. Findings We identified 47 249 users in the top five states who wore a Fitbit consistently during the study period, including more than 13·3 million total RHR and sleep measures. We found the Fitbit data significantly improved ILI predictions in all five states, with an average increase in Pearson correlation of 0·12 (SD 0·07) over baseline models, corresponding to an improvement of 6·3–32·9%. Correlations of the final models with the CDC ILI rates ranged from 0·84 to 0·97. Week-to-week changes in the proportion of Fitbit users with abnormal data were associated with week-to-week changes in ILI rates in most cases. Interpretation Activity and physiological trackers are increasingly used in the USA and globally to monitor individual health. By accessing these data, it could be possible to improve real-time and geographically refined influenza surveillance. This information could be vital to enact timely outbreak response measures to prevent further transmission of influenza cases during outbreaks. Funding Partly supported by the US National Institutes of Health National Center for Advancing Translational Sciences.

Objective Many American adults are insufficiently active. Digital health programs are designed to motivate this population to engage in regular physical activity and often rely on wearable devices and apps to objectively measure physical activity for a large number of participants. The purpose of this epidemiological study was to analyze the rates of physical activity among participants in a digital health program. Method We conducted a cross-sectional study of participants enrolled in a digital health program between January 2014 and December 2016. All activity data were objectively collected through wearable devices. Results Participants ( n = 241,013) were on average 39.7 years old and 65.7% were female. Participants walked on average 3.72 miles per day. Overall, 5.3% and 21.8% of participants were being treated with diabetes and cardiovascular medications respectively, but these rates varied across young, middle and older adults. Participants of all ages being treated with cardiovascular and/or diabetes medications walked significantly less than those not being treated for these conditions. Conclusion The feasibility of using a large database containing data from consumer-grade activity trackers was demonstrated through this epidemiological study of physical activity rates across age and condition status of participants. The approach and findings described may inform future research as the information age brings about new opportunities to manage and study massive amounts of data generated by connected devices.

BackgroundHeart rate is routinely measured as part of the clinical examination but is rarely acted upon unless it is well outside a population-based normal range. With wearable sensor technologies, heart rate can now be continuously measured, making it possible to accurately identify an individual's "normal" heart rate and potentially important variations in it over time. Our objective is to describe inter- and intra-individual variability in resting heart rate (RHR) collected over the course of two years using a wearable device, studying the variations of resting heart rate as a function of time of year, as well as individuals characteristics like age, sex, average sleep duration, and body mass index (BMI).Methods and findingsOur retrospective, longitudinal cohort study includes 92,457 de-identified individuals from the United States (all 50 states), who consistently-over at least 35 weeks in the period from March 2016 to February 2018, for at least 2 days per week, and at least 20 hours per day-wore a heart rate wrist-worn tracker. In this study, we report daily RHR and its association with age, BMI, sex, and sleep duration, and its variation over time. Individual daily RHR was available for a median of 320 days, providing nearly 33 million daily RHR values. We also explored the range in daily RHR variability between individuals, and the long- and short-term changes in the trajectory of an individual's daily RHR. Mean daily RHR was 65 beats per minute (bpm), with a range of 40 to 109 bpm among all individuals. The mean RHR differed significantly by age, sex, BMI, and average sleep duration. Time of year variations were also noted, with a minimum in July and maximum in January. For most subjects, RHR remained relatively stable over the short term, but 20% experienced at least 1 week in which their RHR fluctuated by 10 bpm or more.ConclusionsIndividuals have a daily RHR that is normal for them but can differ from another individual's normal by as much as 70 bpm. Within individuals, RHR was much more consistent over time, with a small but significant seasonal trend, and detectable discrete and infrequent episodes outside their norms.

PURPOSE OF THE REVIEW: Advances in computing power and wireless technologies have reshaped our approach to patient monitoring. Medical grade sensors and apps that were once restricted to hospitals and specialized clinic are now widely available. Here, we review the current evidence supporting the use of connected health technologies for the prevention and management of cardiovascular disease in an effort to highlight gaps and future opportunities for innovation. RECENT FINDINGS: Initial studies in connected health for cardiovascular disease prevention and management focused primarily on activity tracking and blood pressure monitoring but have since expanded to include a full panoply of novel sensors and pioneering smartphone apps with targeted interventions in diet, lipid management and risk assessment, smoking cessation, cardiac rehabilitation, heart failure, and arrhythmias. While outfitting patients with sensors and devices alone is infrequently a lasting solution, monitoring programs that include personalized insights based on patient-level data are more likely to lead to improved outcomes. Advances in this space have been driven by patients and researchers while healthcare systems remain slow to fully integrate and adequately adapt these new technologies into their workflows. SUMMARY: Cardiovascular disease prevention and management continue to be key focus areas for clinicians and researchers in the connected health space. Exciting progress has been made though studies continue to suffer from small sample size and limited follow up. Efforts that combine home patient monitoring, engagement, and personalized feedback are the most promising. Ultimately, combining patient-level ambulatory sensor data, electronic health records, and genomics using machine learning analytics will bring precision medicine closer to reality.

Many barriers to primary healthcare accessibility in the United States exist including an increased opportunity cost associated with seeking primary care. New models of healthcare delivery aimed at addressing these problems are emerging. The potential impact that on-demand primary care physician house calls services can have on healthcare accessibility, patient care, and satisfaction by both patients and physicians is poorly characterized. We performed a retrospective observational analysis on data from 13,849 patients who utilized Heal, Inc, an application (app)-based, on-demand house calls platform between August 2016 and July 2017. We assessed house call wait time and visit duration, diagnoses by International Classification of Diseases, tenth revision, Inc (ICD10) codes, and house call outcomes by post-visit prescription and lab requests, and patient satisfaction survey. Patients who utilized this physician house call service had a bimodal age distribution peaking at age 1 year and 36 years. Same day acute sick exams (93.9% of pediatric (Ped) and 66.9% of adult requests) for fever and/or acute upper respiratory infection represented the most common use. The mean wait time for as soon as possible house calls were 96.1 minutes, with an overall mean house call duration of 27.1 minutes. A house call was primarily chosen over an Urgent Care Clinic or Doctor's office (46.2% and 41.6% of respondents, respectively), due to convenience or fastest appointment available (69.6% and 33.8% of respondents, respectively). Most survey respondents (94.2%) would schedule house calls again. On-demand physician house calls programs can expand access options to primary healthcare, primarily used by younger individuals with acute illness and preference for a smartphone app-based home visit.

This cohort study examines the duration and variation of recovery among COVID-19–positive verses COVID-19–negative individuals.

This cohort study examines sleep and demographic data from wearable activity tracking devices to assess for an association of sleep duration and variability with body mass index.

Objectives: The advent of large databases, wearable technology, and novel communications methods has the potential to expand the pool of candidate research participants and offer them the flexibility and convenience of participating in remote research. However, reports of their effectiveness are sparse. We assessed the use of various forms of outreach within a nationwide randomized clinical trial being conducted entirely by remote means. Methods: Candidate participants at possibly higher risk for atrial fibrillation were identified by means of a large insurance claims database and invited to participate in the study by their insurance provider. Enrolled participants were randomly assigned to one of two groups testing a wearable sensor device for detection of the arrhythmia. Results: Over 10 months, the various outreach methods used resulted in enrollment of 2659 participants meeting eligibility criteria. Starting with a baseline enrollment rate of 0.8% in response to an email invitation, the recruitment campaign was iteratively optimized to ultimately include website changes and the use of a five-step outreach process (three short, personalized emails and two direct mailers) that highlighted the appeal of new technology used in the study, resulting in an enrollment rate of 9.4%. Messaging that highlighted access to new technology outperformed both appeals to altruism and appeals that highlighted accessing personal health information. Conclusions: Targeted outreach, enrollment, and management of large remote clinical trials is feasible and can be improved with an iterative approach, although more work is needed to learn how to best recruit and retain potential research participants. Trial registration: Clinicaltrials.gov NCT02506244. Registered 23 July 2015. Keywords: Clinical trials, Clinical research, Digital technology, Remote enrollment, Remote monitoring, Outreach

Background Paroxysmal atrial fibrillation (PAF) is broadly defined despite high variability in the occurrence and duration of PAF episodes. Objective The purpose of this study was to identify rhythm patterns in a large cohort of individuals with PAF who wore an ambulatory single-lead electrocardiogram (ECG) patch sensor as part of standard clinical care. Methods We performed a retrospective analysis of longitudinal rhythm data obtained from 13,293 individuals with PAF. Results In this study, 7934 men and 5359 women with PAF wore an ambulatory single-lead ECG patch sensor for 11.4 days on average, experiencing 1,041,504 PAF episodes. The median daily rate of PAF was 1.21 episodes per day (interquartile range [IQR] 0.31–4.99), and the median maximum duration per individual was 7.5 hours (IQR 2.4–18.6 hours). There was an inverse relationship between the duration of PAF episodes and the frequency in which they occurred, which became pronounced at moderate and high overall burdens of AF. This produced a spectrum of PAF flanked by 2 distinct subtypes of the disease: the staccato subtype, characterized by many, short AF episodes; and the legato subtype, characterized by fewer, longer episodes. Longer but less frequent episodes became more common with increasing age. Only 49.4% of individuals experienced an episode in the first 24 hours of monitoring, increasing to 89.7% after 1 week of monitoring. Conclusion We identified subtypes of the disease that we labeled staccato and legato. Although further study is required, these subtypes may result from differing elements of pathophysiology and disease progression, and may confer differing stroke risks.

This publication describes uniform definitions for cardiovascular and stroke outcomes developed by the Standardized Data Collection for Cardiovascular Trials Initiative and the US Food and Drug Administration (FDA). The FDA established the Standardized Data Collection for Cardiovascular Trials Initiative in 2009 to simplify the design and conduct of clinical trials intended to support marketing applications. The writing committee recognizes that these definitions may be used in other types of clinical trials and clinical care processes where appropriate. Use of these definitions at the FDA has enhanced the ability to aggregate data within and across medical product development programs, conduct meta-analyses to evaluate cardiovascular safety, integrate data from multiple trials, and compare effectiveness of drugs and devices. Further study is needed to determine whether prospective data collection using these common definitions improves the design, conduct, and interpretability of the results of clinical trials.

Cardiovascular disease remains the leading cause of death and disease worldwide. As demands on an already resource-constrained healthcare system intensify, disease prevention in the future will likely depend on out-of-office monitoring of cardiovascular risk factors. Mobile health tracking devices that can track blood pressure and heart rate, in addition to new cardiac vital signs, such as physical activity level and pulse wave velocity (PWV), offer a promising solution. An initial barrier is the development of accurate and easily-scalable platforms. In this study, we made a customized smartphone app and used mobile health devices to track PWV, blood pressure, heart rate, physical activity, sleep duration, and multiple lifestyle risk factors in ≈250 adults for 17 continual weeks. Eligible participants were identified by a company database and then were consented and enrolled using only a smartphone app, without any special training given. Study participants reported high overall satisfaction, and 73% of participants were able to measure blood pressure and PWV

Aim To evaluate the impact of antithrombotic regimens during the medical phase of treatment among 13,819 patients with non-ST-segment elevation acute coronary syndromes (NSTE-ACS) treated with an early invasive strategy in the acute catheterization and urgent intervention triage strategy (ACUITY) trial. Methods and results Endpoints included composite major adverse cardiac events (MACE), major bleeding, and net adverse clinical events (NACE; MACE or major bleeding). The median (interquartile range) duration of antithrombin use in the medical only treatment phase was 6.5 (1.8–22.5) hours. MACE, major bleeding, and NACE during the medical only phase occurred in 63 (0.5%), 117 (0.9%), and 178 (1.3%) patients, respectively. MACE rates in the medical-treatment-only phase were not significantly different between the four randomized medical regimens used (heparin alone, bivalirudin alone, heparin plus a glycoprotein IIb/IIIa inhibitor [GPI], and bivalirudin plus GPI) (Ptrend = 0.65). The lowest rates of major bleeding and NACE during the medical treatment phase occurred in patients treated with bivalirudin alone (Ptrend = 0.0006 and Ptrend = 0.0004, respectively). Conclusions In patients with NSTE-ACS undergoing an early invasive strategy, treatment with bivalirudin alone significantly reduced major bleeding and improved net clinical outcomes during the upstream medical management phase with comparable rates of MACE. © 2015 Wiley Periodicals, Inc.

So far, the digitization of health care is best exemplified by electronic medical records, which have been far from favorably or uniformly accepted. However, properly implemented digitization can enable better patient outcomes, improve convenience, potentially lower healthcare costs, and possibly lead to much greater physician satisfaction. Precision (also known as personalized or individualized) medicine is frequently discussed today, but, in reality, it is what physicians have attempted to do as best they could for millennia. But now we have new tools that can begin to give us a much more high-definition view of our patients; from affordable and rapid genetic testing to wearable sensors that track a wide range of important physiologic parameters continuously. Although seemingly counterintuitive, the digitization of health care can also markedly improve the physician-patient relationship, allowing more time for human interaction when care is bolstered by digital technologies that better individualize diagnostics and treatments.

Continuous, remote monitoring of multiple vital signs in patients with Ebola, coupled with personalized data analytics, can warn health care workers of critical changes in patients’ status much earlier than with conventional intermittent monitoring—which typically occurs just every 8 hours—and without risking the safety of health care workers. This sophisticated technology, in turn, could lead to earlier initiation of lifesaving interventions, better health outcomes, and reduced risk of spreading the virus. A consortium of academic and industry partners, STAMP2 (which stands for Sensor Technology & Analytics to Monitor, Predict and Protect Ebola patients), has developed such a novel, wearable patient sensor—resembling a band-aid—that tracks and wirelessly transmits multiple vital signs to remote health care workers in non-red zone observation areas. The technology, which includes the MultiSense sensor coupled with state-of-the-art, real-time data analytical capabilities, called Personalized Physiology Analytics (PPA), sends patient-specific automated alerts of any important changes in the patient’s condition without requiring the health care worker to constantly monitor display screens. The technology was developed in response to the Grand Ebola Challenge, sponsored by the United States Agency for International Development (USAID), to develop innovative solutions to improve the safety of health care workers and care of patients with, or at high risk for, Ebola virus infection.

The ABC ’s of primary and secondary cardiovascular disease prevention are not complete without D (diet and weight management) and certainly without E (exercise) [(1)][1]. Although coming in last in the helpful mnemonic, exercise certainly is not the least important. The benefits of increased

BACKGROUND Using the data from 56,365 individuals, from 185 countries, and a Nokia Health Wireless blood pressure (BP) monitor, we investigated real-world characteristics of BP variability (BPV). METHODS All included individuals self-measured and uploaded their BP using Bluetooth at least 20 times over a period of ≥1 month at a frequency and duration of their choosing. In total, 16,904,844 BP measurements were analyzed, with a median of 146 measurements per person (interquartile range [IQR] 73–321) over a median of 14 months (IQR 7–31). SD, coefficient of variation, maximum BP, and maximum minus minimum BP difference were all calculated as measures of BPV. RESULTS BPV showed a distinct pattern, influenced by season of year, day of week, and time of day. BPV index was higher in females compared with males (P < 0.001) and increased with age (P < 0.001). Compared to the weekend, the weekday BPV index was significantly higher, and this finding was more prominent in females (P = 0.001). In multivariate analysis, BPV index were significantly associated with age, gender, geographic location, and mean BP values. CONCLUSION Using the largest BP data set we are aware of, with the benefits and limitations of real-world measurement, we could show the pattern of BPV and provide reference values that may be helpful in understanding the nature of BPV as self-measurement at home becomes more common, and help guide individualized management.

Background— Clinical trial data associate extended clopidogrel therapy with increased mortality and cancer. We sought to determine the impact of continued clopidogrel use on mortality and cancer within a patient-level meta-analysis of randomized clinical trials. Methods and Results— Meta-analytic clinical event rates for all-cause, cardiovascular, noncardiovascular, and cancer-related mortality; cancer; myocardial infarction; stroke; and fatal and major nonfatal bleeding were generated using patient-level data from 6 randomized trials comparing prolonged versus no or short-duration clopidogrel on a background of aspirin in patients with cardiovascular and cerebrovascular disease. Among 48 817 randomized patients (median follow-up 546 days), there was no difference in all-cause (7.23% versus 7.26%; P =0.97), cardiovascular (5.25% versus 5.22%; P =0.86), noncardiovascular (1.98% versus 2.03%; P =0.73), and cancer-related (0.93% versus 0.99%; P =0.59) mortality or in new cancer diagnoses (2.97% versus 2.96%; P >0.99). Rates of myocardial infarction (3.21% versus 4.05%; P P P =0.03), as were major nonfatal bleeding (4.06% versus 2.68%; P P =0.02). Conclusions— Across trials of cardiovascular and cerebrovascular disease, extended-duration clopidogrel on a background of aspirin has no overall effect on mortality or cancer but does reduce rates of myocardial infarction and stroke and increase rates of bleeding. These findings emphasize the need for selective use of extended clopidogrel therapy in patients in whom the risks of ischemia are not fully counterbalanced by the risks of bleeding.

Early detection of Heart Failure (HF) could mitigate the enormous individual and societal burden from this disease. Clinical detection is based, in part, on recognition of the multiple signs and symptoms comprising the Framingham HF diagnostic criteria that are typically documented, but not necessarily synthesized, by primary care physicians well before more specific diagnostic studies are done. We developed a natural language processing (NLP) procedure to identify Framingham HF signs and symptoms among primary care patients, using electronic health record (EHR) clinical notes, as a prelude to pattern analysis and clinical decision support for early detection of HF.We developed a hybrid NLP pipeline that performs two levels of analysis: (1) At the criteria mention level, a rule-based NLP system is constructed to annotate all affirmative and negative mentions of Framingham criteria. (2) At the encounter level, we construct a system to label encounters according to whether any Framingham criterion is asserted, denied, or unknown.Precision, recall, and F-score are used as performance metrics for criteria mention extraction and for encounter labeling.Our criteria mention extractions achieve a precision of 0.925, a recall of 0.896, and an F-score of 0.910. Encounter labeling achieves an F-score of 0.932.Our system accurately identifies and labels affirmations and denials of Framingham diagnostic criteria in primary care clinical notes and may help in the attempt to improve the early detection of HF. With adaptation and tooling, our development methodology can be repeated in new problem settings.

The rapidly increasing availability of electronic health records (EHRs) from multiple heterogeneous sources has spearheaded the adoption of data-driven approaches for improved clinical research, decision making, prognosis, and patient management. Unfortunately, EHR data do not always directly and reliably map to medical concepts that clinical researchers need or use. Some recent studies have focused on EHR-derived phenotyping, which aims at mapping the EHR data to specific medical concepts; however, most of these approaches require labor intensive supervision from experienced clinical professionals. Furthermore, existing approaches are often disease-centric and specialized to the idiosyncrasies of the information technology and/or business practices of a single healthcare organization.In this paper, we propose Limestone, a nonnegative tensor factorization method to derive phenotype candidates with virtually no human supervision. Limestone represents the data source interactions naturally using tensors (a generalization of matrices). In particular, we investigate the interaction of diagnoses and medications among patients. The resulting tensor factors are reported as phenotype candidates that automatically reveal patient clusters on specific diagnoses and medications. Using the proposed method, multiple phenotypes can be identified simultaneously from data.We demonstrate the capability of Limestone on a cohort of 31,815 patient records from the Geisinger Health System. The dataset spans 7years of longitudinal patient records and was initially constructed for a heart failure onset prediction study. Our experiments demonstrate the robustness, stability, and the conciseness of Limestone-derived phenotypes. Our results show that using only 40 phenotypes, we can outperform the original 640 features (169 diagnosis categories and 471 medication types) to achieve an area under the receiver operator characteristic curve (AUC) of 0.720 (95% CI 0.715 to 0.725). Moreover, in consultation with a medical expert, we confirmed 82% of the top 50 candidates automatically extracted by Limestone are clinically meaningful.