Search

Background: The coronavirus disease (COVID-19) pandemic has resulted in significant morbidity and mortality; large numbers of patients require intensive care, which is placing strain on health care systems worldwide. There is an urgent need for a COVID-19 disease severity assessment that can assist in patient triage and resource allocation for patients at risk for severe disease.Objective: The goal of this study was to develop, validate, and scale a clinical decision support system and mobile app to assist in COVID-19 severity assessment, management, and care.Methods: Model training data from 701 patients with COVID-19 were collected across practices within the Family Health Centers network at New York University Langone Health. A two-tiered model was developed. Tier 1 uses easily available, nonlaboratory data to help determine whether biomarker-based testing and/or hospitalization is necessary. Tier 2 predicts the probability of mortality using biomarker measurements (C-reactive protein, procalcitonin, D-dimer) and age. Both the Tier 1 and Tier 2 models were validated using two external datasets from hospitals in Wuhan, China, comprising 160 and 375 patients, respectively.Results: All biomarkers were measured at significantly higher levels in patients who died vs those who were not hospitalized or discharged (P<.001). The Tier 1 and Tier 2 internal validations had areas under the curve (AUCs) of 0.79 (95% CI 0.74-0.84) and 0.95 (95% CI 0.92-0.98), respectively. The Tier 1 and Tier 2 external validations had AUCs of 0.79 (95% CI 0.74-0.84) and 0.97 (95% CI 0.95-0.99), respectively.Conclusions: Our results demonstrate the validity of the clinical decision support system and mobile app, which are now ready to assist health care providers in making evidence-based decisions when managing COVID-19 patient care. The deployment of these new capabilities has potential for immediate impact in community clinics and sites, where application of these tools could lead to improvements in patient outcomes and cost containment. [ABSTRACT FROM AUTHOR]

Background: Older adults' health maintenance may be enhanced by having access to online health information. However, usability issues may prevent older adults from easily accessing such information. Prior research has shown that aging is associated with a unique pattern of cognitive changes, and knowledge of these changes may be used in the design of health websites for older adults., Objective: The goal of the current study was to examine whether older adults use of a health information website was affected by an alternative information architecture and access interface (hierarchical versus tag-based)., Methods: Fifty younger adults (aged 18-23) and 50 older adults (aged 60-80) navigated a health information website, which was organized hierarchically or used tags/keywords, to find answers to health-related questions while their performance was tracked. We hypothesized that older adults would perform better in the tag-based health information website because it placed greater demands on abilities that remain intact with aging (verbal ability and vocabulary)., Results: The pattern of age-related differences in computer use was consistent with prior research with older adults. We found that older adults had been using computers for less time (F(1,98)= 10.6, P= .002) and used them less often (F(1,98)= 11.3, P= .001) than younger adults. Also consistent with the cognitive aging literature, younger adults had greater spatial visualization and orientation abilities (F(1,98)= 34.6, P< .001 and F(1,98)= 6.8, P= .01) and a larger memory span (F(1,98)= 5.7, P= .02) than older adults, but older adults had greater vocabulary (F(1,98)= 11.4, P= .001). Older adults also took significantly more medications than younger adults (F(1,98)= 57.7, P< .001). In the information search task, older adults performed worse than younger adults (F(1,96)= 18.0, P< .001). However, there was a significant age x condition interaction indicating that while younger adults outperformed older adults in the hierarchical condition (F(1,96)= 25.2, P< .001), there were no significant age-related differences in the tag-based condition, indicating that older adults performed as well as younger adults in this condition., Conclusions: Access to online health information is increasing in popularity and can lead to a more informed health consumer. However, usability barriers may differentially affect older adults. The results of the current study suggest that the design of health information websites that take into account age-related changes in cognition can enhance older adults' access to such information.

Background: Patients in the United States commonly use the Internet to acquire health information. While a significant amount of health-related information is available on the Internet, the accuracy of this information is highly variable., Objectives: The objective of the study was to determine how effectively students can assess the accuracy of Internet-based material when gathering information on a controversial medical topic using simple keyword searches., Methods: A group of 34 students from the science magnet high school in Houston, Texas searched for the terms "vaccine safety" and "vaccine danger" using Google and then answered questions regarding the accuracy of the health information on the returned sites. The students were also asked to describe the lessons they learned in the exercise and to answer questions regarding the strength of evidence for seven statements regarding vaccinations. Because of the surprising revelation that the majority of students left the exercise with inaccurate information concerning the safety and efficacy of vaccines, these same students participated in a follow-up study in which a fact-based vaccine video was shown, after which the assessment of student knowledge was repeated., Results: Of the 34 participants, 20 (59%) thought that the Internet sites were accurate on the whole, even though over half of the links (22 out of 40, 55%) that the students viewed were, in fact, inaccurate on the whole. A high percentage of the students left the first exercise with significant misconceptions about vaccines; 18 of the 34 participants (53%) reported inaccurate statements about vaccines in the lessons they learned. Of the 41 verifiable facts about vaccines that were reported by participants in their lessons-learned statement, 24 of those facts (59%) were incorrect. Following presentation of the film, the majority of students left the exercise with correct information about vaccines, based on their lessons-learned statement. In this case, 29 of the 31 participants (94%) reported accurate information about vaccines. Of the 49 verifiable facts about vaccines that were reported by participants, only 2 (4%) were incorrect. Students had higher correct scores in the "strength of evidence" exercise following exposure to the video as well., Conclusions: Allowing students to use the Internet to gain information about medical topics should be approached with care since students may take away predominantly incorrect information. It is important to follow up conflicting information with a solid, unambiguous message that communicates those lessons that the instructor deems most important. This final message should be fact based but may need to contain an anecdotal component to counter the strong emotional message that is often delivered by inaccurate Internet sites.

Background: The Internet is a viable channel to deliver evidence-based smoking cessation treatment that has the potential to make a large population impact on reducing smoking prevalence. There is high demand for smoking cessation information and support on the Internet. Approximately 7% (10.2 million) of adult American Internet users have searched for information on quitting smoking. Little is known about these individuals, their smoking status, what type of cessation services they are seeking on the Internet, or how frequently these searches for cessation information are conducted., Objective: The primary goal of this study was to characterize individuals who search for smoking cessation information on the Internet to determine appropriate triage and treatment strategies. The secondary goal was to estimate the incidence of searches for cessation information using publicly available search engine data., Methods: We recruited individuals who clicked on a link to a leading smoking cessation website (QuitNet) from within the results of a search engine query. Individuals were "intercepted" before seeing the QuitNet home page and were invited to participate in the study. Those accepting the invitation were routed to an online survey about demographics, smoking characteristics, preferences for specific cessation services, and Internet search patterns. To determine the generalizability of our sample, national datasets on search engine usage patterns, market share, and keyword rankings were examined. These datasets were then used to estimate the number of queries for smoking cessation information each year., Results: During the 10-day study period, 2265 individuals were recruited and 29% (N = 655) responded. Of these, 59% were female and overall tended to be younger than the previously characterized general Internet population. Most (76%) respondents were current smokers; 17% had quit within the last 7 days, and 7% had quit more than 7 days ago. Slightly more than half of active smokers (53%) indicated that they were planning to quit in the next 30 days. Smokers were more likely to seek information on how to quit and on medications; former smokers were more interested in how to cope with withdrawal. All participants rated withdrawal information and individually tailored information as being more useful, while displaying little interest in telephone counseling, expert support, or peer support. Publicly available data from large search engines suggest that 4 million Americans search for resources on smoking cessation each year., Conclusions: This study adds to the limited data available on individuals who search for smoking cessation information on the Internet, supports the prior estimates of the size of the population, and indicates that these individuals are in appropriate stages for both active cessation interventions and aggressive relapse prevention efforts. Continued development and evaluation of online interventions is warranted, and organizations seeking to promote cessation should carefully evaluate the Internet as a possible modality for treatment and as a gateway to other traditional programs.

Background: Information about drug withdrawals may not reach patients in a timely manner, and this could result in adverse events. Increasingly, the public turns to consumer health websites for health information, but such sites may not update their content for days or weeks following important events like Food and Drug Administration (FDA) drug withdrawal actions. There is no recognized standard for how quickly consumer health websites should respond to such events, and reports addressing this issue are lacking., Objective: The objective of this study was to develop and implement an approach to enhance the efficiency with which a consumer health website (NetWellness.org) responds to FDA drug withdrawal actions., Methods: Evaluation of the current approach used by NetWellness staff to update content affected by FDA action revealed a slow process driven by the goal of performing thorough and comprehensive review and editing. To achieve our desired goal of accurately updating affected content within 24 hours of FDA action, we developed a strategy that included rapid updating of affected Web pages with warning boxes and hyperlinks to the information about the withdrawal. With the next FDA withdrawal event, that of valdecoxib (Bextra) on April 7, 2005, we applied this new approach, observed the time and resource requirements, and monitored the rate at which consumers viewed the updated information to gauge its potential impact., Results: Application of the new approach allowed one person to modify the affected Web pages in less than 1 hour and within 18 hours of the FDA announcement. Using the old strategy, response to a similar event, the withdrawal of rofecoxib (Vioxx) 6 months earlier, had taken over 3 weeks and the efforts of several personnel. Updated valdecoxib content received 188 hits within the first month and 4285 hits within 1 year., Conclusions: Rapid updating of a consumer health website's content in response to an FDA drug withdrawal event was easily accomplished by applying the approach described. This allowed consumers to view accurate information regarding the withdrawn drug much sooner than would otherwise have been the case. Given that consumers increasingly turn to websites for their health information, adoption of a rapid response standard for important health events like FDA drug withdrawals should be considered by the consumer health informatics community.

Advancing the science and practice of health promotion and disease management on the Internet requires a systematic program of research examining the population impact of such programs. With impact described as the combination of effectiveness and participation, such research needs to include the examination of the quality and effectiveness of programs that are available to the general public, as well as descriptive and predictive knowledge about population readiness to participate in such programs. There have been few studies examining the quality of interactive health behavior change (HBC) programs on the Internet, and even fewer investigations of the effectiveness of such programs. Based on the review of over 300 HBC programs on the Internet using the "5 A's" of Health Behavior Change on the Internet (HBC-I Screener), which represent standard minimum guidelines for evaluation, it appears HBC on the Internet is in the early stages of development. As health behavior change on the Internet matures from the provision of health information to meeting the requirements necessary to produce health behavior change, and as program developers take advantage of the interactive nature of the Internet, the basic screening and expanded evaluation criteria developed in this project will provide templates for both consumers and developers of programs. The second component necessary for evaluating the impact of HBC on the Internet is the extent to which the population is ready to participate in such programs. We need to move beyond a narrow focus on early adopters and produce a population perspective that includes those not ready, those getting ready, and those ready to use such programs, as well as those already participating. By understanding participation levels of such programs, and what drives this participation, the development and dissemination of practical tailored and targeted interventions can help maximize population participation in Internet programs for health behavior change.

The development of online communication systems related to prevention, decision making, and coping with cancer has outpaced theoretical attention to the attributes that appeal to system users and that create effective interactions. This essay reviews a number of sociotechnical attributes related to online discussion systems and tutorials, including interactivity, presence, homophily, social distance, anonymity/privacy, and interaction management. These attributes are derived from different theoretical perspectives which have led to clinical trials and other empirical studies demonstrating effectiveness or attraction to end users. The effects of a subset of these attributes are connected to learning, social influence, and coping, as illustrated in evaluations of an interactive smoking prevention site and a cancer advice/support discussion system.

Background: The incidence of vaccine-preventable diseases is directly related to the number of unvaccinated children. Parents who refuse vaccination of their children frequently express concerns about vaccine safety. The Internet can influence perceptions about vaccines because it is the fastest growing source of consumer health information. However, few studies have analyzed vaccine criticism on the Web., Objective: The purposes of this paper are to examine vaccine criticism on the Internet and to analyze the websites in order to identify common characteristics and ethical allegations., Methods: A structured Web search was conducted for the terms "vaccine," "vaccination," "vaccinate," and "anti-vaccination" using a metasearch program that incorporated 8 search engines. This yielded 1138 Web pages representing 750 sites. Two researchers reviewed the sites for inclusion/exclusion criteria, resulting in 78 vaccine-critical sites, which were then abstracted for design, content, and allegations., Results: The most common characteristic of vaccine-critical websites was the inclusion of statements linking vaccinations with specific adverse reactions, especially idiopathic chronic diseases such as multiple sclerosis, autism, and diabetes. Other common attributes (> or = 70% of websites) were links to other vaccine-critical websites; charges that vaccines contain contaminants, mercury, or "hot lots" that cause adverse events; claims that vaccines provide only temporary protection and that the diseases prevented are mild; appeals for responsible parenting through education and resisting the establishment; allegations of conspiracies and cover-ups to hide the truth about vaccine safety; and charges that civil liberties are violated through mandatory vaccination., Conclusions: Vaccine-critical websites frequently make serious allegations. With the burgeoning of the Internet as a health information source, an undiscerning or incompletely educated public may accept these claims and refuse vaccination of their children. As this occurs, the incidence of vaccine-preventable diseases can be expected to rise.

Background: The Internet is becoming an increasingly important resource for health-information seekers. However, consumers often do not use effective search strategies. Query reformulation is one potential intervention to improve the effectiveness of consumer searches., Objective: We endeavored to answer the research question: "Does reformulating original consumer queries with preferred terminology from the Unified Medical Language System (UMLS) Metathesaurus lead to better search returns?", Methods: Consumer-generated queries with known goals (n=16) that could be mapped to UMLS Metathesaurus terminology were used as test samples. Reformulated queries were generated by replacing user terms with Metathesaurus-preferred synonyms (n=18). Searches (n=36) were performed using both a consumer information site and a general search engine. Top 30 precision was used as a performance indicator to compare the performance of the original and reformulated queries., Results: Forty-two percent of the searches utilizing reformulated queries yielded better search returns than their associated original queries, 19% yielded worse results, and the results for the remaining 39% did not change. We identified ambiguous lay terms, expansion of acronyms, and arcane professional terms as causes for changes in performance., Conclusions: We noted a trend towards increased precision when providing substitutions for lay terms, abbreviations, and acronyms. We have found qualitative evidence that reformulating queries with professional terminology may be a promising strategy to improve consumer health-information searches, although we caution that automated reformulation could in fact worsen search performance when the terminology is ill-fitted or arcane.

Background: Although leading children's hospitals are recognized as preeminent in the provision of health care to children, the quality of their Web sites has not been described., Objective: To describe technical characteristics of the Web sites of leading children's hospitals., Methods: This is a cross-sectional descriptive infodemiology study. Two reviewers independently reviewed and analyzed the Web sites of 26 nationally prominent children's hospitals in June 2003, using objective criteria based on accessibility (based on age and language), attribution, completeness, credibility, currency, disclosure, readability, and other technical elements., Results: One-third of Web sites included content for children and adolescents. Twenty-four (92%) of the Web sites had health and disease-specific information. One-third contained only English, while two-thirds included other languages. All 26 Web sites included a disclaimer, although none had a requirement to read the disclaimer before accessing health and disease specific information. Twenty-four (92%) had search options. Although most (85%) listed a copyright date, only 10% listed the date last updated., Conclusions: This is the first study to examine the Web sites of leading children's hospitals. Although the Web sites were designed for children's hospitals, only a few sites included content for children and adolescents. Primary care physicians who refer patients to these sites should be aware that many have limited content for children, and should assess them for other limitations, such as inconsistent documentation of disclaimers or failure to show the date of the last Web site update. These Web sites are a potentially useful source of patient information. However, as the public increasingly looks to the Internet for health information, children's hospitals need to keep up with increasingly high standards and demands of health-care consumers.

Background: The World Wide Web (WWW) has become an increasingly essential resource for health information consumers. The ability to obtain accurate medical information online quickly, conveniently and privately provides health consumers with the opportunity to make informed decisions and participate actively in their personal care. Little is known, however, about whether the content of this online health information is equally accessible to people with disabilities who must rely on special devices or technologies to process online information due to their visual, hearing, mobility, or cognitive limitations., Objective: To construct a framework for an automated Web accessibility evaluation; to evaluate the state of accessibility of consumer health information Web sites; and to investigate the possible relationships between accessibility and other features of the Web sites, including function, popularity and importance., Methods: We carried out a cross-sectional study of the state of accessibility of health information Web sites to people with disabilities. We selected 108 consumer health information Web sites from the directory service of a Web search engine. A measurement framework was constructed to automatically measure the level of Web Accessibility Barriers (WAB) of Web sites following Web accessibility specifications. We investigated whether there was a difference between WAB scores across various functional categories of the Web sites, and also evaluated the correlation between the WAB and Alexa traffic rank and Google Page Rank of the Web sites., Results: We found that none of the Web sites we looked at are completely accessible to people with disabilities, i.e., there were no sites that had no violation of Web accessibility rules. However, governmental and educational health information Web sites do exhibit better Web accessibility than the other categories of Web sites (P < 0.001). We also found that the correlation between the WAB score and the popularity of a Web site is statistically significant (r = 0.28, P < 0.05), although there is no correlation between the WAB score and the importance of the Web sites (r = 0.15, P = 0.111)., Conclusions: Evaluation of health information Web sites shows that no Web site scrupulously abides by Web accessibility specifications, even for entities mandated under relevant laws and regulations. Government and education Web sites show better performance than Web sites among other categories. Accessibility of a Web site may have a positive impact on its popularity in general. However, the Web accessibility of a Web site may not have a significant relationship with its importance on the Web.

Background: Many consumers use the Internet to find information about their medicines. It is widely acknowledged that health information on the Internet is of variable quality and therefore the search and appraisal skills of consumers are important for selecting and assessing this information. The way consumers choose and evaluate information on medicines on the Internet is important because it has been shown that written information on medicines can influence consumer attitudes to and use of medicines., Objective: To explore consumer experiences in searching for and appraising Internet-based information on medicines., Methods: Six focus groups (N = 46 participants) were conducted in metropolitan Sydney, Australia from March to May 2003 with consumers who had used the Internet for information on medicines. Verbatim transcripts of the group discussions were analyzed using a grounded theory approach., Results: All participants reported using a search engine to find information on medicines. Choice of search engine was determined by factors such as the workplace or educational environments, or suggestions by family or friends. Some participants found information solely by typing the medicine name (drug or brand name) into the search engine, while others searched using broader terms. Search skills ranged widely from more-advanced (using quotation marks and phrases) to less-than-optimal (such as typing in questions and full sentences). Many participants selected information from the first page of search results by looking for keywords and descriptions in the search results, and by looking for the source of the information as apparent in the URL. Opinions on credible sources of information on medicines varied with some participants regarding information by pharmaceutical companies as the "official" information on a medicine, and others preferring what they considered to be impartial sources such as governments, organizations, and educational institutions. It was clear that although most participants were skeptical of trusting information on the Internet, they had not paid conscious attention to how they selected information on medicines. Despite this, it was evident that participants viewed the Internet as an important source for information on medicines., Conclusions: The results showed that there was a range of search and appraisal skills among participants, with many reporting a limited awareness of how they found and evaluated Internet-based information on medicines. Poor interpretation of written information on medicines has been shown to lead to anxiety and poor compliance to therapy. This issue is more important for Internet-based information since it is not subject to quality control and standardization as is written information on medicines. Therefore, there is a need for promoting consumer search and appraisal skills when using this information. Educating consumers in how to find and interpret Internet-based information on medicines may help them use their medicines in a safer and more-effective way.

Background: Most existing tools for measuring the quality of Internet health information focus almost exclusively on structural criteria or other proxies for quality of information, rather than evaluating information accuracy and comprehensiveness., Objective: This research sought to build a conceptual framework that could lay the groundwork for a robust performance-measurement system for evaluating the quality of Internet health information., Methods: Application of the quality-of-care measurement paradigm to developing a conceptual framework for defining and evaluating the quality of diabetes consumer-information Web sites., Results: Performance measures related to accuracy and comprehensiveness of information can be added to structural criteria to provide a more-robust approach to Web site evaluation., Conclusions: The development and implementation of a reliable and valid method for evaluating the quality of Internet health sites could provide lay people with a tool to identify useful content more easily and distinguish between beneficial and misleading information.

Background: Most existing tools for measuring the quality of Internet health information focus almost exclusively on structural criteria or other proxies for quality information rather than evaluating actual accuracy and comprehensiveness., Objective: This research sought to develop a new performance-measurement tool for evaluating the quality of Internet health information, test the validity and reliability of the tool, and assess the variability in diabetes Web site quality., Methods: An objective, systematic tool was developed to evaluate Internet diabetes information based on a quality-of-care measurement framework. The principal investigator developed an abstraction tool and trained an external reviewer on its use. The tool included 7 structural measures and 34 performance measures created by using evidence-based practice guidelines and experts' judgments of accuracy and comprehensiveness., Results: Substantial variation existed in all categories, with overall scores following a normal distribution and ranging from 15% to 95% (mean was 50% and median was 51%). Lin's concordance correlation coefficient to assess agreement between raters produced a rho of 0.761 (Pearson's r of 0.769), suggesting moderate to high agreement. The average agreement between raters for the performance measures was 0.80., Conclusions: Diabetes Web site quality varies widely. Alpha testing of this new tool suggests that it could become a reliable and valid method for evaluating the quality of Internet health sites. Such an instrument could help lay people distinguish between beneficial and misleading information.

Background: The Web has become an important and influential source of health information. With the vast number of Web sites on the Internet, users often resort to popular search sites when searching for information. However, little is known about the characteristics of Web sites returned by simple Web searches for information about smoking cessation for teenagers., Objective: To determine the characteristics of Web sites retrieved by search engines about smoking cessation for teenagers and how information quality correlates with the search ranking., Methods: The top 30 sites returned by 4 popular search sites in response to the search terms "teen quit smoking" were examined. The information relevance and quality characteristics of these sites were evaluated by 2 raters. Objective site characteristics were obtained using a page-analysis Web site., Results: Only 14 of the 30 Web sites are of direct relevance to smoking cessation for teenagers. The readability of about two-thirds of the 14 sites is below an eighth-grade school level and they ranked significantly higher (Kendall rank correlation, tau = -0.39, P =.05) in search-site results than sites with readability above or equal to that grade level. Sites that ranked higher were significantly associated with the presence of e-mail address for contact (tau = -0.46, P =.01), annotated hyperlinks to external sites (tau = -0.39, P =.04), and the presence of meta description tag (tau = -0.48, P =.002). The median link density (number of external sites that have a link to that site) of the Web pages was 6 and the maximum was 735. A higher link density was significantly associated with a higher rank (tau = -0.58, P =.02)., Conclusions: Using simple search terms on popular search sites to look for information on smoking cessation for teenagers resulted in less than half of the sites being of direct relevance. To improve search efficiency, users could supplement results obtained from simple Web searches with human-maintained Web directories and learn to refine their searches with more advanced search syntax.

Background: Public use of the Internet for health information is increasing but its effect on health care is unclear. We studied physicians' experience of patients looking for health information on the Internet and their perceptions of the impact of this information on the physician-patient relationship, health care, and workload., Methods: Cross-sectional survey of a nationally-representative sample of United States physicians (1050 respondents; response rate 53%)., Results: Eighty-five percent of respondents had experienced a patient bringing Internet information to a visit. The quality of information was important: accurate, relevant information benefited, while inaccurate or irrelevant information harmed health care, health outcomes, and the physician-patient relationship. However, the physician's feeling that the patient was challenging his or her authority was the most consistent predictor of a perceived deterioration in the physician-patient relationship (OR = 14.9; 95% CI, 5.5-40.5), in the quality of health care (OR = 3.4; 95% CI, 1.1-10.9), or health outcomes (OR = 5.6; 95% CI, 1.7-18.7). Thirty-eight percent of physicians believed that the patient bringing in information made the visit less time efficient, particularly if the patient wanted something inappropriate (OR = 2.5; 95% CI, 1.5-4.4), or the physician felt challenged (OR = 3.6; 95% CI, 1.8-7.2)., Conclusions: The quality of information on the Internet is paramount: accurate relevant information is beneficial, while inaccurate information is harmful. Physicians appear to acquiesce to clinically-inappropriate requests generated by information from the Internet, either for fear of damaging the physician-patient relationship or because of the negative effect on time efficiency of not doing so. A minority of physicians feels challenged by patients bringing health information to the visit; reasons for this require further research.

Background: There have been many studies showing the variable quality of Internet health information and it has often been assumed that patients will blindly follow this and frequently come to harm. There have also been reports of problems for doctors and health services following patient Internet use, but their frequency has not been quantified. However, there have been no large, rigorous surveys of the perceptions of Internet-aware doctors about the actual benefits and harms to their patients of using the Internet., Objective: To describe Internet-literate doctors' experiences of their patients' use of the Internet and resulting benefits and problems., Methods: Online survey to a group of 800 Web-using doctors (members of a UK medical Internet service provider, Medix) in September and October 2001., Results: Responses were received from 748 (94%) doctors, including 375 general practitioners (50%). Respondents estimated that 1%-2% of their patients used the Internet for health information in the past month with no regional variation. Over two thirds of the doctors considered Internet health information to be usually (20%) or sometimes (48%) reliable; this was higher in those recently qualified. Twice as many reported patients experiencing benefits (85%; 95% confidence interval, 80%-90%) than problems (44%; 95% confidence interval, 37%-50%) from the Internet. Patients gaining actual physical benefits from Internet use were reported by 40% of respondents, while 8% reported physical harm. Patients' overall experiences with the Internet were judged excellent 1%, good 29%, neutral 62%, poor 9%, or bad <1%. Turning to the impact of patient Internet use on the doctors themselves, 13% reported no problems, 38% 1 problem, and 49% 2 or more problems. Conversely, 20% reported no benefits for themselves, 49% 1 benefit, and 21% 2 or more benefits., Conclusions: These doctors reported patient benefits from Internet use much more often than harms, but there were more problems than benefits for the doctors themselves. Reported estimates of patient Internet usage rates were low. Overall, this survey suggests that patients are deriving considerable benefits from using the Internet and that some of the claimed risks seem to have been exaggerated.

Background: The use of medical experts in rating the content of health-related sites on the Internet has flourished in recent years. In this research, it has been common practice to use a single medical expert to rate the content of the Web sites. In many cases, the expert has rated the Internet health information as poor, and even potentially dangerous. However, one problem with this approach is that there is no guarantee that other medical experts will rate the sites in a similar manner., Objectives: The aim was to assess the reliability of medical experts' judgments of threads in an Internet newsgroup related to a common disease. A secondary aim was to show the limitations of commonly-used statistics for measuring reliability (eg, kappa)., Methods: The participants in this study were 5 medical doctors, who worked in a specialist unit dedicated to the treatment of the disease. They each rated the information contained in newsgroup threads using a 6-point scale designed by the experts themselves. Their ratings were analyzed for reliability using a number of statistics: Cohen's kappa, gamma, Kendall's W, and Cronbach's alpha., Results: Reliability was absent for ratings of questions, and low for ratings of responses. The various measures of reliability used gave conflicting results. No measure produced high reliability., Conclusions: The medical experts showed a low agreement when rating the postings from the newsgroup. Hence, it is important to test inter-rater reliability in research assessing the accuracy and quality of health-related information on the Internet. A discussion of the different measures of agreement that could be used reveals that the choice of statistic can be problematic. It is therefore important to consider the assumptions underlying a measure of reliability before using it. Often, more than one measure will be needed for "triangulation" purposes.

Background: The MEDICOM (Medical Products Electronic Commerce) Portal provides the electronic means for medical-equipment manufacturers to communicate online with their customers while supporting the Purchasing Process and Post Market Surveillance. The Portal offers a powerful Internet-based search tool for finding medical products and manufacturers. Its main advantage is the fast, reliable and up-to-date retrieval of information while eliminating all unrelated content that a general-purpose search engine would retrieve. The Universal Medical Device Nomenclature System (UMDNS) registers all products. The Portal accepts end-user requests and generates a list of results containing text descriptions of devices, UMDNS attribute values, and links to manufacturer Web pages and online catalogues for access to more-detailed information. Device short descriptions are provided by the corresponding manufacturer. The Portal offers technical support for integration of the manufacturers Web sites with itself. The network of the Portal and the connected manufacturers sites is called the MEDICOM system., Objectives: To establish an environment hosting all the interactions of consumers (health care organizations and professionals) and providers (manufacturers, distributors, and resellers of medical devices)., Methods: The Portal provides the end-user interface, implements system management, and supports database compatibility. The Portal hosts information about the whole MEDICOM system (Common Database) and summarized descriptions of medical devices (Short Description Database); the manufacturers servers present extended descriptions. The Portal provides end-user profiling and registration, an efficient product-searching mechanism, bulletin boards, links to on-line libraries and standards, on-line information for the MEDICOM system, and special messages or advertisements from manufacturers. Platform independence and interoperability characterize the system design. Relational Database Management Systems are used for the system s databases. The end-user interface is implemented using HTML, Javascript, Java applets, and XML documents. Communication between the Portal and the manufacturers servers is implemented using a CORBA interface. Remote administration of the Portal is enabled by dynamically-generated HTML interfaces based on XML documents. A representative group of users evaluated the system. The aim of the evaluation was validation of the usability of all of MEDICOM s functionality. The evaluation procedure was based on ISO/IEC 9126 Information technology - Software product evaluation - Quality characteristics and guidelines for their use., Results: The overall user evaluation of the MEDICOM system was very positive. The MEDICOM system was characterized as an innovative concept that brings significant added value to medical-equipment commerce., Conclusions: The eventual benefits of the MEDICOM system are (a) establishment of a worldwide-accessible marketplace between manufacturers and health care professionals that provides up-to-date and high-quality product information in an easy and friendly way and (b) enhancement of the efficiency of marketing procedures and after-sales support.

Background: Hospital homepages should provide comprehensive information on the hospital's services, such as departments and treatments available, prices, waiting time, leisure facilities, and other information important for patients and their relatives. Norway, with its population of approximately 4.3 million, ranks among the top countries globally for its ability to absorb and use technology. It is unclear to what degree Norwegian hospitals and patients use the Internet for information about health services., Objectives: This study was undertaken to evaluate the quality of the biggest Norwegian cancer hospitals Web sites and to gather some preliminary data on patients use of the Internet., Methods: In January 2001, we analyzed Web sites of 5 of the 7 biggest Norwegian hospitals treating cancer patients using a scoring system. The scoring instrument was based on recommendations developed by the Norwegian Central Information Service for Web sites and reflects the scope and depth of service information offered on hospital Web pages. In addition, 31 cancer patients visiting one hospital-based medical oncologist were surveyed about their use of the Internet., Results: Of the 7 hospitals, 5 had a Web site. The Web sites differed markedly in quality. Types of information included - and number of Web sites that included each type of information - were, for example: search option, 1; interpreter service, 2; date of last update, 2; postal address, phone number, and e-mail service, 3; information in English, 2. None of the Web sites included information on waiting time or prices. Of the 31 patients surveyed, 12 had personal experience using the Internet and 4 had searched for medical information. The Internet users were significantly younger (mean age 47.8 years, range 28.4-66.8 years) than the nonusers (mean age 61.8 years, range 33.1-90.0 years) (P = 0.007)., Conclusions: The hospitals Web sites offer cancer patients and relatives useful information, but the Web sites were not impressive.

Background: The massive growth of health information on the Internet; the global nature of the Internet; the seismic shift taking place in the relationships of various actors in this arena, and the absence of real protection from harm for citizens who use the Internet for health purposes are seen to be real problems. One response to many of these problems has been the burgeoning output of codes of conduct by numerous organizations trying to address quality of health information., Objectives: Review the major self-regulatory initiatives in the English-speaking world to develop quality and ethical standards for health information on the Internet. Compare and analyze the approaches taken by the different initiatives. Clarify the issues around the development and enforcement of standards., Methods: Quality initiatives selected meet one or more of the following criteria: Self-regulatory. A reasonable constituency. Diversity (eg, of philosophy, approach and process)-to achieve balance and wide representation, and to illustrate and compare different approaches. Historic value. A wider reach than a national audience, except when its reach is a significant sector of the Internet health information industry. The initiatives were compared in 3 ways: (1) Analysis and comparison of: key concepts, mechanism, or approach. Analysis of: the obligations that a provider has to meet to comply with the given initiative, the intended beneficiaries of that initiative, and the burdens imposed on different actors. These burdens are described in terms of their effect on the long-term sustainability and maintenance of the initiative by its developers. Analysis of the enforcement mechanisms. (2) Analysis and comparison by type of sponsoring organization, the reach of the initiative, and the sources of funding of the initiative or the sponsoring organization. (3) How the various initiatives fall under 1 of 3 key mechanisms and comparison of the advantages and disadvantages of these key mechanisms., Results: The issues that affect the initiatives and future work on the quality of health information on the Internet are identified and analyzed. These issues are: (a) Three key mechanisms used in the quality initiatives (b) Sustainability issues that affect the initiatives: Burdens placed on health information providers, citizens and others. Currency and maintenance issues of the initiatives. Funding. Cost. Acceptance. Market conditions. User indifference or ambivalence. (c) Enforcement issues surrounding the initiatives (d) Adequacy of approach, scope, reach, and enforcement provisions of the various quality initiatives (e) Gaps that need to be addressed to achieve good quality of health information on the internet, Conclusions: Ten conclusions are presented. A framework of action to be undertaken by the World Health Organization in the field of quality of health information on the Internet is recommended.

Background: Digital information technology can facilitate informed decision making by individuals regarding their personal health care. The digital divide separates those who do and those who do not have access to or otherwise make use of digital information. To close the digital divide, health care communications research must address a fundamental issue, the consumer vocabulary problem: consumers of health care, at least those who are laypersons, are not always familiar with the professional vocabulary and concepts used by providers of health care and by providers of health care information, and, conversely, health care and health care information providers are not always familiar with the vocabulary and concepts used by consumers. One way to address this problem is to develop a consumer entry vocabulary for health care communications., Objectives: To evaluate the potential of controlled vocabulary resources for supporting the development of consumer entry vocabulary for diabetes., Methods: We used folk medical terms from the Dictionary of American Regional English project to create extended versions of 3 controlled vocabulary resources: the Unified Medical Language System Metathesaurus, the Eurodicautom of the European Commission's Translation Service, and the European Commission Glossary of popular and technical medical terms. We extracted consumer terms from consumer-authored materials, and physician terms from physician-authored materials. We used our extended versions of the vocabulary resources to link diabetes-related terms used by health care consumers to synonymous, nearly-synonymous, or closely-related terms used by family physicians. We also examined whether retrieval of diabetes-related World Wide Web information sites maintained by nonprofit health care professional organizations, academic organizations, or governmental organizations can be improved by substituting a physician term for its related consumer term in the query., Results: The Dictionary of American Regional English extension of the Metathesaurus provided coverage, either direct or indirect, of approximately 23% of the natural language consumer-term-physician-term pairs. The Dictionary of American Regional English extension of the Eurodicautom provided coverage for 16% of the term pairs. Both the Metathesaurus and the Eurodicautom indirectly related more terms than they directly related. A high percentage of covered term pairs, with more indirectly covered pairs than directly covered pairs, might be one way to make the most out of expensive controlled vocabulary resources. We compared retrieval of diabetes-related Web information sites using the physician terms to retrieval using related consumer terms We based the comparison on retrieval of sites maintained by non-profit healthcare professional organizations, academic organizations, or governmental organizations. The number of such sites in the first 20 Results from a search was increased by substituting a physician term for its related consumer term in the query. This suggests that the Dictionary of American Regional English extensions of the Metathesaurus and Eurodicautom may be used to provide useful links from natural language consumer terms to natural language physician terms., Conclusions: The Dictionary of American Regional English extensions of the Metathesaurus and Eurodicautom should be investigated further for support of consumer entry vocabulary for diabetes.

Background: Little is known about to what extent patients who underwent medical treatment access the Internet and whether they benefit from consulting the Internet., Objective: To understand if cardiopathic patients use the Internet for health-related information and whether they find retrieved information understandable and useful., Methods: Telephone interviews, using a semi-structured questionnaire, were conducted with 82 patients who had undergone off-pump coronary-artery bypass grafting at the Center for Less Invasive and Robotic Heart Surgery in Buffalo, New York, USA. Study design was multidisciplinary, combining expertise of medical and communication science. Sources of medical information were identified (doctor, Internet, magazines, newspapers, television, radio, family members). Accessibility, quality, and readability of Internet medical information from the patients point of view were investigated., Results: Out of 82 patients, 35 (35/82, 42.7%) were Internet users. Internet users had a significantly higher education level than Internet non-users (college education: 42.9% of users, 10.6% of non-users; P <.001). Among the Internet users, 18 (18/35, 51.4%) had used the Internet for retrieving medical information; 17 (17/35, 48.6%) had not. No statistically significant differences in demographic data were found when comparing these 2 sub-groups of patients. Family-members involvement was high (15/18, 83.3%). Internet medical information was rated helpful in most cases; readability was acceptable for only 3 patients (3/18, 16.7%). To improve on-line medical information, all patients interviewed suggested sites designed by their physicians., Conclusions: Although 1 in 5 patients in our sample has used the Internet to retrieve medical information, the majority of them experiences difficulties comprehending the information retrieved. Health-care providers should provide Internet medical information that is adequate for the non-medical public's needs.

Background: Internet technology is transforming the general approach to communication and dissemination of information in the field of healthcare. However, it is also creating problems in terms of finding information, and knowing what credibility to place on the information found. The chaotic nature of the World Wide Web (WWW) and the simplistic approach adopted by search engines can make the task of finding relevant information difficult, and the user can waste considerable amounts of time on the process. Even when information is found, there is no general quality assurance process that can guarantee the credibility of the resulting information., Objective: The aim of this research was to develop an approach for establishing co-operative health information networks (CHINs) with different focuses, which can be used in different parts of Europe. The resulting CHINs would provide organised healthcare information and support comprehensive and integrated sets of healthcare telematic services for a broad range of users. Such developments would reduce the difficulties of finding information and knowing what credibility to ascribe to it., Methods: A common approach has been developed based on drawing together contributions from the major healthcare service providers in the region. Standard structures are recommended so that information is presented in a uniform way. Appropriate mechanisms ensure adequate security and a level of quality assurance for the end user., Results: Since 1996, CHINs have been developed in six European countries as part of a European Union (EU) project. This paper presents the overall approach adopted, and the achievements in two different regions of Europe (Greece and Scotland). Although the circumstances in these two regions are very different, in both cases the resulting CHIN has proved successful., Conclusions: CHINs offer a solution to the difficulty of finding relevant health information on the Internet and guaranteeing its credibility. They can be used in different ways in different regions, and have major benefits for both information providers and end users.

The Internet is changing how people receive health information and health care. All who use the Internet for health-related purposes must join together to create an environment of trusted relationships to assure high quality information and services; protect privacy; and enhance the value of the Internet for both consumers and providers of health information, products, and services. The goal of the e-Health Code of Ethics is to ensure that people worldwide can confidently and with full understanding of known risks realise the potential of the Internet in managing their own health and the health of those in their care. The final e-Health Code of Ethics, presented in this paper, has been prepared as a result of the "e-Health Ethics Summit," which convened in Washington DC on 31 January 2000 - 2 February 2000. The summit, organized by the Internet Healthcare Coalition and hosted by the World Health Organisation/Pan-American Health Organisation (WHO/PAHO), was attended by a panel of about 50 invited experts from all over the world and produced the foundation for a draft code, which was released 18 February [1] for an online public consultation period which ended on 14 April 2000. The final Washington e-Health Code of Ethics sets forth guiding principles under eight main headings: candor; honesty; quality; informed consent; privacy; professionalism in online health care; responsible partnering; and accountability.

MedCERTAIN (MedPICS Certification and Rating of Trustworthy Health Information on the Net, http://www.medcertain.org/) is a recently launched international project funded under the European Union's (EU) "Action Plan for safer use of the Internet". It provides a technical infrastructure and a conceptual basis for an international system of "quality seals", ratings and self-labelling of Internet health information, with the final aim to establish a global "trustmark" for networked health information. Digital "quality seals" are evaluative metadata (using standards such as PICS=Platform for Internet Content Selection, now being replaced by RDF/XML) assigned by trusted third-party raters. The project also enables and encourages self-labelling with descriptive metainformation by web authors. Together these measures will help consumers as well as professionals to identify high-quality information on the Internet. MedCERTAIN establishes a fully functional demonstrator for a self- and third-party rating system enabling consumers and professionals to filter harmful health information and to positively identify and select high quality information. We aim to provide a trustmark system which allows citizens to place greater confidence in networked information, to encourage health information providers to follow best practices guidelines such as the Washington eHealth Code of Ethics, to provide effective feedback and law enforcement channels to handle user complaints, and to stimulate medical societies to develop standard for patient information. The project further proposes and identifies standards for interoperability of rating and description services (such as libraries or national health portals) and fosters a worldwide collaboration to guide consumers to high-quality information on the web.

Background: The Internet offers a great amount of health related websites, but concern has been raised about their reliability. Several subjective evaluation criteria and websites rating systems have been proposed as a help for the Internet users to distinguish among web resources with different quality, but their efficacy has not been proven., Objectives: To evaluate the agreement of a subset of Internet rating systems editorial boards regarding their evaluations of a sample of pediatric websites. To evaluate certain websites characteristics as possible quality indicators for pediatric websites., Methods: Comparative survey of the Results of systematic evaluations of the contents and formal aspects of a sample of pediatric websites, with the number of daily visits to those websites, the time since their last update, the impact factor of their authors or editors, and the number of websites linked to them., Results: 363 websites were compiled from eight rating systems. Only 25 were indexed and evaluated by at least two rating systems. This subset included more updated and more linked websites. There was no correlation among the Results of the evaluation of these 25 websites by the rating systems. The number of inbound links to the websites significantly correlated with their updating frequency (p<.001), with the number of daily visits (p=.005), and with the Results of their evaluation by the largest rating system, HealthAtoZ (p<.001). The websites updating frequency also significantly correlated with the RESULTS of the websites evaluation by HealthAtoZ, both about their contents (p=.001) and their total values (p<.05). The number of daily visits significantly correlated (p<.05) with the Results of the evaluations by Medical Matrix., Conclusions: Some websites characteristics as the number of daily visits, their updating frequency and, overall, the number of websites linked to them, correlate with their evaluation by some of the largest rating systems on the Internet, what means that certain indexes obtained from the usage analysis of pediatric websites could be used as quality indicators. On the other hand, the citation analysis on the Web by the quantification of inbound links to medical websites could be an objective and feasible tool in rating great amounts of websites.