Search

Health information technology (HIT) is one of the most significant developments in health care in recent years. However, there is still a large gap between how HIT could support clinical work versus how it does. In this project, we developed a visionary scenario to identify opportunities for improving patient care in dentistry. In the scenario, patients and care providers are supported by a ubiquitous, embedded computing infrastructure that captures and processes data streams from multiple sources. Practical decision support, as well as automated background data processing (e.g., to screen for common conditions), helps clinicians provide quality care. A holistic view of clinical information technology (IT) focuses on supporting clinicians and patients in a user-centered manner. While clinical IT is still in very much a work in progress, scenarios such as the one presented may be helpful to keep us focused on the possibilities of tomorrow, not on the limitations of today.

Objective: Oral and pharyngeal cancers are responsible for over 7,600 deaths each year in the United States. Given the significance of the disease and the fact that many individuals increasingly rely on health information on the Internet, it is important that patients and others can access clear and accurate oral cancer information on the Web. The objective of this study was threefold: a) develop an initial method to evaluate surface and content quality of selected English- and Spanish-language oral cancer Web sites; b) conduct a pilot evaluation; and c) discuss implications of our findings for dental public health. Methods: We developed a search strategy to find oral cancer sites frequented by the public using Medline Plus, Google, and Yahoo in English and Spanish. We adapted the Information Quality Tool (IQT) to perform a surface evaluation and developed a novel tool to evaluate site content for 24 sites each in English and Spanish. Results: English-language sites had an average IQT score of 76.6 (out of 100) and an average content score of 52.1 (out of 100). Spanish-language sites had an average IQT score of 50.3 and an average content score of 25.6. Conclusions: The study produced a quality assessment of oral cancer Web sites useful for clinicians and patients. Sites provided more information on clinical presentation, and etiology, and risk factors, than other aspects of oral cancer. The surface and quality of Spanish-language sites was low, possibly putting Hispanic populations at a disadvantage regarding oral cancer information on the Web. © 2011 American Association of Public Health Dentistry.

Unlike expertise location systems which users query actively when looking for an expert, expert recommender systems suggest individuals without the context of a specific problem. An interesting research question is whether expert recommender systems should consider a users' social context when recommending potential research collaborators. One may argue that it might be easier for scientists to collaborate with colleagues in their social network, because initiating collaboration with socially unconnected researchers is burdensome and fraught with risk, despite potentially relevant expertise. However, many scientists also initiate collaborations outside of their social network when they seek to work with individuals possessing relevant expertise or acknowledged experts. In this paper, we studied how well content-based, social and hybrid recommendation algorithms predicted co-author relationships among a random sample of 17,525 biomedical scientists. To generate recommendations, we used authors' research expertise inferred from publication metadata and their professional social networks derived from their coauthorship history. We used 80% of our data set (articles published before 2007) as our training set, and the remaining data as our test set (articles published in 2007 or later). Our results show that a hybrid algorithm combining expertise and social network information outperformed all other algorithms with regards to Top 10 and Top 20 recommendations. For the Top 2 and Top 5 recommendations, social network-based information alone generated the most useful recommendations. Our study provides evidence that integrating social network information in expert recommendations may outperform a purely expertise-based approach.

Background. A major challenge in designing useful clinical information systems in dentistry is to incorporate clinical evidence based on dentists' information needs and then integrate the system seamlessly into the complex clinical workflow. However, little is known about the actual information needs of dentists during treatment sessions. The purpose of this study is to identify general dentists' information needs and the information sources they use to meet those needs in clinical settings so as to inform the design of dental information systems. Methods. A semi-structured interview was conducted with a convenience sample of 18 general dentists in the Pittsburgh area during clinical hours. One hundred and five patient cases were reported by these dentists. Interview transcripts were coded and analyzed using thematic analysis with a constant comparative method to identify categories and themes regarding information needs and information source use patterns. Results. Two top-level categories of information needs were identified: foreground and background information needs. To meet these needs, dentists used four types of information sources: clinical information/tasks, administrative tasks, patient education and professional development. Major themes of dentists' unmet information needs include: (1) timely access to information on various subjects; (2) better visual representations of dental problems; (3) access to patient-specific evidence-based information; and (4) accurate, complete and consistent documentation of patient records. Resource use patterns include: (1) dentists' information needs matched information source use; (2) little use of electronic sources took place during treatment; (3) source use depended on the nature and complexity of the dental problems; and (4) dentists routinely practiced cross-referencing to verify patient information. Conclusions. Dentists have various information needs at the point of care. Among them, the needs for better visual re

In this paper, we examine the degree of difference between two types of metadata for biomedical articles generated by different groups of people. The first type of metadata is social tags, which are assigned to articles by their readers using uncontrolled vocabulary. The second type is index terms, which are assigned by professionally trained indexers and domain experts using a controlled vocabulary. When the two kinds of metadata are assigned to the same item, we may expect that they overlap to a large extent and could substitute for one another. In this study, we compared social tags and index terms for a set of papers that appear both in CiteULike and MEDLINE, and assessed their differences. Due to the idiosyncratic nature of social tags, we preprocessed the tags through normalization, stop-word removal, stemming and spell-checking. Our results show that social tags and Medical Subject Heading (MeSH) index have little overlap and embody largely heterogeneous understanding of items. © 2010 ACM.

Information overload is an often-cited phenomenon that reduces the productivity, efficiency and efficacy of scientists. One challenge for scientists is to find appropriate collaborators in their research. The literature describes various solutions to the problem of expertise location, but most current approaches do not appear to be very suitable for expert recommendations in biomedical research. In this study, we present the development and initial evaluation of a vector space model-based algorithm to calculate researcher similarity using four inputs: 1) MeSH terms of publications; 2) MeSH terms and author rank; 3) exploded MeSH terms; and 4) exploded MeSH terms and author rank. We developed and evaluated the algorithm using a data set of 17,525 authors and their 22,542 papers. On average, our algorithms correctly predicted 2.5 of the top 5/10 coauthors of individual scientists. Exploded MeSH and author rank outperformed all other algorithms in accuracy, followed closely by MeSH and author rank. Our results show that the accuracy of MeSH term-based matching can be enhanced with other metadata such as author rank.

Objective This study's objective was to formally describe the work process for charting and treatment planning in general dental practice to inform the design of a new clinical computing environment.Methods Using a process called contextual inquiry, researchers observed 23 comprehensive examination and treatment planning sessions during 14 visits to 12 general US dental offices. For each visit, field notes were analysed and reformulated as formalised models. Subsequently, each model type was consolidated across all offices and visits. Interruptions to the workflow, called breakdowns, were identified.Results Clinical work during dental examination and treatment planning appointments is a highly collaborative activity involving dentists, hygienists and assistants. Personnel with multiple overlapping roles complete complex multi-step tasks supported by a large and varied collection of equipment, artifacts and technology. Most of the breakdowns were related to technology which interrupted the workflow, caused rework and increased the number of steps in work processes.Conclusion Current dental software could be significantly improved with regard to its support for communication and collaboration, workflow, information design and presentation, information content, and data entry.

Usability methods, such as heuristic evaluation, cognitive walk-throughs and user testing, are increasingly used to evaluate and improve the design of clinical software applications. There is still some uncertainty, however, as to how those methods can be used to support the development process and evaluation in the most meaningful manner. In this study, we compared the results of a heuristic evaluation with those of formal user tests in order to determine which usability problems were detected by both methods. We conducted heuristic evaluation and usability testing on four major commercial dental computer-based patient records (CPRs), which together cover 80% of the market for chairside computer systems among general dentists. Both methods yielded strong evidence that the dental CPRs have significant usability problems. An average of 50% of empirically-determined usability problems were identified by the preceding heuristic evaluation. Some statements of heuristic violations were specific enough to precisely identify the actual usability problem that study participants encountered. Other violations were less specific, but still manifested themselves in usability problems and poor task outcomes. In this study, heuristic evaluation identified a significant portion of problems found during usability testing. While we make no assumptions about the generalizability of the results to other domains and software systems, heuristic evaluation may, under certain circumstances, be a useful tool to determine design problems early in the development cycle. © 2009 ITCH 2009 Steering Committee and IOS Press. All rights reserved.

Natural language processing applications that extract information from text rely on semantic representations. The objective of this paper is to describe a methodology for creating a semantic representation for information that will be automatically extracted from textual clinical records. We illustrate two of the four steps of the methodology in this paper using the case study of encoding information from dictated dental exams: (1) develop an initial representation from a set of training documents and (2) iteratively evaluate and evolve the representation while developing annotation guidelines. Our approach for developing and evaluating a semantic representation is based on standard principles and approaches that are not dependent on any particular domain or type of semantic representation.

During any course of study, students are assessed usually through a range of methods which may include written examinations, coursework assignments, professional practice, oral tests and practical examinations. This article considers the various forms of assessment in dental education and how information and communication technology is being applied to them. As innovative teaching and learning methods such as computer simulations are introduced, the assessment of results, successes and failures is taking on new forms in many traditional courses. The web is also spreading its tentacles into assessment, with the benefits of offering almost instant feedback and support. However, technology brings its own problems, not least by making ever more ingenious methods of plagiarism easier. Educational establishments, therefore, must be aware of such problems and have policies in place to counteract them. © 2008 Macmillan Publishers Limited. All rights reserved.

Background: As biomedical research projects become increasingly interdisciplinary and complex, collaboration with appropriate individuals, teams, and institutions becomes ever more crucial to project success. While social networks are extremely important in determining how scientific collaborations are formed, social networking technologies have not yet been studied as a tool to help form scientific collaborations. Many currently emerging expertise locating systems include social networking technologies, but it is unclear whether they make the process of finding collaborators more efficient and effective. Objective: This study was conducted to answer the following questions: (1) Which requirements should systems for finding collaborators in biomedical science fulfill? and (2) Which information technology services can address these requirements? Methods: The background research phase encompassed a thorough review of the literature, affinity diagramming, contextual inquiry, and semistructured interviews. This phase yielded five themes suggestive of requirements for systems to support the formation of collaborations. In the next phase, the generative phase, we brainstormed and selected design ideas for formal concept validation with end users. Then, three related, well-validated ideas were selected for implementation and evaluation in a prototype. Results: Five main themes of systems requirements emerged: (1) beyond expertise, successful collaborations require compatibility with respect to personality, work style, productivity, and many other factors (compatibility); (2) finding appropriate collaborators requires the ability to effectively search in domains other than your own using information that is comprehensive and descriptive (communication); (3) social networks are important for finding potential collaborators, assessing their suitability and compatibility, and establishing contact with them (intermediation); (4) information profiles must be complete, correct, u

Background. The usability of dental computer-based patient record (CPR) systems has not been studied, despite early evidence that poor usability is a problem for dental CPR system users at multiple levels. Methods. The authors conducted formal usability tests of four dental CPR systems by using a purposive sample of four groups of five novice users. The authors measured task outcomes (correctly completed, incorrectly completed and incomplete) in each CPR system while the participants performed nine clinical documentation tasks, as well as the number of usability problems identified in each CPR system and their potential relationship to task outcomes. The authors reviewed the software application design aspects responsible for these usability problems. Results. The range for correctly completed tasks was 16 to 64 percent, for incorrectly completed tasks 18 to 38 percent and for incomplete tasks 9 to 47 percent. The authors identified 286 usability problems. The main types were three unsuccessful attempts, negative affect and task incorrectly completed. They also identified six problematic interface and interaction designs that led to usability problems. Conclusion. The four dental CPR systems studied have significant usability problems for novice users, resulting in a steep learning curve and potentially reduced system adoption. Clinical Implications. The significant number of data entry errors raises concerns about the quality of documentation in clinical practice.

Choosing the most appropriate collaborators is becoming increasingly crucial to biomedical research as many research questions evolve into complex and multidisciplinary projects. Despite a growing emphasis on translational and interdisciplinary research, little is known about how biomedical researchers form collaborations. We conducted 27 semistructured interviews with scientists from the University of Pittsburgh, used grounded theory methodology to identify major themes, and compared these themes to the literature in order to develop a model of how biomedical researchers establish collaborations. We identify and discuss four major aspects of collaboration: motivation for collaboration, evaluation of prospective collaboration partners, search and selection, and barriers to collaboration formation. © 2008 IEEE.

Recent trends in science are increasing the need for researchers to form collaborations. To date, however, electronic systems have played only a minor role in helping scientists do so. This study used a literature review, and contextual inquiries and semistructured interviews with biomedical scientists to develop a preliminary set of requirements for electronic systems designed to help optimize how biomedical researchers choose collaborators. We then reviewed the requirements in light of emerging research on expertise location using the Semantic Web. The requirements include aspects such as comprehensive, complete and up-to-date online profiles that are easy to create and maintain; the ability to exploit social networks when searching for collaborators; information to help gauge the compatibility of personalities and work styles; and recommendations for effective searching and making "non-intuitive" connections between researchers. The Semantic Web offers significant opportunities for operationalizing the requirements, for instance through aggregating profile data from disparate sources, annotating contributions to social media using methods such as Semantically Interlinked Online Communities, and concept-based querying using ontologies. Future work should validate the preliminary requirements and explore in detail how the Semantic Web can help address them.

Objective: Approximately 25% of all general dentists practicing in the United States use a computer in the dental operatory. Only 1.8% maintain completely electronic records. Anecdotal evidence suggests that dental computer-based patient records (CPR) do not represent clinical information with the same degree of completeness and fidelity as paper records. The objective of this study was to develop a basic content model for clinical information in paper-based records and examine its degree of coverage by CPRs. Design: We compiled a baseline dental record (BDR) from a purposive sample of 10 paper record formats (two from dental schools and four each from dental practices and commercial sources). We extracted all clinical data fields, removed duplicates, and organized the resulting collection in categories/subcategories. We then mapped the fields in four market-leading dental CPRs to the BDR. Measurements: We calculated frequency counts of BDR categories and data fields for all paper-based and computer-based record formats, and cross-mapped information coverage at both the category and the data field level. Results: The BDR had 20 categories and 363 data fields. On average, paper records and CPRs contained 14 categories, and 210 and 174 fields, respectively. Only 72, or 20%, of the BDR fields occurred in five or more paper records. Categories related to diagnosis were missing from most paper-based and computer-based record formats. The CPRs rarely used the category names and groupings of data fields common in paper formats. Conclusion: Existing paper records exhibit limited agreement on what information dental records should contain. The CPRs only cover this information partially, and may thus impede the adoption of electronic patient records. © 2007 J Am Med Inform Assoc.

We present the results of a field study of some of the work practices and software used by dentists. We also present the design, implementation, and evaluation of a user interface that streamlines some of these practices, as well as providing a novel 3D visualization of a patient's mouth that also displays relevant radiographs depending on what teeth are visible in the visualization. Dentists found the 3D synchronized navigation intuitive, even with little to no 3D navigation experience, but further research is needed to see the effect on real clinical outcomes.

Speech recognition allows clinicians a hands-free option for interacting with computers, which is important for dentists who have difficulty using a keyboard and a mouse when working with patients. While roughly 13% of all general dentists with computers at chairside use speech recognition for data entry, 16% have tried and discontinued using this technology. In this study, researches explored the speech recognition features and functionality of four dental software applications. For each system, the documentation as well as the working program was evaluated to determine speech recognition capabilities. A comparison checklist was created to highlight each program's speech functionality. Next, after the development of charting scripts, feasibility user tests were conducted to determine if performance comparisons could be made across systems. While four systems were evaluated in the feature comparison, only two of the systems were reviewed during the feasibility user tests. Results show that current speech functionality, instead of being intuitive, is directly comparable to using a mouse. Further, systems require memorizing an enormous amount of specific terminology opposed to using natural language. User testing is a feasible way to measure the performance of speech recognition across systems and will be conducted in the near future. Overall, limited speech functionality reduces the ability of clinicians to interact directly with the computer during clinical care. This can hinder the benefits of electronic patient records and clinical decision support systems. © 2007 The authors. All rights reserved.

This paper illustrates the use of biometrics through the application of an iris-based biometrics system for identifying twins and their parents in a longitudinal research study. It explores the use of biometrics (science of measuring physical or anatomical characteristics of individuals) as a technology for correct identification of individuals during longitudinal studies to help ensure data fidelity. Examples of these circumstances include longitudinal epidemiological and genetic studies, clinical trials, and multicenter collaborative studies where accurate identification of subjects over time can be difficult when the subject may be young or an unreliable source of identification information. The use of technology can automate the process of subject identification thereby reducing the need to depend on subject recall during repeated visits thus helping to ensure data quality. This case report provides insights that may serve as useful hints for those responsible for planning system implementation that involves participants' authentication that would require a more secure form of identification.

Various organizations offer online continuing dental education (CDE) courses. While previous investigations focused on objective measures to determine the quality of the courses, this exploratory study evaluates the participants' experience with them. We surveyed 436 past course participants from nine online CDE courses (courses provided by six organizations) regarding their experience with the courses taken. Our analysis of the 169 responses (38.8 percent response rate) focuses on how the participants of online CDE courses can be characterized; whether the participants' expectations were met by the courses; how the participants evaluated the content of the courses; why they enrolled; and the participants' experience of the online environment. The results suggest that online CDE courses partially meet the needs and expectations of dental professionals. The lack of communication with peers and instructors as well as the fact that courses appeared outdated were main reasons for dissatisfaction. Most of the participants accomplished their goals of gaining new knowledge and deepening their understanding of the subject. Based on this evaluation, future courses can be tailored to meet more closely the expectations and needs of dental professionals.

ASP practice management systems are a new and potentially useful service for the dental profession. ASP systems have the potential to reduce the time, effort, and expense required to maintain computer-based practice management functions. ASP systems can deliver a rich array of functions at a distance, eliminate concerns about upgrades, back-ups, and server hardware, and make management of client workstations easier. The ASP industry is young, however, and its companies and products will be in constant flux. Practitioners should exercise great caution in selecting an ASP system. If the ASP model proves viable in dentistry over the long term, we can expect to see a growing number of services being delivered over the Internet. Many ASPs exist already that serve many needs of small businesses, such as accounting, personnel management, and supply ordering. Application service provider systems also have the potential to improve communication between dental care providers. The ease of exchanging selected patient information between providers may result in more patient-related information exchange than is currently the case. The quality of dental care can only benefit.

Dental practice and education are becoming more globalized. Greater practitioner and patient mobility, the free flow of information, increasingly global standards of care and new legal and economic frameworks (such as European Union [EU] legislation) are forcing a review of dental licensure, specialization and continuing education systems. The objective of this study was to compare these systems in Canada, France, Germany, the UK and the US. Representatives from the five countries completed a 29-item questionnaire, and the information was collated and summarized qualitatively. Statutory bodies are responsible for licensing and re-licensing in all countries. In the two North American countries, this responsibility rests with individual states, and in Europe, with the countries themselves, mainly governed by the legal framework of the EU. In some countries, re-licensure requires completion of continuing education credits. Approaches to dental specialization tend to differ widely with regard to definition of specialities, course and duration of training, training facilities, and accreditation of training programmes. In most countries, continuing education is provided by a number of different entities, such as universities, dental associations, companies, institutes and private individuals. Accreditation and recognition of continuing education is primarily process-driven, not outcome-orientated. Working towards a global infrastructure for dental licensing, specialization and continuing education depends on a thorough understanding of the international commonalities and differences identified in this article. © Blackwell Munksgaard, 2002.

Background. Dental Internet portals can offer dental practitioners "one-stop shopping" for many information needs. To date, no studies have described and evaluated dental portals' services and content. Methods. The authors evaluated five dental portals from Jan. 22, 2001, to April 5, 2001, using 90 evaluation criteria in seven categories: general, services, miscellaneous, navigation and usability, site currency, site performance and responsiveness, and site integrity. Groups of three to four dental students rated each portal. The authors rated certain criteria using commercial monitoring and analysis services. Results. The portals evaluated in this study provided a wide range of services such as product purchasing, online continuing education, practice management services, news, dental practice Web pages and event calendars. Portals differed in many characteristics, such as the number of services, product pricing, discussion forum activity, navigability, reaction time in response to questions and site responsiveness. Conclusions. The implementation and usefulness of each portal's services varied. No portal can fit all needs best, and many portals change rapidly owing to the volatility of the Internet industry. Dentists should be familiar with portals' services and alternatives for using them. Practice Implications. Portals can provide useful services to dental practitioners. Practitioners, however, should evaluate portals carefully to ensure that their needs are met optimally.

Our section's assignment entails exploration of the current challenges to develop criteria for quality assurance of dental e-learning material. Our work has involved comparison of current methods of assessment, the identification of best practice and the formulation of guidelines and criteria for producers and assessors. We anticipate the need for a standing international body responsible for the revision and refinement of guidelines and criteria and that might award a 'Seal of Approval'. © Blackwell Munksgaard, 2002.

Background. Dental informatics is a relatively new field that has significant potential for supporting clinical care. Most dentists are unaware of what dental informatics is, what its goals are, what it has achieved and how they can get involved in it. Methods. The authors conducted a literature review and several round-table discussions with dental informatics experts to discuss the preceding issues surrounding dental informatics. Results. Dental informatics is the application of computer and information sciences to improve dental practice, research, education and management. Numerous applications that support clinical care, education and research have been developed. Dental informatics is beginning to exhibit the characteristics of a discipline: core literature, trained specialists and educational programs. Conclusions. Dental informatics presents possible solutions to many long-standing problems in dentistry, but it also faces significant obstacles and challenges. Its maturation will depend as much on the efforts of people as on the collective efforts of the profession. Practice Implications. Dental informatics will produce an increasing number of applications and tools for clinical practice. Dentists must keep up with these developments to make informed choices.

This project developed a protocol for the inaugural Instructional Software Competition of the American Dental Education Association (ADEA). The evaluation instrument was derived from the Guidelines for the Design of Educational Software developed by the ANSI-accredited Standards Committee for Dental Informatics. Eleven judges were calibrated in a conference call and rated a total of 30 submissions using a 66-question instrument. The maximum score was 204 points. The mean score of WWW-based programs was 106.7 points, and of CD-ROM-based programs 109.5 points. The summative review of the judging process identified several potential improvements, such as distinguishing between standalone programs and educational support material; increasing the number of answer choices on rating scales; differential weighting of criteria; and a more discriminative approach to judging formative and summative evaluations. We plan to improve the protocol by supporting the process through a Web-based application; calibrating judges with an online handbook; improving and adapting the rating instrument itself; using at least three judges for each program; and conducting a measurement study.

The ready availability of computers, combined with the advent of sophisticated software development tools, has resulted in a proliferation of dental instructional software. The quality of these products varies widely. Working Group 5 of the Standards Committee for Dental Informatics has published an initial set of guidelines to help course designers develop high-quality instructional materials. In the future, these guidelines will evolve into standards. This article summarizes the guidelines and considers future actions that may be taken once the standards are in place.

Objective: The purpose of this study was to determine the instructional characteristics of online continuing dental education courses on the World Wide Web. Method and materials: One hundred fifty-seven online courses offered by 32 providers were reviewed for 34 criteria. Results: Courses of various types spanned a variety of topics. Approximately half of all courses offered continuing dental education credit. The relationship between credit hours and course length was quite variable. Most courses consisted of text and images. Very few courses used advanced media such as video clips. Measured against an index of instructional quality developed for this study, the instructional quality of the courses, in general, was poor. Most for-credit courses contained self-assessment questions, but only 28% of courses scored the questions online. Basic information that was missing on many courses included the authors' names (29%); the intended audience (81%); goals and objectives (77%); and references (85%). In 47% of the courses, there was no opportunity to provide feedback to either the author or the provider. Conclusion: The theoretical advantages of Web-based continuing dental education are numerous, but the currently available online resources are mostly of unacceptable quality.

Academic computing initiatives rank high on the list of priorities of many dental schools. However, outcomes of academic computing initiatives have not been presented. The objectives of this program evaluation were to: 1) document a strategic initiative for academic computing over a five-year period; 2) assess outcomes; and 3) demonstrate how outcomes assessment changed strategic goals for the future. In 1992, Temple University School of Dentistry developed an academic computing plan. The plan proposed to develop the computer literacy of faculty, teach students the computer skills they need to be successful in their careers, and introduce computer-aided instruction as a new teaching tool. Before a new five-year plan was developed in 1997, the original plan's outcomes were summarily assessed. Assessment instruments included faculty and student surveys, budgets, inventory records, and utilization statistics. The school has reached two of three goals of the 1992 plan. Eighty percent of all full-time faculty have computers, are computer literate, and use computers for a variety of purposes. The school has implemented a comprehensive predoctoral dental informatics curriculum. However, the implementation of computer-aided instruction has not met expectations. Goals of the 1998-2003 plan include establishing an online learning infrastructure, improving student access, implementing computer-based oral health records, and further improving the computer literacy of faculty and students. Planning and supporting academic computing initiatives is a substantial challenge. Factors such as institutional culture, capital investment, ongoing support, and technological change influence plans and their success. While process and structure can be assessed relatively easily, measures for changed educational outcomes are still lacking.