Search

Abstract Background Health-policy decision making is a complex and dynamic process, for which strong evidentiary support is required. This includes scientifically produced research, as well as information that relates to the context in which the decision takes place. Unlike scientific evidence, this “contextual evidence” is highly variable and often includes information that is not scientifically produced, drawn from sources such as political judgement, program management experience and knowledge, or public values. As the policy decision-making process is variable and difficult to evaluate, it is often unclear how this heterogeneous evidence is identified and incorporated into “evidence-based policy” decisions. Population-based colorectal cancer screening poses an ideal context in which to examine these issues. In Canada, colorectal cancer screening programs have been established in several provinces over the past five years, based on the fecal occult blood test (FOBT) or the fecal immunochemical test. However, as these programs develop, new scientific evidence for screening continues to emerge. Recently published randomized controlled trials suggest that the use of flexible sigmoidoscopy for population-based screening may pose a greater reduction in mortality than the FOBT. This raises the important question of how policy makers will address this evidence, given that screening programs are being established or are already in place. This study will examine these issues prospectively and will focus on how policy makers monitor emerging scientific evidence and how both scientific and contextual evidence are identified and applied for decisions about health system improvement. Methods This study will employ a prospective multiple case study design, involving participants from Ontario, Alberta, Manitoba, Nova Scotia, and Quebec. In each province, data will be collected via document analysis and key informant interviews. Documents will include policy briefs, reports, meeting minutes, media releases, and correspondence. Interviews will be conducted in person with senior administrative leaders, government officials, screening experts, and high-level cancer system stakeholders. Discussion The proposed study comprises the third and final phase of an Emerging Team grant to address the challenges of health-policy decision making and colorectal cancer screening decisions in Canada. This study will contribute a unique prospective look at how policy makers address new, emerging scientific evidence in several different policy environments and at different stages of program planning and implementation. Findings will provide important insight into the various approaches that are or should be used to monitor emerging evidence, the relative importance of scientific versus contextual evidence for decision making, and the tools and processes that may be important to support challenging health-policy decisions.

Abstract Background Qualitative research has the potential to inform and improve health care decisions but a study based on one year of publications suggests that it is not published in prominent health care journals. A more detailed, longitudinal analysis of its availability is needed. The purpose of this study was to identify, count and compare the number of qualitative and non-qualitative research studies published in high impact health care journals, and explore trends in these data over the last decade. Methods A bibliometric approach was used to identify and quantify qualitative articles published in 20 top general medical and health services and policy research journals from 1999 to 2008. Eligible journals were selected based on performance in four different ranking systems reported in the 2008 ISI Journal Citation Reports. Qualitative and non-qualitative research published in these journals were identified by searching MEDLINE, and validated by hand-searching tables of contents for four journals. Results The total number of qualitative research articles published during 1999 to 2008 in ten general medical journals ranged from 0 to 41, and in ten health services and policy research journals from 0 to 39. Over this period the percentage of empirical research articles that were qualitative ranged from 0% to 0.6% for the general medical journals, and 0% to 6.4% for the health services and policy research journals. Conclusions This analysis suggests that qualitative research it is rarely published in high impact general medical and health services and policy research journals. The factors that contribute to this persistent marginalization need to be better understood.

Abstract Background Research regarding the decision to adopt and implement technological innovations in radiation oncology is lacking. This is particularly problematic since these technologies are often complex and rapidly evolving, requiring ongoing revisiting of decisions regarding which technologies are the most appropriate to support. Variations in adoption and implementation decisions for new radiation technologies across cancer centres can impact patients' access to appropriate and innovative forms of radiation therapy. This study examines the key steps in the process of adopting and implementing intensity modulated radiation therapy (IMRT) in publicly funded cancer centres and identifies facilitating or impeding factors. Methods A multiple case study design, utilizing document analysis and key informant interviews was employed. Four cancer centres in Ontario, Canada were selected and interviews were conducted with radiation oncologists, medical physicists, radiation therapists, and senior administrative leaders. Results Eighteen key informants were interviewed. Overall, three centres made fair to excellent progress in the implementation of IMRT, while one centre achieved only limited implementation as of 2009. Key factors that influenced the extent of IMRT implementation were categorized as: 1) leadership, 2) training, expertise and standardization, 3) collaboration, 4) resources, and 5) resistance to change. Conclusion A framework for the adoption and implementation of complex and evolving technologies is presented. It identifies the key factors that should be addressed by decision-makers at specific stages of the adoption/implementation process.

Abstract Background Decision aids have been developed in a number of health disciplines to support evidence-informed decision making, including patient decision aids and clinical practice guidelines. However, policy contexts differ from clinical contexts in terms of complexity and uncertainty, requiring different approaches for identifying, interpreting, and applying many different types of evidence to support decisions. With few studies in the literature offering decision guidance specifically to health policymakers, the present study aims to facilitate the structured and systematic incorporation of research evidence and, where there is currently very little guidance, values and other non-research-based evidence, into the policy making process. The resulting decision aid is intended to help public sector health policy decision makers who are tasked with making evidence-informed decisions on behalf of populations. The intent is not to develop a decision aid that will yield uniform recommendations across jurisdictions, but rather to facilitate more transparent policy decisions that reflect a balanced consideration of all relevant factors. Methods/design The study comprises three phases: a modified meta-narrative review, the use of focus groups, and the application of a Delphi method. The modified meta-narrative review will inform the initial development of the decision aid by identifying as many policy decision factors as possible and other features of methodological guidance deemed to be desirable in the literatures of all relevant disciplines. The first of two focus groups will then seek to marry these findings with focus group members' own experience and expertise in public sector population-based health policy making and screening decisions. The second focus group will examine issues surrounding the application of the decision aid and act as a sounding board for initial feedback and refinement of the draft decision aid. Finally, the Delphi method will be used to further inform and refine the decision aid with a larger audience of potential end-users. Discussion The product of this research will be a working version of a decision aid to support policy makers in population-based health policy decisions. The decision aid will address the need for more structured and systematic ways of incorporating various evidentiary sources where applicable.

Abstract Background In research, diagrams are most commonly used in the analysis of data and visual presentation of results. However there has been a substantial growth in the use of diagrams in earlier stages of the research process to collect data. Despite this growth, guidance on this technique is often isolated within disciplines. Methods A multidisciplinary systematic review was performed, which included 13 traditional healthcare and non-health-focused indexes, non-indexed searches and contacting experts in the field. English-language articles that used diagrams as a data collection tool and reflected on the process were included in the review, with no restriction on publication date. Results The search identified 2690 documents, of which 80 were included in the final analysis. The choice to use diagrams for data collection is often determined by requirements of the research topic, such as the need to understand research subjects' knowledge or cognitive structure, to overcome cultural and linguistic differences, or to understand highly complex subject matter. How diagrams were used for data collection varied by the degrees of instruction for, and freedom in, diagram creation, the number of diagrams created or edited and the use of diagrams in conjunction with other data collection methods. Depending on how data collection is structured, a variety of options for qualitative and quantitative analysis are available to the researcher. The review identified a number of benefits to using diagrams in data collection, including the ease with which the method can be adapted to complement other data collection methods and its ability to focus discussion. However it is clear that the benefits and challenges of diagramming depend on the nature of its application and the type of diagrams used. Discussion/Conclusion The results of this multidisciplinary systematic review examine the application of diagrams in data collection and the methods for analyzing the unique datasets elicited. Three recommendations are presented. Firstly, the diagrammatic approach should be chosen based on the type of data needed. Secondly, appropriate instructions will depend on the approach chosen. And thirdly, the final results should present examples of original or recreated diagrams. This review also highlighted the need for a standardized terminology of the method and a supporting theoretical framework.

Abstract Background The use of electronic health records (EHRs) to support the organization and delivery of healthcare is evolving rapidly. However, little is known regarding potential variation in access to EHRs by provider type or care setting. This paper reports on observed variation in the perceptions of access to EHRs by a wide range of cancer care providers covering diverse cancer care settings in Ontario, Canada. Methods Perspectives were sought regarding EHR access and health record completeness for cancer patients as part of an internet survey of 5663 cancer care providers and administrators in Ontario. Data were analyzed using a multilevel logistic regression model. Provider type, location of work, and access to computer or internet were included as covariates in the model. Results A total of 1997 of 5663 (35%) valid responses were collected. Focusing on data from cancer care providers (N = 1247), significant variation in EHR access and health record completeness was observed between provider types, location of work, and level of computer access. Providers who worked in community hospitals were half as likely as those who worked in teaching hospitals to have access to their patients' EHRs (OR 0.45 95% CI: 0.24–0.85, p < 0.05) and were six times less likely to have access to other organizations' EHRs (OR 0.15 95% CI: 0.02–1.00, p < 0.05). Compared to surgeons, nurses (OR 3.47 95% CI: 1.80–6.68, p < 0.05), radiation therapists/physicists (OR 7.86 95% CI: 2.54–25.34, p < 0.05), and other clinicians (OR 4.92 95% CI: 2.15–11.27, p < 0.05) were more likely to report good access to their organization's EHRs. Conclusion Variability in access across different provider groups, organization types, and geographic locations illustrates the fragmented nature of EHR adoption in the cancer system. Along with focusing on technological aspects of EHR adoption within organizations, it is essential that there is cross-organizational and cross-provider access to EHRs to ensure patient continuity of care, system efficiency, and high quality care.

Abstract Background This paper assesses interviewee transcript review (ITR) as a technique for improving the rigour of interview-based, qualitative research. ITR is a process whereby interviewees are provided with verbatim transcripts of their interviews for the purposes of verifying accuracy, correcting errors or inaccuracies and providing clarifications. ITR, in various forms, is widely used among qualitative researchers, however there is limited methodological guidance on how it should be employed and little is known about its actual impact on the transcript, the data, the interviewee or the researcher. Methods ITR was incorporated into a qualitative research study in which 51 key informant interviews were conducted with a range of senior stakeholders within the Canadian health care system. The changes made by interviewees to their transcripts were systematically tracked and categorized using a set of mutually exclusive categories. Results The study found that ITR added little to the accuracy of the transcript and may create complications if the goal of the researcher is to produce a transcript which reflects precisely what was said at the time of the interview. The advantages of ITR are that it allows interviewees the opportunity to edit or clarify information provided in the original interview, with many interviewees providing corrections, clarifications, and in some cases, adding new material to their transcripts. There are also potential disadvantages, such as a bias created by inconsistent data sources or the loss of data when an interviewee chooses to remove valuable material. The impact of ITR on the interviewee may be both positive and negative, depending on the specific circumstances and the nature of the study. The impact of ITR on the researcher was minimal in this study, but is again subject to specific circumstances of the research context. Conclusion While ITR is employed by many researchers across numerous fields, the advantages of its use may be relatively small in terms of verifying the accuracy of qualitative interview transcripts. Researchers are advised to carefully consider both the potential advantages and disadvantages of ITR outlined in this paper before deciding to incorporate the practice within their qualitative study designs.

Abstract Background This paper focuses on measuring the efficiency and effectiveness of two diagramming methods employed in key informant interviews with clinicians and health care administrators. The two methods are 'participatory diagramming', where the respondent creates a diagram that assists in their communication of answers, and 'graphic elicitation', where a researcher-prepared diagram is used to stimulate data collection. Methods These two diagramming methods were applied in key informant interviews and their value in efficiently and effectively gathering data was assessed based on quantitative measures and qualitative observations. Results Assessment of the two diagramming methods suggests that participatory diagramming is an efficient method for collecting data in graphic form, but may not generate the depth of verbal response that many qualitative researchers seek. In contrast, graphic elicitation was more intuitive, better understood and preferred by most respondents, and often provided more contemplative verbal responses, however this was achieved at the expense of more interview time. Conclusion Diagramming methods are important for eliciting interview data that are often difficult to obtain through traditional verbal exchanges. Subject to the methodological limitations of the study, our findings suggest that while participatory diagramming and graphic elicitation have specific strengths and weaknesses, their combined use can provide complementary information that would not likely occur with the application of only one diagramming method. The methodological insights gained by examining the efficiency and effectiveness of these diagramming methods in our study should be helpful to other researchers considering their incorporation into qualitative research designs.

BackgroundInter-professional collaborative care (ICC) for cancer leads to multiple system, organizational, professional, and patient benefits, but is limited by numerous challenges. Empirical research on interventions that promote or enable ICC is sparse so guidance on how to achieve ICC is lacking. Research shows that ICC for diagnosis could be improved. Diagnostic assessment programs (DAPs) appear to be a promising model for enabling ICC. The purpose of this study was to explore how DAP structure and function enable ICC, and whether that may be associated with organizational and clinical outcomes.MethodsA case study approach will be used to explore ICC among eight DAPs that vary by type of cancer (lung, breast), academic status, and geographic region. To describe DAP function and outcomes, and gather information that will enable costing, recommendations expressed in DAP standards and clinical guidelines will be assessed through retrospective observational study. Data will be acquired from databases maintained by participating DAPs and the provincial cancer agency, and confirmed by and supplemented with review of medical records. We will conduct a pilot study to explore the feasibility of estimating the incremental cost-effectiveness ratio using person-level data from medical records and other sources. Interviews will be conducted with health professionals, staff, and referring physicians from each DAP to learn about barriers and facilitators of ICC. Qualitative methods based on a grounded approach will be used to guide sampling, data collection and analysis.DiscussionFindings may reveal opportunities for unique structures, interventions or tools that enable ICC that could be developed, implemented, and evaluated through future research. This information will serve as a formative needs assessment to identify the nature of ongoing or required improvements, which can be directly used by our decision maker collaborators, and as a framework by policy makers, cancer system managers, and DAP managers elsewhere to strategically plan for and implement diagnostic cancer services.

This article explores the perceived value of public health research and the types of research considered useful in Ghana. Sixty-nine decision makers, researchers and stakeholders were interviewed. A broad range of research was highly valued. Two traits were important: an applied, relevant topic and quickly produced findings. Interviewees valued research which explored implementation issues or identified and increased understanding of health problems. Effectiveness research often lacked these traits and was not generally considered as important because it was implicitly assumed that as long as interventions (that were based on a good understanding of the situation) were implemented well, they would be effective.

Semi-structured interviews in Ghana and England explored perceptions of the usefulness and use of foreign research (ie, beyond the original study country) compared to locally conducted research (ie, conducted in Ghana). There was a preference for locally conducted studies, although interviewees generally recognised the potential usefulness of foreign research. Various factors affected whether foreign research was considered useful or used; it was not used automatically or indiscriminately. Researchers should recognise the potential usefulness of their research beyond the original study country. Further work is needed to explore how to maximise the utility of foreign research, as a means of enabling evidence-informed decision making where locally conducted research is not available.

To link to full-text access for this article, visit this link: http://dx.doi.org/10.1016/j.socscimed.2006.04.020 Byline: Mark J. Dobrow (a), Vivek Goel (a), Louise Lemieux-Charles (a), Nick A. Black (b) Keywords: Canada; Evidence; Context; Utilization; Health policy; Decision-making; Expert groups; Knowledge translation Abstract: Should the same evidence lead to the same decision outcomes in different decision-making contexts? In order to improve comprehension of this issue, this study considers how context influences evidence utilization in the development of health policy recommendations. We used an embedded multiple case study design to study how four expert groups formulated policy recommendations for breast, cervical, colorectal, and prostate cancer screening in Ontario, Canada. We interviewed expert group members and analysed meeting agendas/minutes, interim/final reports and other case-related documents. Our analyses revealed varying policy objectives; the use, neglect, or overextended consideration of three key decision support tools; the varying skills/abilities of expert group members in using different decision support tools; the varying impact of effect modifiers, resource constraints and political interests; and the differing development/consideration of context-specific evidence to address uncertainty in the external decision-making context. While more work is needed to determine if these findings are generalizable beyond cancer screening policy, we believe the central challenge for evidence-based policy is not to develop international evidence, but rather to develop more systematic, rigorous, and global methods for identifying, interpreting, and applying evidence in different decision-making contexts. Our analyses suggest that identification of evidence must distinguish between different policy objectives in order to link a broad conceptualization of evidence to appropriate policy questions. Interpretation of evidence must acknowledge the varying nature of evidence for different policy objectives, balancing existing emphasis on evidentiary quality with more sophisticated methods for assessing the generalizability of evidence. The application of evidence must also acknowledge different policy objectives, appropriately employing rule-based grading schemes and agreement-based consensus methods that are sensitive to the nature of the evidence and contexts involved. Author Affiliation: (a) Department of Health Policy, Management and Evaluation, University of Toronto, Canada (b) Department of Public Health and Policy, London School of Hygiene and Tropical Medicine, UK

Evidence-based decision-making is centred on the justification of decisions. In the shift from an individual-clinical to a population-policy level, the decision-making context becomes more uncertain, variable and complex. To address this we have developed a conceptual framework for evidence-based decision-making, focusing on how context impacts on what constitutes evidence and how that evidence is utilised. We present two distinct orientations towards what constitutes evidence, representing different relationships between evidence and context. We also categorise the decision-making context based on the ways in which context impacts on evidence-based decision-making. Furthermore, we invoke the concept of axes of evidence-based decision-making to describe the relationship between evidence and context as we move from evidence-based medicine to evidence-based health policy. From this, we suggest that it may be more important how evidence is utilised than how it is defined. Based on the research and knowledge utilisation literature, we present a process model of evidence utilisation, which forms the basis for the conceptual framework for context-based evidence-based decision-making. The conceptual framework attempts to capture the role that context plays in the introduction, interpretation and application of evidence. We illustrate this framework with examples from policy development for colorectal cancer screening. Keywords: Evidence-based decision-making; Health policy; Context; Evidence utilisation

Background Internet survey modalities often compare unfavorably with traditional survey modalities, particularly with respect to response rates. Response to Internet surveys can be affected by the distribution options and response/collection features employed as well as the existence of automated (out-of-office) replies, automated forwarding, server rejection, and organizational or personal spam filters. However, Internet surveys also provide unparalleled opportunities to track study subjects and examine many of the factors influencing the determination of response rates. Tracking data available for Internet surveys provide detailed information and immediate feedback on a significant component of response that other survey modalities cannot match. This paper presents a response audit of a large Internet survey of more than 5000 cancer care providers and administrators in Ontario, Canada. Objective Building upon the CHEcklist for Reporting Results of Internet E-Surveys (CHERRIES), the main objectives of the paper are to (a) assess the impact of a range of factors on the determination of response rates for Internet surveys and (b) recommend steps for improving published descriptions of Internet survey methods. Methods We audited the survey response data, analyzing the factors that affected the numerator and denominator in the ultimate determination of response. We also conducted a sensitivity analysis to account for the inherent uncertainty associated with the impact of some of the factors on the response rates. Results The survey was initially sent out to 5636 health care providers and administrators. The determination of the numerator was influenced by duplicate/unattached responses and response completeness. The numerator varied from a maximum of 2031 crude (unadjusted) responses to 1849 unique views, 1769 participants, and 1616 complete responses. The determination of the denominator was influenced by forwarding of the invitation email to unknown individuals, server rejections, automated replies, spam filters, and ‘opt out’ options. Based on these factors, the denominator varied from a minimum of 5106 to a maximum of 5922. Considering the different assumptions for the numerator and the denominator, the sensitivity analysis resulted in a 12.5% variation in the response rate (from minimum of 27.3% to maximum of 39.8%) with a best estimate of 32.8%. ConclusionsDepending on how the numerator and denominator are chosen, the resulting response rates can vary widely. The CHERRIES statement was an important advance in identifying key characteristics of Internet surveys that can influence response rates. This response audit suggests the need to further clarify some of these factors when reporting on Internet surveys for health care providers and administrators, particularly when using commercially available Internet survey packages for specified, rather than convenience, samples.

Rationale, aim, and objectives: Heart failure (HF) clinics are highly effective, yet not optimally utilized. A realist review was performed to identify contexts (eg, health system characteristics, clinic capacity, and siting) and underlying mechanisms (eg, referring provider knowledge of clinics and referral criteria, barriers in disadvantaged patients) that influence utilization (provider referral [ie, of all appropriate and no inappropriate patients] and access [ie, patient attends ≥1 visit]) of HF clinics. Methods: Following an initial scoping search and field observation in a HF clinic, we developed an initial program theory in conjunction with our expert panel, which included patient partners. Then, a literature search of seven databases was searched from inception to December 2019, including Medline; Grey literature was also searched. Studies of any design or editorials were included; studies regarding access to cardiac rehabilitation, or a single specialist for example, were excluded. Two independent reviewers screened the abstracts, and then full‐texts. Relevant data from included articles were used to refine the program theory. Results: A total of 29 papers from five countries (three regions) were included. There was limited information to support or refute many elements of our initial program theory (eg, referring provider knowledge/beliefs, clinic inclusion/exclusion criteria), but refinements were made (eg, specialized care provided in each clinic, lack of patient encouragement). Lack of capacity, geography, and funding arrangements were identified as contextual factors, explaining a range of mechanistic processes, including patient clinical characteristics and social determinants of health as well as clinic characteristics that help to explain inappropriate and low use of HF clinics (outcome). Conclusion: Given the burden of HF and benefit of HF clinics, more research is needed to understand, and hence overcome sub‐optimal use of HF clinics. In particular, an understanding from the perspective of referring providers is needed. [ABSTRACT FROM AUTHOR]

Introduction: In 25% to 35% of patients with early stage I non-small-cell lung cancer (NSCLC), surgery is not feasible, and external-beam radiation becomes their standard treatment. Conventionally fractionated radiotherapy (CFRT) is the traditional radiation treatment standard; however, stereotactic body radiotherapy (SBRT) is increasingly being adopted as an alternate radiation treatment. Our objective was to conduct a cost-effectiveness analysis, comparing SBRT with CFRT for stage I NSCLC in a public payer system. Methods: Consecutive patients were reviewed using 2010 Canadian dollars for direct medical costs from a public payer perspective. A subset of direct radiation treatment delivery costs, excluding physician billings and hospitalization, was also included. Health outcomes as life-years gained (LYGs) were computed using time-to-event methods. Sensitivity analyses identified critical factors influencing costs and benefits. Results: From January 2002 to June 2010, 168 patients (CFRT, n = 50; SBRT, n = 118) were included; median follow-up was 24 months. Mean overall survival was 2.83 years (95% CI, 1.8 to 4.1) for CFRT and 3.86 years (95% CI, 3.2 to not reached) for SBRT (P = .06). Mean costs for CFRT were $6,886 overall and $5,989 for radiation treatment delivery only versus $8,042 and $6,962, respectively, for SBRT. Incremental costs (incremental cost-effectiveness ratio [ICER]) per LYG for SBRT versus CFRT were $1,120 for the public payer and $942 for radiation treatment alone. Varying survival and labor costs individually (±20%) created the largest changes in the ICER, and simultaneous adjustment (±5% to ±30%) confirmed cost effectiveness of SBRT. Conclusion: Using a threshold of $50,000 per LYG, SBRT seems cost effective. Results require confirmation with randomized data. [ABSTRACT FROM AUTHOR]