Search

Background To date, efforts to measure impact have largely focused on health research in high-income countries, reflecting where the majority of health research funding is spent. Nevertheless, there is a growing body of health and medical research being undertaken in low- and middle-income countries (LMICs), supported by both development aid and established research funders. The Framework to Assess the Impact of Translational health research (FAIT) combines three approaches to measuring research impact (Payback, economic assessment and case study narrative). Its aim is to strengthen the focus on translation and impact measurement in health research. FAIT has been used by several Australian research initiatives; however, it has not been used in LMICs. Our aim was to apply FAIT in an LMIC context and evaluate its utility. Methods We retrospectively applied all three FAIT methods to two LMIC studies using available data, supplemented with group discussion and further economic analyses. Results were presented in a scorecard format. Results FAIT helped clarify pathways of impact for the projects and provided new knowledge on areas of impact in several domains, including capacity-building for research, policy development and economic impact. However, there were constraints, particularly associated with calculating the return on investment in the LMIC context. The case study narrative provided a layperson’s summary of the research that helped to explain outcomes and succinctly communicate lessons learnt. Conclusion Use of FAIT to assess the impact of LMIC research was both feasible and useful. We make recommendations related to prospective use, identification of metrics to support use of the Payback framework, and simplification of the economic assessment, which may facilitate further application in LMIC environments. Electronic supplementary material The online version of this article (10.1186/s12961-019-0451-0) contains supplementary material, which is available to authorized users.

ObjectivesTo (1) apply the Framework to Assess the Impact from Translational health research (FAIT) to Lessons from the Best to Better the Rest (LFTB), (2) report on impacts from LFTB and (3) assess the feasibility and outcomes from a retrospective application of FAIT.SettingThree Indigenous primary healthcare (PHC) centres in the Northern Territory, Australia; project coordinating centre distributed between Townsville, Darwin and Cairns and the broader LFTB learning community across Australia.ParticipantsLFTB research team and one representative from each PHC centre.Primary and secondary outcome measuresImpact reported as (1) quantitative metrics within domains of benefit using a modified Payback Framework, (2) a cost-consequence analysis given a return on investment was not appropriate and (3) a narrative incorporating qualitative evidence of impact. Data were gathered through in-depth stakeholder interviews and a review of project documentation, outputs and relevant websites.ResultsLFTB contributed to knowledge advancement in Indigenous PHC service delivery; enhanced existing capacity of health centre staff, researchers and health service users; enhanced supportive networks for quality improvement; and used a strengths-based approach highly valued by health centres. LFTB also leveraged between $A1.4 and $A1.6 million for the subsequent Leveraging Effective Ambulatory Practice (LEAP) Project to apply LFTB learnings to resource development and creation of a learning community to empower striving PHC centres.ConclusionRetrospective application of FAIT to LFTB, although not ideal, was feasible. Prospective application would have allowed Indigenous community perspectives to be included. Greater appreciation of the full benefit of LFTB including a measure of return on investment will be possible when LEAP is complete. Future assessments of impact need to account for the limitations of fully capturing impact when intermediate/final impacts have not yet been realised and captured.

This aim of this paper is to set the scene for the need for impact assessment and return on investment in funded cardiovascular research in Australia, starting with the historical perspective on waste in health and medical research. Recently there has been a substantial move from discussion and policy about the need for research translation, into practice and application via the evolution of funding streams like the Australian Medical Research Future Fund (MRFF). Health and medical research funders play a critical role in both setting the expectations for research translation and impact and helping researchers to meet these expectations. As a leading cause of death, cardiovascular disease is a national health priority, recognised as such with a AUD$220 million MRFF allocation to the Cardiovascular Health Mission. Focussing on cardiovascular research, we address some of the barriers researchers face in prospectively planning for research translation and impact assessment, and call for an ecosystem that supports a return on investment for all stakeholders, especially the community and patient end-users.

Background The question of how to measure, assess and optimise the returns from investment in health and medical research (HMR) is a highly policy-relevant issue. Research Impact Assessment Frameworks (RIAFs) provide a conceptual measurement framework to assess the impact from HMR. The aims of this study were (1) to elicit the views of Medical Research Institutes (MRIs) regarding objectives, definitions, methods, barriers, potential scope and attitudes towards RIAFs, and (2) to investigate whether an assessment framework should represent a retrospective reflection of research impact or a prospective approach integrated into the research process. The wider objective was to inform the development of a draft RIAF for Australia’s MRIs. Methods Purposive sampling to derive a heterogeneous sample of Australian MRIs was used alongside semi-structured interviews with senior executives responsible for research translation or senior researchers affected by research impact initiatives. Thematic analysis of the interview transcriptions using the framework approach was then performed. Results Interviews were conducted with senior representatives from 15 MRIs. Participants understood the need for greater research translation/impact, but varied in their comprehension and implementation of RIAFs. Common concerns included the time lag to the generation of societal impacts from basic or discovery science, and whether impact reflected a narrow commercialisation agenda. Broad support emerged for the use of metrics, case study and economic methods. Support was also provided for the rationale of both standardised and customised metrics. Engendering cultural change in the approach to research translation was acknowledged as both a barrier to greater impact and a critical objective for the assessment process. Participants perceived that the existing research environment incentivised the generation of academic publications and track records, and often conflicted with the generation of wider impacts. The potential to improve the speed of translation through prospective implementation of impact assessment was supported, albeit that the mechanism required development. Conclusion The study found that the issues raised regarding research impact assessment are less about methods and metrics, and more about the research activities that the measurement of research translation and impact may or may not incentivise. Consequently, if impact assessment is to contribute to optimisation of the health gains from the public, corporate and philanthropic investment entrusted to the institutes, then further inquiry into how the assessment process may re-align research behaviour must be prioritised.

Background There is growing recognition amongst health and medical research funders and researchers that translation of research into policy and practice needs to increase and that more transparency is needed on how impacts are realised. Several approaches are advocated for achieving this, including co-production of research or academic-practitioner research. The Population Health Unit (PHU) within the Hunter New England Local Health District in regional Australia, as an early adopter of this model, has been working to increase the likelihood that its research is translated into community health benefits. With the New South Wales Ministry of Health, the PHU responded to the burden of child overweight and obesity by combining service delivery with research expertise. The ‘Good for Kids, Good for Life’ (Good for Kids) dissemination trial was developed and implemented in seven community settings in the Hunter region of Australia between 2006 and 2010. This study aims to undertake a retrospective impact assessment to measure the research translation and impact of Good for Kids. Methods The method will be based upon the application of the Framework to Assess the Impact from Translational health research (FAIT), comprising three core elements, namely quantified metrics, economic assessment and a narrative of the process by which the research in question translates and generates impact. Discussion Increasingly, funders are interested both in the outcomes resulting from investments in health research and in the expected return on their investments. FAIT was developed specifically for this purpose and its use is anticipated to provide transparency to the pathway to translation and potentially drive increased investment in translational research programmes such as Good for Kids.

Retrospective medical record review to assess compliance with low back pain (LBP) care indicators. To establish baseline estimates of the appropriateness of LBP care in the general Australian population provided by a range of healthcare providers in various real-world settings. LBP is a costly condition and accounts for the greatest burden of disease worldwide, yet the care provided is often at variance with guidelines. No baseline estimates of performance are currently available in Australia across various aspects of LBP care, practitioners, and settings. A population-based sample of patients with 22 common conditions was recruited by telephone; consents were obtained to review their medical records against indicators (“CareTrack”). Care for LBP was reviewed against 10 indicators used in a previous study and ratified by experts as representing appropriate LBP care in Australia during 2009 and 2010. Of the 22 CareTrack conditions, LBP had the highest number of eligible healthcare encounters (6588 of 35,573, 19%), 125 to 884 per indicator among 164 LBP patients. Overall compliance with LBP indicators was 72% (range 42%–98%). Allied health practitioners and hospitals were the most compliant (82%–83% respectively), followed by general practitioners (54%). Some aspects of care were poor, such as documenting a thorough neurological examination, screening for serious diseases such as infection and inappropriate use of drugs such as steroids and treatments such as traction. Over a quarter of LBP care was not appropriate despite the availability of guidelines. There is a need for national and, potentially, international agreement on clinical standards, indicators and tools to guide, document and monitor the appropriateness of care for LBP, and for measures to increase their uptake, particularly where deficiencies have been identified. Level of Evidence: N /A

Background There is growing recognition that the wider benefits of research (economic, social and health impacts) should be assessed and valued alongside traditional research performance metrics such as peer-reviewed papers. Translation of findings into policy and practice needs to accelerate and pathways to impact need to be better understood. This research protocol outlines a mixed methods study to apply the Framework to Assess the Impact from Translational health research (FAIT) to the Centre for Research Excellence in Stroke Rehabilitation and Brain Recovery (CRE-Stroke). FAIT is purpose-designed to encourage research translation and assess research impact but lacks validation. Methods/Design Phase 1 involves application of the FAIT-modified programme logic model to each CRE-Stroke research stream including identifying process, output and impact metrics, as well as end users of the research. A scoping review will inform potential impacts anticipated from CRE-Stroke. In Phase 2, audit and feedback on achievements against plans will track and encourage research translation. Logic models will be updated to account for changes in the research pathways over time. In Phase 3, three proven methods for measuring research impact – Payback, economic assessment and narratives – will be applied to each research stream and the data triangulated and reported in Phase 4. The feasibility of applying FAIT will also be assessed as part of Phase 3. Discussion Use of prospective, comprehensive research impact frameworks for large interdisciplinary programmes of research is rare. FAIT’s application to CRE-Stroke will provide opportunity for the impact of CRE-Stroke to be assessed and a range of impacts beyond standard academic achievements to be reliably reported. The feasibility of FAIT’s application will also be assessed and, if necessary, refined. The usefulness of FAIT for encouraging research translation will also be described and may prove useful for other programmes looking to implement a research impact framework.

In Australia, Indigenous people experience poor access to healthcare and the highest rates of morbidity and mortality of any population group. Despite modest improvements in recent years, concerns remains that Indigenous people have been over-researched without corresponding health improvements. Embedding Indigenous leadership, participation and priorities in health research is an essential strategy for meaningful change for Indigenous people. To centralize Indigenous perspectives in research processes, a transformative shift away from traditional approaches that have benefited researchers and non-Indigenous agendas is required. This shift must involve concomitant strengthening of the research capacity of Indigenous and non-Indigenous researchers and research translators – all must teach and all must learn. However, there is limited evidence about how to strengthen systems and stakeholder capacity to participate in and lead CQI research in Indigenous primary healthcare, to the benefit of Indigenous people. This paper describes the collaborative development of, and principles underpinning a research capacity strengthening model in a national Indigenous primary healthcare continuous quality improvement research network. The development process identified the need to address power imbalances, cultural contexts, relationships, systems requirements and existing knowledge, skills and experience of all parties. Taking a strengths-based perspective, we harnessed existing knowledge, skills and experiences; hence our emphasis on capacity ‘strengthening’. New insights are provided into the complex processes of research capacity strengthening within the context of continuous quality improvement in Indigenous primary healthcare.

Background Although “non-specific” in 90% of cases, low back pain (LBP) is often treated as an independent entity, even though comorbidities are commonly associated with it. There is evidence that some LBP may be related to chronic conditions or be a symptom of poor health. The purpose of this study was to clarify the extent of comorbidities amongst a cohort of Australian adults with LBP and examine if having concurrent conditions has any association with appropriateness of care for LBP. Methods A population-based sample of patients with one or more of 22 common conditions was recruited by telephone; consents were obtained to review their medical records. Trained surveyors extracted information from their medical records to examine the care patients received for their LBP with respect to ten indicators of appropriate care, ratified by LBP experts. Using LBP as the index condition, lists of self-reported comorbidities and those that were documented in medical records were compared. Medical records were reviewed and analysed with respect to appropriateness of care to identify any significant differences in care received between patients with LBP only and those with LBP plus comorbidities. Results One hundred and sixty four LBP patients were included in the analysis. Over 60% of adults with LBP in Australia had one of 17 comorbidities documented, with females being more likely than males to have comorbid conditions (63% vs 37%, p = 0.012). The more comorbidities, the poorer their reported health status (63% vs 30%, p = 0.006). Patients with comorbidities were significantly less likely to receive appropriate LBP care on nine of the ten LBP indicators (p

IMPORTANCE The quality of routine care for children is rarely assessed, and then usually in single settings or for single clinical conditions. OBJECTIVE To estimate the quality of health care for children in Australia in inpatient and ambulatory health care settings. DESIGN, SETTING, AND PARTICIPANTS Multistage stratified sample with medical record review to assess adherence with quality indicators extracted from clinical practice guidelines for 17 common, high-burden clinical conditions (noncommunicable [n = 5], mental health [n = 4] , acute infection [n = 7], and injury [n = 1] ), such as asthma, attention-deficit/hyperactivity disorder, tonsillitis, and head injury. For these 17 conditions, 479 quality indicators were identified, with the number varying by condition, ranging from 9 for eczema to 54 for head injury. Four hundred medical records were targeted for sampling for each of 15 conditions while 267 records were targeted for anxiety and 133 for depression. Within each selected medical record, all visits for the 17 targeted conditions were identified, and separate quality assessments made for each. Care was evaluated for 6689 children 15 years of age and younger who had 15 240 visits to emergency departments, for inpatient admissions, or to pediatricians and general practitioners in selected urban and rural locations in 3 Australian states. These visits generated 160 202 quality indicator assessments. EXPOSURES Quality indicators were identified through a systematic search of local and international guidelines. Individual indicators were extracted from guidelines and assessed using a 2-stage Delphi process. MAIN OUTCOMES AND MEASURES Quality of care for each clinical condition and overall. RESULTS Of 6689 children with surveyed medical records, 53.6% were aged 0 to 4 years and 55.5% were male. Adherence to quality of care indicators was estimated at 59.8% (95% CI, 57.5%-62.0%; n = 160 202) across the 17 conditions, ranging from a high of 88.8% (95% CI, 83.0%-93.1%; n = 2638) for autism to a low of 43.5% (95% CI, 36.8%-50.4%; n = 2354) for tonsillitis. The mean adherence by condition category was estimated as 60.5% (95% CI, 57.2%-63.8%; n = 41 265) for noncommunicable conditions (range, 52.8%-75.8%); 82.4% (95% CI, 79.0%-85.5%; n = 14 622) for mental health conditions (range, 71.5%-88.8%); 56.3% (95% CI, 53.2%-59.4%; n = 94 037) for acute infections (range, 43.5%-69.8%); and 78.3% (95% CI, 75.1%-81.2%; n = 10 278) for injury. CONCLUSIONS AND RELEVANCE Among a sample of children receiving care in Australia in 2012-2013, the overall prevalence of adherence to quality of care indicators for important conditions was not high. For many of these conditions, the quality of care may be inadequate. Refereed/Peer-reviewed

IntroductionThere is growing recognition among health researchers and funders that the wider benefits of research such as economic, social and health impacts ought to be assessed and valued alongside academic outputs such as peer-reviewed papers. Research translation needs to increase and the pathways to impact ought to be more transparent. These processes are particularly pertinent to the Indigenous health sector given continued concerns that Indigenous communities are over-researched with little corresponding improvement in health outcomes. This paper describes the research protocol of a mixed methods study to apply FAIT (Framework to Assess the Impact from Translational health research) to the Centre for Research Excellence in Integrated Quality Improvement (CRE-IQI). FAIT will be applied to five selected CRE-IQI Flagship projects to encourage research translation and assess the wider impact of that research.Methods and analysisPhase I will develop a modified programme logic model for each Flagship project including identifying process, output and impact metrics so progress can be monitored. A scoping review will inform potential benefits. In phase II, programme logic models will be updated to account for changes in the research pathways over time. Audit and feedback will be used to encourage research translation and collect evidence of achievement of any process, output and interim impacts. In phase III, three proven methodologies for measuring research impact—Payback, economic assessment and narratives—will be applied. Data on the application of FAIT will be collected and analysed to inform and improve FAIT’s performance.Ethics and disseminationThis study is funded by a nationally competitive grant (ID 1078927) from the Australian National Health and Medical Research Council. Ethics approval was obtained from the University of Newcastle’s Human Research Ethics Committee (ID: H-2017–0026). The results from the study will be presented in several peer-reviewed publications, through conference presentations and via social media.

ObjectiveThere is a growing need for researchers to demonstrate impact, which is reliant on successful research translation. The Australian National Health and Medical Research Council funded a Centre of Research Excellence in Stroke Rehabilitation and Brain Recovery (CRE-Stroke) to enhance collaborations between researchers conducting different types of stroke rehabilitation research. The purpose of this study was to explore opinions about research translation held by CRE-Stroke researchers conducting preclinical and clinical research, in terms of scope, importance, responsibility and perceived skills and knowledge.DesignMixed-methods study, comprising a paper-based survey and semistructured interviews. Interview data were inductively coded and thematically analysed. Survey and interview data were compared and synthesised.Participants55 (7 preclinical, 48 clinical) researchers attending a CRE-Stroke research forum completed a paper-based survey. Semistructured interviews with 22 CRE-Stroke (5 preclinical, 17 clinical) researchers were conducted.ResultsResearch translation was described as translating to other research and translating to clinical practice and policy. Most researchers (n=54, 98%) reported that research translation was important, particularly in terms of generating research impact, but the most common sign of project completion reported by researchers (n=7, 100% preclinical; n=37, 77% clinical) was publication. Most researchers (preclinical n=4, 57%; clinical n=37, 77%) reported having responsibility for translating research, but less than half reported having the necessary skills (n=1, 14% preclinical; n=17, 35% clinical) and knowledge (n=3, 43% preclinical; n=19, 40% clinical). Differing opinions about who should be responsible for translating findings to clinical practice were expressed.ConclusionsStroke rehabilitation researchers appear confident to translate their research via the traditional mechanism of publications. To optimise impact, clarity is needed regarding who is best placed to translate research findings to clinical practice and policy. Education and skills development to apply broader translation processes are needed to maximise the use of research at all stages.

Efforts to strengthen health systems require the engagement of diverse, multidisciplinary stakeholder networks. Networks provide a forum for experimentation and knowledge creation, information exchange and the spread of good ideas and practice. They might be useful in addressing complex issues or ‘wicked’ problems, the solutions to which go beyond the control and scope of any one agency.Innovation platforms are proposed as a novel type of network because of their diverse stakeholder composition and focus on problem solving within complex systems. Thus, they have potential applicability to health systems strengthening initiatives, even though they have been predominantly applied in the international agricultural development sector.In this paper, we compare and contrast the concept of innovation platforms with other types of networks that can be used in efforts to strengthen primary healthcare systems, such as communities of practice, practice-based research networks and quality improvement collaboratives. We reflect on our ongoing research programme that applies innovation platform concepts to drive large-scale quality improvement in primary healthcare for Aboriginal and Torres Strait Islander Australians and outline our plans for evaluation. Lessons from our experience will find resonance with others working on similar initiatives in global health.

Objectives The prevention and management of venous thromboembolism (VTE) is often at variance with guidelines. The CareTrack Australia (CTA) study reported that appropriate care (in line with evidence-based or consensus-based guidelines) is being provided for VTE at just over half of eligible encounters. The aim of this paper is to present and discuss the detailed CTA findings for VTE as a baseline for compliance with guidelines at a population level. Setting The setting was 27 hospitals in 2 states of Australia. Participants A sample of participants designed to be representative of the Australian population was recruited. Participants who had been admitted overnight during 2009 and/or 2010 were eligible. Of the 1154 CTA participants, 481(42%) were admitted overnight to hospital at least once, comprising 751 admissions. There were 279 females (58%), and the mean age was 64 years. Primary and secondary outcome measures The primary measure was compliance with indicators of appropriate care for VTE. The indicators were extracted from Australian VTE clinical practice guidelines and ratified by experts. Participants’ medical records from 2009 to 2010 were analysed for compliance with 38 VTE indicators. Results Of the 35 145 CTA encounters, 1078 (3%) were eligible for scoring against VTE indicators. There were 2–84 eligible encounters per indicator at 27 hospitals. Overall compliance with indicators for VTE was 51%, and ranged from 34% to 64% for aggregated sets of indicators. Conclusions The prevention and management of VTE was appropriate for only half of the at-risk patients in our sample; this provides a baseline for tracking progress nationally. There is a need for national and, ideally, international agreement on clinical standards, indicators and tools to guide, document and monitor care for VTE, and for measures to increase their uptake, particularly where deficiencies have been identified.

Introduction In recent years in keeping with international best practice, clinical guidelines for common conditions have been developed, endorsed and disseminated by peak national and professional bodies. Yet evidence suggests that there remain considerable gaps between the care that is regarded as appropriate by such guidelines and the care received by patients. With an ageing population and increasing treatment options and expectations, healthcare is likely to become unaffordable unless more appropriate care is provided. This paper describes a study protocol that seeks to determine the percentage of healthcare encounters in which patients receive appropriate care for 22 common clinical conditions and the reasons why variations exist from the perspectives of both patients and providers. Methods/design A random stratified sample of at least 1000 eligible participants will be recruited from a representative cross section of the adult Australian population. Participants' medical records from the years 2009 and 2010 will be audited to assess the appropriateness of the care received for 22 common clinical conditions by determining the percentage of healthcare encounters at which the care provided was concordant with a set of 522 indicators of care, developed for these conditions by a panel of 43 disease experts. The knowledge, attitudes and beliefs of participants and healthcare providers will be examined through interviews and questionnaires to understand the factors influencing variations in care. Ethics and dissemination Primary ethics approvals were sought and obtained from the Hunter New England Local Health Network. The authors will submit the results of the study to a relevant journal as well as undertaking oral presentations to researchers, clinicians and policymakers., Article summary Article focus What is the percentage of healthcare encounters at which Australians receive appropriate care? What influences variations in care from the perspectives of patients and healthcare providers? Key messages A protocol for a population-based study of appropriate care of 1000 patients using medical record review. Strengths and limitations of this study Obtaining a snapshot and using a consistent method for 522 indicators across 22 common conditions power diagnostic indicators because they only present once for each patient. The potential attrition rate of healthcare providers and telephone recruitment of participants may introduce selection biases.