Search

Prediction tools driven by artificial intelligence (AI) and machine learning are becoming increasingly integrated into health care delivery in the United States. However, organizational approaches to the governance of AI tools are highly varied. There is growing recognition of the need for evidence on best governance practices and multilayered oversight that could provide appropriate guardrails at the organizational and federal levels to address the unique dimensions of AI prediction tools. We sought to qualitatively characterize salient dimensions of AI-enabled predictive model governance at U.S. academic medical centers (AMCs). We analyzed how AMCs in the United States currently govern predictive models and consider the implications. A total of 17 individuals from 13 AMCs across the country participated in interviews. Half-hour to 1-hour interviews were conducted via Zoom from October 2022 to January 2023. The interview guide focused on the capacity, governance, regulation, and evaluation of AI-driven predictive models. Analysis of interview data was inductive. The research team wrote memos throughout the process of interviewing and analysis. We identified three governance phenotypes: welldefined, emerging, and interpersonal. In the well-defined governance phenotype, health systems have explicit, comprehensive procedures for the review and evaluation of AI and predictive models. In the emerging governance phenotype, systems are in the process of adjusting or adapting previously established approaches for clinical decision support or electronic health records (EHR) to govern AI. In health systems using interpersonal or individual-driven governance approaches, an individual is tasked with making decisions about model implementation without consistent evaluation requirements. We found that the influence of EHR vendors is an important consideration for those tasked with governance at AMCs, given concerns about regulatory gaps and the need for model evaluation. Even well-resourced AMCs are struggling to effectively identify, manage, and mitigate the myriad potential problems and pitfalls related to the implementation of predictive AI tools. The range of governance structures that we identified indicates a need for additional guidance, both regulatory and otherwise, for health systems as prediction and AI proliferate. Rather than concentrating responsibility for governance within organizations, multiple levels of governance that include the industry and regulators would better promote quality care and patient safety. This sort of structure would also provide desired guidance and support to the individuals tasked with governing these tools. [ABSTRACT FROM AUTHOR]

BackgroundIn the past decade, Lynn Etheredge presented a vision for the Learning Health System (LHS) as an opportunity for increasing the value of health care via rapid learning from data and immediate translation to practice and policy. An LHS is defined in the literature as a system that seeks to continuously generate and apply evidence, innovation, quality, and value in health care. ObjectiveThis review aimed to examine themes in the literature and rhetoric on the LHS in the past decade to understand efforts to realize the LHS in practice and to identify gaps and opportunities to continue to take the LHS forward. MethodsWe conducted a thematic analysis in 2018 to analyze progress and opportunities over time as compared with the initial Knowledge Gaps and Uncertainties proposed in 2007. ResultsWe found that the literature on the LHS has increased over the past decade, with most articles focused on theory and implementation; articles have been increasingly concerned with policy. ConclusionsThere is a need for attention to understanding the ethical and social implications of the LHS and for exploring opportunities to ensure that these implications are salient in implementation, practice, and policy efforts.