by Paul Govern

The prospect of the adoption of artificial intelligence-based tools by health care teams has begun to raise practical and ethical concerns, with no consensus apparently yet available regarding how health professionals might best prepare themselves to evaluate and work with such tools.

This state of affairs prompted researchers at Vanderbilt University Medical Center and IBM Watson Health to engage experts from around the country in defining a set of AI-related clinical competencies for health care professionals. The group’s competency statements and research report appeared recently in Academic Medicine.

“We started with a formal literature review of the health professions education and informatics literature, finding that there’ve been numerous calls for a list of AI-related competencies, because of the complexity of the tools and the risk of misuse and unintended consequences,” said Bonnie Miller, MD, MMHC. “But we could find no list of competencies, and very scant mention of how clinicians were being trained to use AI-based tools as they’ve begun entering into testing.”

Miller, professor of Medical Education and Administration, led the project with Regina Russell, PhD, assistant professor of Medical Education and Administration.

Selecting 15 subject matter experts from around the country, the team conducted structured interviews on the use of AI-based tools in health care settings. Based on the interviews, draft competency statements were provided to the experts for feedback and were eventually finalized based on consensus among the eight members of the research team.

“In interviews our subject experts expressed a mix of optimism and caution,” Russell said. “They recognize the great potential of these new technologies to support health, but the need for caution was repeatedly  expressed around issues of bias and fairness as AI-based tools are rolled out. It’s well understood that AI-based tools can exacerbate biases present in the data used to train them. With health disparities of all sorts having long been baked into our health system and our society, it’s clear that health professionals will need to develop baseline knowledge, skills and attitudes to work appropriately with these tools.”

The final statement includes 25 sub-competencies under six competency domains: basic knowledge of AI; social and ethical implications of AI; AI-enhanced clinical encounters; evidence-based evaluation of AI tools; workflow analysis for AI-based tools; and practice-based learning and improvement regarding AI-based tool.

Others on the study from VUMC include Laurie Novak, PhD, Kim Garvey, PhD, MS, MLIS, Gretchen Jackson, MD, PhD, and Don Moore, PhD.