Project profile: 2024 Delphi Study during the Duke Summit on AI for Health Innovation
Status: Completed
To probe for consensus on the barriers and facilitators of innovation in Health AI, we conducted a Delphi study during the Duke Summit on AI for Health Innovation. Held over three days in October 2024, the in-person summit brought together experts from the fields of engineering and health in order to foster a community of practice around health-oriented AI development. Representatives from industry, medicine, academia, and funders came together to discuss the current and future landscape of Health AI development and innovation, providing an ideal opportunity for a Delphi study on this topic.
We invited Summit attendees to participate in a classical Delphi study with three rounds of questionnaires. We asked three questions:
- What is the greatest barrier to innovation in Health AI?
- Which is the most needed training or skillset that people in Health AI are not getting, or not getting enough of, currently?
- Where would implementing AI result in the most significant impact on productivity?
For the questions about barriers and impact, a clear consensus emerged around lack of trust and documentation, respectively. For the question about needed skills, participants consolidated around two options, implementation and basic understanding of AI. Across all three questions, the engineering and health participants clustered around different options in rounds 1 and 2, but largely converged on the consensus choices in round 3.
Lead evaluation scientist: Whitney Welsh, PhD
Read more about the study and view detailed results:
Poster presentation by Dr. Welsh at the annual meeting of the Association for Clinical and Translational Science in Washington, D.C. in April of 2025. Poster file: 2025-actscience-poster-final
The Delphi method is an iterative, group-based process for exploring whether a consensus exists on a given topic. A panel of subject matter experts completes multiple rounds of questionnaires, with the chance to change their responses in each round based on anonymized feedback about how their responses compare with the group’s and the reasoning that other participants provide for their choices (Keeney, Hasson & McKenna 2011; Rowe & Wright 1999). The method is best suited to investigate “issues about which uncertain or incomplete data exists” (Neiderberger & Renn 2023).
References
Keeney, Sinead, Felicity Hasson, and Hugh McKenna. 2011. The Delphi Technique in Nursing and Health Research. Oxford: Wiley Blackwell.
Niederberger, Marlen, and Ortwin Renn. 2023. Delphi Methods in the Health and Social Sciences: Concepts, Applications, and Case Studies. Springer.
Rowe, Gene and George Wright. 1999. “The Delphi technique as a forecasting tool: issues and analysis.” International Journal of Forecasting, 15:353-75.
