Status: Active

People are increasingly receiving healthcare information from large language models (LLMs), such as via chatbots and AI-augmented search engines. Despite its commonality, the nature of these patient-LLM interactions remain largely unexplored. A multi-institutional effort, spearheaded by Duke researchers, aims to characterize patient-facing usage of large language models for health information seeking, understanding what patients are looking for, and where models can go wrong.

One direction of this work generated HealthChat, a curated dataset of over 11,000 patient-chatbot conversations. We use a clinician-driven taxonomy to systematically study user interactions across 21 distinct health specialties. Our analysis reveals insights into the nature of how and why users seek health information, such as common interactions, instances of incomplete context, affective behaviors, and leading questions) that can induce sycophancy. Another direction studies how currently deployed generative AI solutions can mislead patients. In particular, we highlight how these models tend to decontextualize facts, omit critical relevant sources, and reinforce patient misconceptions or biases. We propose a series of recommendations—such as the incorporation of communication pragmatics and enhanced comprehension of source documents— that could help mitigate these issues. Together, these works underscore the need for improvements in the healthcare support capabilities of LLMs deployed for patient-facing use.

Research supported by: Duke Whitehead Scholar Award

Principal Investigator: Monica Agrawal