AI Health

Friday Roundup

The AI Health Friday Roundup highlights the week’s news and publications related to artificial intelligence, data science, public health, and clinical research.

May 2, 2025

In this week’s Duke AI Health Friday Roundup: SHADES lets researchers spot bias in non-English datasets; therapeutic chatbots and chatbots that claim to be therapists; nurses’ perspective powers AI surveillance in ICU; vaccine simulation study predicts return of endemic measles; an oral history of LLMs; ditching daylight savings; cuts to research may rock national economy; much more:

AI, STATISTICS & DATA SCIENCE

“Although tools that spot stereotypes in AI models already exist, the vast majority of them work only on models trained in English. They identify stereotypes in models trained in other languages by relying on machine translations from English, which can fail to recognize stereotypes found only within certain non-English languages, says Zeerak Talat, at the University of Edinburgh, who worked on the project. To get around these problematic generalizations, SHADES was built using 16 languages from 37 geopolitical regions.” MIT Technology Review’s Rhiannon Williams reports on the SHADES dataset, which is designed to help researchers probe LLMs for cultural biases in languages other than English.
“As the first RCT of its kind, our study supports the feasibility, acceptability, and effectiveness of a fine-tuned, fully GenAI–powered chatbot for treating mental health symptoms. Users demonstrated sustained engagement and rated their alliance with Therabot as comparable to human therapists during the 4-week trial.” A research article published in NEJM AI by Heinz and colleagues reports results from a randomized trial of an LLM-based mental health therapeutic chatbot.
“Patients with hospital encounters during which the interprofessional care team was informed by CONCERN EWS were a third less likely to die and a quarter more likely to be transferred to intensive care. Unanticipated transfer to the ICU increased in the intervention group. Nurses can observe subtle patient changes that suggest clinical deterioration is more likely, and early recognition and escalation of treatment for these patients by the interprofessional care team, including transfer to the ICU, can improve outcomes.” A research article published in Nature Medicine by Rossetti and colleagues presents results from a pragmatic trial of an ICU “early-warning” machine learning system based on nursing surveillance patterns.
“In 2019, Quanta reported on a then-groundbreaking NLP system called BERT without once using the phrase “large language model.” A mere five and a half years later, LLMs are everywhere, igniting discovery, disruption and debate in whatever scientific community they touch. But the one they touched first — for better, worse and everything in between — was natural language processing. What did that impact feel like to the people experiencing it firsthand?” At Quanta magazine, John Pavlus presents a series of “oral histories” of the conception and development of the technology we now know as large language models (H/T @markriedl.bsky.social).

BASIC SCIENCE, CLINICAL RESEARCH & PUBLIC HEALTH

“A key finding is that, under current vaccination levels, measles may be likely to return to endemic levels within the next 20 years, driven by states with routine vaccination coverage below historical levels and below the threshold needed to maintain elimination of transmission. This study suggests that, over the long-term, the US may be approaching the threshold of losing sufficient population immunity to reliably prevent endemic transmission of measles, and higher childhood vaccination rates may be required to avert this.” A simulation study published in JAMA by Kiang and colleagues has some sobering implications regarding the trajectory of some vaccine-preventable diseases in the US.
“Among US adults aged 20 to 59 years, the prevalence of obesity by BMI only was nearly identical with the obesity prevalence after confirmation of excess adiposity. Approximately 98% of individuals identified as having obesity based on BMI had excess adiposity…. Although certain patient populations (eg, athletes) may warrant further evaluation, our results suggest that these individuals make up a very small portion of the population.” A research letter published in JAMA by Aryee and colleagues examines the accuracy of BMI alone in identifying obesity when compared with confirmatory measures.
“Dementia has no single trigger. As with many cancers, it can emerge from a lifetime of accumulated strain—from genetics, environment, and behavior. Researchers have identified a dozen risk factors that, if mitigated, could theoretically delay or prevent roughly 40 percent of cases worldwide: traumatic brain injury; conditions including high blood pressure, hearing loss, diabetes, and depression; habits such as smoking, inactivity, and heavy drinking; environmental and social forces including air pollution, social isolation, and limited education.” An Atlantic article by Marion Renault and Cheney Orr explores the causes and effects of a cluster of dementia in a rural Texas community.
“…it comes as a surprise to find that gut microbes are capable of removing fluorine from carbon centers, as reported in this preprint. People have seen this sort of enzymatic activity in (for example) soil bacteria and sludge samples from the bottom of polluted rivers and the like, but I believe this is the first report of it happening with gut microbiota….That’s an impressive feat by itself, since fluoride is toxic to bacteria as it stands.” A post at Derek Lowe’s In the Pipeline blog highlights a recent discovery of naturally occurring human gut microflora capable of breaking up otherwise highly tenacious carbon-flourine chemical bonds (such as the ones found in large numbers in “forever” PFAS chemicals).

COMMUNICATION, Health Equity & Policy

“Daylight saving time transitions cause acute disruptions in human circadian rhythm, and the medical literature shows detrimental effects for public health. Also, the establishment of permanent DST would cause chronic disruption of the natural seasonal adjustment of circadian rhythm. This disruption risks social jet lag in the population, which is convincingly associated with harmful health effects.” A perspective article published in JAMA by Kim and colleagues advances the view that the health toll imposed by switching to Daylight Savings Time is a strong argument for abandoning it.
“Economists have warned that such cuts could undermine American competitiveness in areas like vaccine development, artificial intelligence and quantum computing, and could slow growth in income and productivity in the long term. The private sector can’t fully replace government dollars, they argue, because basic research is too risky and takes too long to pay off to attract sufficient private investment.” An article by the New York Times’ Ben Casselman reports on a recent American University study that warns of potential profound damage to the US economy resulting from recent cuts in federal support for scientific research.
“I’ve had similar conversations with chatbot therapists for weeks on Meta’s AI Studio, with chatbots that other users created and with bots I made myself. When pressed for credentials, most of the therapy bots I talked to rattled off lists of license numbers, degrees, and even private practices. Of course these license numbers and credentials are not real, instead entirely fabricated by the bot as part of its back story.” 404’s Samantha Cole with a particularly yikes-inducing story of chatbots confidently asserting nonexistent credentials.