AI Health

Friday Roundup

The AI Health Friday Roundup highlights the week’s news and publications related to artificial intelligence, data science, public health, and clinical research.

February 14, 2025

In this week’s Duke AI Health Friday Roundup: TRAIN AI consortium debuts; possible role for LLMs in doctors’ management decisions; AI company loses copyright lawsuit; is genAI use affecting human cognition and memory?; AI interprets ambulatory EKG data; looking at promise vs reality in CRISPR therapeutics; transplant data infrastructure needs updating; weighing regulatory possibilities for health AI; much more:

AI, STATISTICS & DATA SCIENCE

Motion-blurred photograph of a metro train passing a stop in an underground station. Image credit: William Daigneault/Unsplash
Image credit: William Daigneault/Unsplash
  • “…the rapid evolution and adoption of health AI technologies often proceeds without strong evidence for their safety and effectiveness across diverse populations and settings, limiting their potential benefits and raising the risks of harm. Such ongoing AI adoption efforts across diverse settings represent “natural experiments” and present opportunities for developing sorely needed real-world evidence and for understanding the responsible use of health AI, but current mechanisms for capturing and sharing such experiences and evidence are limited.” In a new viewpoint article appearing in JAMA, a group of authors including Duke AI Health director Michael Pencina describe TRAIN, a consortium of healthcare stakeholders focused on ensuring the deployment of trustworthy health AI.
  • “…the DeepRhythmAI model could safely replace technician interpretation of ambulatory ECG recordings, with an impressive sensitivity for critical arrhythmias and a modest increase of false-positive detections. The DeepRhythmAI model had a negative predictive value for critical arrhythmias that exceeded 99.9% and, compared to technicians, resulted in 17 times fewer patients with a missed diagnosis of a critical arrhythmia. This was at a cost of 2.4 times more false-positive detections, which for critical arrhythmias occurred once every 6 recordings for AI and once every 14 recordings for technicians.” A research article by Johnson and colleagues, appearing in Nature Medicine, describes the use of an AI monitoring system for noncritical heart arrhythmias.
  • “…the availability of an LLM improved physician management reasoning compared to conventional resources only, with comparable scores between physicians randomized to use AI and AI alone. This suggests a future use for LLMs as a helpful adjunct to clinician judgment, while also highlighting the potential for standalone LLM applications in certain clinical scenarios. Delineating specific contexts where LLM assistance provides added value to physicians versus areas where AI might be useful independently is becoming increasingly important.” A research article published in Nature Medicine by Goh and colleagues examines the use of a large language model to assist physicians in making patient management decisions (as opposed to narrower diagnostic decisions).
  • “We developed ML models to predict, at the time of case posting, the postsurgical LOS [length of stay] and DD [discharge disposition] for adult elective inpatient cases across multiple services. These models could support case scheduling, resource allocation, optimal bed utilization, earlier discharge planning, and preventing case cancelation due to bed unavailability.” In a paper published in the Annals of Surgery Open, Zaribafzadeh and colleagues describe a machine learning model that was developed to predict postsurgical length of stay and discharge disposition for surgical patients at Duke.

BASIC SCIENCE, CLINICAL RESEARCH & PUBLIC HEALTH

This image shows a pixelated room, it looks like a typical bedroom or office. Most of it is heavily pixelated, but a shelf, table and plant, windows and clock can be recognized. These are all outlined in yellow boxes. Image credit: Elise Racine / Better Images of AI / Morning View / CC-BY 4.0
Image credit: Elise Racine / Better Images of AI / Morning View / CC-BY 4.0
  • “Researchers do agree that, in theory, the effects of generative-AI tools, such as ChatGPT, could be different from those of previous memory aids. ‘Before, I would write down the phone number in my address book and then when I looked it up, I knew my handwriting’ and therefore knew the information was authentic, says Jason Burton, who studies decision-making at the Max Planck Institute for Human Development in Berlin. But LLMs are different, he says: when asked a question, they generate writing afresh and can famously ‘hallucinate’ errors. This makes them a potentially unreliable external memory source and raises the risk that people could incorporate false information into their memories.” A feature article by Helen Pearson appearing in Nature examines recent evidence about the effects of the internet and AI technologies on human cognition and memory.
  • “…we found that the CDSS [clinical decision support system] did not reduce physician inappropriate imaging requests in academic hospital settings. There were no differences in the absolute number of imaging requests, radiation exposure, or cost….Notably, few physicians changed their initial imaging requests that were categorized as inappropriate, and, in rare cases, physicians even switched their order to less appropriate diagnostic tests.” In a research article published in JAMA, Dijk and colleagues report findings from a study that used a cluster-randomized design to assess the usefulness of clinical decision-support system for clinician imaging requests.
  • “As a lab tool, CRISPR has become as indispensable to scientists as whisks are to pastry chefs. It allowed researchers to probe cancer’s hidden vulnerabilities. It’s invaluable for bioengineers crafting cell therapies to chase down tumors or replace damaged tissues, even if data from those approaches remains sparse….Still, reaching most patients, including the thousands facing devastating and untreatable genetic disorders, may require regulatory as much as scientific changes.” STAT News’ Jason Mast reports on a loss of momentum in investment in CRISPR-based therapeutics – despite some prominent and seemingly promising early applications.

COMMUNICATION, Health Equity & Policy

An antique brass compass, lid open and face toward the camera, rests in the sand on a beach near the tide line, with waves out of focus in the background. Image credit: Denise Jans/Unsplash
Image credit: Denise Jans/Unsplash
  • “Reorienting AI surveillance to post-market performance will not be as simple as suggesting monitoring plans. Instead, a new framework will be needed to estimate uncertainty potential as well as patient risk. Given that the FDA has already suggested it is understaffed to adequately monitor all AI devices, the effective monitoring of post-market AI performance should enlist the efforts of not only AI developers but also healthcare providers who use these technologies and have direct access to the relevant data.” In an article appearing on the website of the Paragon Health Institute, Research Fellow Kev Coleman and Duke AI Health Director Michael Pencina describe the need for a new framework of regulation and monitoring to meet the challenges presented by a diverse array of AI-based healthcare applications and tools.
  • “Low quality books that appear to be AI generated are making their way into public libraries via their digital catalogs, forcing librarians who are already understaffed to either sort through a functionally infinite number of books to determine what is written by humans and what is generated by AI, or to spend taxpayer dollars to provide patrons with information they don’t realize is AI-generated.” At 404, Emanuel Maiberg reports on the infiltration of public library e-book offerings by the low-quality, AI-generated content referred to as “slop.”
  • “In the absence of accessible, integrated, multidimensional patient data, the transplant system will remain both highly variable across centers and vulnerable to bias that is unmeasured and unacknowledged. Forward progress now depends on clinical informatics and advanced data science to collect, organize, and share data that reflect the breadth of patient needs—from biological to social—and the full extent of care provided.” In an article appearing in Health Affairs Forefront, Duke’s Lisa McElroy, Jessica Tenenbaum, and Ricardo Henao make a case for reforming the digital data infrastructure underpinning the national system for organ transplantation.
  • “…this ruling is a blow to AI companies, according to Cornell University professor of digital and internet law James Grimmelmann: ‘If this decision is followed elsewhere, it’s really bad for the generative AI companies.’ Grimmelmann believes that Bibas’ judgement suggests that much of the case law that generative AI companies are citing to argue fair use is ‘irrelevant.’”At Wired, Kate Knibbs reports on a landmark court decision in an ongoing dispute over AI companies’ use of copyrighted material in training generative AI models.

Sites@Duke Express is powered by WordPress. Read the Sites@Duke Express policies and FAQs, or request help.