AI Health
Friday Roundup
The AI Health Friday Roundup highlights the week’s news and publications related to artificial intelligence, data science, public health, and clinical research.
February 28, 2025
In this week’s Duke AI Health Friday Roundup: when AI departs from its original goals; flood of electronic messages for docs has not abated; weird patents are latest angle in bogus science credentials; foundation model for genomic prediction; clinicians report on ambient AI-powered scribing for chart notes; junk food excursions may affect the brain; respect for human rights understood as opportunity for AI; much more:
AI, STATISTICS & DATA SCIENCE
- “In complex scenarios, AI systems might pursue their objectives in ways that conflict with human interests. This may happen more often as AI systems develop stronger situational awareness and reason strategically about their environment… Our results suggest that frontier LLM agents can strategically circumvent the intended rules of their environment to achieve objectives, with more capable models showing this behavior more frequently.” A preprint by Bondarenko and colleagues, available from arXiv, demonstrates conditions in which large language models may attempt to circumvent rules in pursuit of goals.
- “Clinicians reported using the notes generated by ambient scribing to help recall their patient encounters with a level of detail they would not have captured themselves. Similarly, many clinicians noted that it was much easier to build from ambient scribing–generated notes rather than write a new note entirely from scratch….it is important to acknowledge potential downsides of these findings. For example, our results suggest that the time and mental effort saved with ambient scribing was associated with a better clinician experience and increased focus on patients; however, it is also possible that the use of ambient scribing–generated notes could decrease clinicians’ opportunities to reflect while note-writing and address missed diagnoses or treatments.” In a quality improvement study published in JAMA Network Open, Duggan and colleagues evaluate self-reported physician experiences using ambient-listening AI transcription technology to document patient visits.
- “This work demonstrates that a generative model of the underlying genomic language enables a machine learning model to achieve generalist prediction and design capabilities across all domains of life. By learning statistical properties of DNA across 9 trillion tokens of genomic sequences, Evo 2 can predict mutational effects on protein function, ncRNA function, and organismal fitness. Evo 2 is the first alignment-free language model that robustly predicts the pathogenicity of different mutation types in ClinVar, including indels, achieving state-of-the-art performance for noncoding and splice variants.” A preprint by Brixi and colleagues, available from the Arc Institute and from BioRxiv, describes Evo 2, a biological foundation model designed for genome prediction (H/T @smcgrath.phd).
- “…we overviewed and emphasized the importance of developing operational, explainable, and trustworthy AI systems. Addressing these challenges requires a coordinated effort across disciplines involving AI researchers, environmental and climate scientists, field experts, and policymakers. This collaborative approach is crucial for advancing AI applications in extreme event analysis and ensuring that these technologies are adapted to real-world needs and constraints.” A review article published in Nature Communications by Camps-Valls and colleagues examines the use of AI to analyze, predict, and respond to severe weather events.
- “Despite significant excitement surrounding AI-enabled tools, a gap remains between the perceived potential of these technologies and their proven clinical value. Numerous studies have highlighted that ML algorithms may, at times, offer insights that clinicians can already infer or observe directly. In the realm of hemodynamic assessment for surgical and critically ill patients, certain examples stand out. For instance, the HPI has been shown to largely mirror the MAP, resulting in similar predictive capabilities for hypotensive events.” A review article published in Annals of Intensive Care by Michard and colleagues surveys the use of AI-based applications for monitoring of blood pressure and echocardiography data.
BASIC SCIENCE, CLINICAL RESEARCH & PUBLIC HEALTH
- “The volume of EHR messages received by physicians increased at the onset of the COVID-19 pandemic in March 2020 and remained elevated. Patient medical advice request messages meaningfully increased, and we found no evidence that patients were substituting other communication, such as telephone calls…. these results suggest that the increase in message volume is likely a sustained, ongoing source of physician work. Health systems and policymakers should prioritize strategies to reduce inbox burden for physicians, especially PCPs, while maintaining patient access to care.” A study by Holmgren and colleagues, published as a research letter in JAMA Internal Medicine, suggests that the flood tide of electronic messages inundating physicians has yet to recede.
- “A more than doubled rate of nirmatrelvir-ritonavir prescriptions was not associated with reductions in hospitalizations or mortality among highly vaccinated older adults in Ontario…This study’s 95% CI suggests that at best, nirmatrelvir-ritonavir could reduce COVID-19–related hospitalizations by 1.3 percentage points, which is 4 times smaller than the absolute risk reduction of 5.5 percentage points reported in the original nirmatrelvir-ritonavir trial among unvaccinated middle-aged adults.” A research letter published in JAMA by Mafi and colleagues presents findings from a “natural experiment” comparing COVID outcomes among vaccinated older adults according to whether or not they received treatment with Paxlovid (nirmatrelvir-ritonavir).
- “Through the application of advanced phylogenetic comparative methods, we reveal that empirical evidence does not support Peto’s paradox across the four major classes of terrestrial vertebrates: Larger species do, in fact, face higher cancer prevalence than their smaller counterparts. Furthermore, we show that as variation in body size has evolved in birds and mammals, it has also driven the evolution of enhanced cellular growth control.” A new investigation, published in PNAS by Butler and colleagues, upends Peto’s paradox – the theory that as species increase in physical size, there is no consistent corresponding increase in cancer incidence (H/T @erictopol.bsky.social)
- “…short-term overeating with commonly used ultra-processed high-caloric snacks can trigger liver fat accumulation and short-term disrupted brain insulin action that outlast the time-frame of the HCD [high-caloric diet] in men. We postulate that the brain response to insulin adapts to short-term changes in diet before weight gain and may facilitate the development of obesity and associated diseases.” A brief communication published in Nature Metabolism by Kullman traces the biological and cognitive effects of a bout of junk-food indulgence.
COMMUNICATION, Health Equity & Policy
- “Over the past 2 years, these firms have registered thousands of bizarre designs for medical equipment and other devices with the U.K.’s Intellectual Property Office (IPO), listing scientists as the owners of the designs for a fee. The companies target researchers in countries including India and Pakistan, where universities reward researchers who patent inventions with career advancement and sometimes bonuses.” As if paper mills and predatory publishers weren’t bad enough: Science’s Cathleen O’Grady reports on the rise of bogus patent filings to pad out researcher resumes.
- “The total number of employees rehired is unclear, but in at least some cases the reinstatements appeared to be broad. All 12 of the people who worked in the office reviewing surgical and infection control devices were reinstated, an agency source said. Two of three people in the FDA’s digital health office who had been let go were rehired, as were a handful of employees reviewing AI-enabled imaging devices, diabetes devices and cardiovascular devices, according to four other agency sources.” STAT News’ Lizzy Lawrence reports that a swath of employees abruptly fired from the US Food and Drug Administration are being rehired.
- “Human rights advocacy has consistently driven technological innovation, demanding accountability, fostering public trust, encouraging wider adoption, and creating new tools to protect vulnerable communities. We must reject the false narrative that prioritizing human rights harms AI development. A human rights-centered approach is not a burden; it is an opportunity. It is the only way to ensure that AI innovation is sustainable, just, and truly beneficial to humanity.” An essay by Shirin Anlen at Tech Policy Press makes the case that prioritizing ethics and human rights will boost, rather than encumber, innovation in fields such as artificial intelligence.
- “Scientists of all stripes have been affected, but none more so than early-career researchers, a group already struggling with low pay and job insecurity. Now, some wonder how many of those budding researchers will throw in the towel and leave science, or the United States, entirely. ‘There’s going to be a missing age class of researchers that will reverberate for years,’ one federal scientist fears.” At Science, Katie Langin covers the upheaval and uncertainty affecting early-career researchers amid abrupt and confusing policy and funding changes.
- “On Monday, NIH employees within the Office of the Director were told that Federal Register notices for study sections run by the agency’s Center for Scientific Review — which reviews most major academic research grants, fellowships, and small business grants — will start being permitted again, according to an email reviewed by STAT. However, it appears that notices of meetings of advisory councils, which provide additional review and make final funding recommendations, will not.” STAT News’ Megan Molteni and Usha Lee McFarling report that some hitherto-suspended NIH study sections are now meeting again, but funding prospects remain uncertain.
