AI Health
Friday Roundup
The AI Health Friday Roundup highlights the week’s news and publications related to artificial intelligence, data science, public health, and clinical research.
February 20, 2025
In this week’s Duke AI Health Friday Roundup: agentic AI goes rogue, seeks revenge; evolutionary link between prokaryotes and eukaryotes; field of computer science being swamped by AI output; people find AI persuasive, even when they know it’s AI; consensus-based guidelines for assessing AI usefulness; semantic ablation and the problem with AI writing; Dutch study: all-cause mortality declines with COVID vaccination; much more:
AI, STATISTICS & DATA SCIENCE
- “The agent that tried to ruin my reputation is untraceable, unaccountable, and unburdened by an inner voice telling it right from wrong. It is ephemeral, editable, and can be endlessly duplicated. We have no feedback mechanism to correct bad behavior. And without a way to identify AI agents and tie them back to the operators who are responsible for their behavior, we risk having real human voices on the internet completely drowned out.” In a truly mind-boggling account, engineer and blogger Scott Shambaugh recounts how his routine refusal of a request for a code commit by an OpenClaw agentic AI led to a bizarre series of events that included an attempted character smear by the (apparently) unprompted AI and an embarrassing AI-related lapse in journalistic standards by an outlet covering the story.
- “We found that most of the ‘benefit’ tends to relate to older, smaller and leaner forms of machine learning, what has been called ‘traditional AI’, while we also know that most of the new harm is likely stemming from consumer generative AI over-deployment…We also found that the evidence presented for examples of climate benefit, regardless of AI type, tend to be weak….The potential benefits are overstated, in surprising and significant ways.” An analysis by climate writer Ketan Joshi suggests that industry estimates of benefit (real or potential) for the climate from AI use may be misleading or overstated.
- “Computer science was a growing field before the advent of LLMs, but it is now at breaking point. The 2026 International Conference on Machine Learning (ICML) has received more than 24,000 submissions — more than double that of the 2025 meeting….Volume is not the only problem. Many authors fail to properly validate or verify AI-generated contents, says Lee. Analyses of submissions to prominent AI conferences show that some papers are entirely AI-generated and dozens contain AI fabrications, known as hallucinations.” In an article for Nature, Elizabeth Gibney describes how the field of computer science is facing an ungovernable flood of papers, some of dubious merit, enabled by the use of LLM tools.
- “Even the most accurate and well-intentioned AI tools will fail to improve healthcare if they are not used by clinicians and other end users. Usefulness—defined by real-world impact, workflow fit, and contextual adaptability—must be a central benchmark for evaluating AI. Considerations such as usability, trust, and workflow integration, must be considered early and throughout development, rather than deferred to late-stage testing or post-launch adjustments.” A perspective paper, published in the American Journal of Bioethics by Salwei and colleagues, presents a set of consensus-based standards for evaluating the usefulness of health AI, developed by members of the Coalition for Health AI.
BASIC SCIENCE, CLINICAL RESEARCH & PUBLIC HEALTH
- “The movement of the Asgards offers some tantalizing clues to the origin of eukaryotes. They crawl by reshaping their cellular skeleton, building long tentacles that they use to reach out and grip the slide. Other prokaryotes don’t move this way — but eukaryotes do. The videos suggest that Asgards evolved some of the key hallmarks of eukaryotes, such as a skeleton they could use to crawl, long before eukaryotes existed.” The New York Times’ Carl Zimmer reports on recent research findings, published this week in Nature by Appler and colleagues, that illuminate the evolutionary pathways between simple prokaryotic cells (such as bacteria) and much more complex eukaryotic cells.
- “We conducted a retrospective data-linkage study including all Dutch inhabitants to investigate the impact of COVID-19 vaccination on excess mortality using a modified self-controlled case series method. We found a 44% lower relative incidence of all-cause deaths in the first three weeks after the primary vaccination compared to more than three weeks after vaccination….” A research article by Slurink and colleagues, published in the European Journal of Epidemiology, presents findings from a retrospective study encompassing the entire Dutch population that showed a decline in all-cause mortality associated with receiving vaccination for COVID.
- “Over the past few years, thanks to stunning advances in imaging and genetic engineering, scientists have been able to observe and measure crowding inside cells in living organisms for the first time. The experiments have revealed a more dynamic and crowded place than anyone expected, and are the latest evidence that cells actively regulate their internal crowdedness to optimize for the chemical reactions required for life.” In an article for Quanta, Gabriel Popkin unpacks recent research that reveals the interior of cells as a turbulent, active, crowded environment reminiscent of a “crowded nightclub.”
COMMUNICATIONS & Policy
- “When an author uses AI for ‘polishing’ a draft, they are not seeing improvement; they are witnessing semantic ablation. The AI identifies high-entropy clusters – the precise points where unique insights and ‘blood’ reside – and systematically replaces them with the most probable, generic token sequences. What began as a jagged, precise Romanesque structure of stone is eroded into a polished, Baroque plastic shell…” In an article for The Register, Claudio Nastruzzi argues that LLM-generated writing, while often displaying a surface-level smoothness, tends to erode distinctive and engaging qualities.
- “We found that messages were generally persuasive, influencing participants’ views of the policies by 9.74 percentage points on average. However, while 92.0% of participants assigned to the AI and human label conditions believed the authorship labels, labels had no significant effects on participants’ attitude change toward the policies, judgments of message accuracy, nor intentions to share the message with others.” A research article published in PNAS Nexus by Gallegos and colleagues presents findings that show that labelling text as AI-generated had little bearing on whether readers found the information persuasive or worthy of sharing with others,
- “Among the features that Swiss Timing’s new systems provide is blade-angle detection, which gives judges precise technical data to augment their technical and aesthetic decisions. Zobrist says future versions will also determine whether a given rotation is complete, so that ‘if the rotation is 355 degrees, there is going to be a deduction,” he says.’ In an article for IEEE Spectrum, Maurizio Arseni highlights some of the ways AI is undergirding evaluations of athletic performance at this year’s Winter Olympics in Milano/Cortina.
- “Chatbots do not need a narrative hook or a fresh idea to get engaged. Mitch Stoller, a co-founder of Literate AI, an agency with 25 clients focused on influencing the A.I. algorithms, said he stressed to clients that the chatbots sought clarity, thoroughness and an extreme level of detail.” The New York Times’ Erin Griffith describes how the presence of LLM-based online content summaries is reshaping how online brands are able to ensure that reliable information about their companies or products is surfaced by potential customers’ queries.
