AI Health
Friday Roundup
The AI Health Friday Roundup highlights the week’s news and publications related to artificial intelligence, data science, public health, and clinical research.
November 22, 2024
In this week’s Duke AI Health Friday Roundup: are LLMs facing a cliff in performance?; role of digital health in combating antimicrobial resistance; AI’s effects on job markets already being felt; epigenetic changes and obesity; how stress warps memory, feeds anxiety; developing medical AI curricula via Delphi; tech bans and teen mental health; growing strains on scientific publishing; operator guidance can help bystanders in performing CPR; more:
AI, STATISTICS & DATA SCIENCE
- “The canary in the coal mine came when The Information reported this week that behind the scenes, OpenAI researchers discovered that its upcoming flagship model, code-named Orion, demonstrated noticeably less improvement over its predecessor GPT-4, than GPT-4 did over GPT-3. In areas like coding — a major appeal for these LLMs — there may even be no improvements at all.” At Futurism, Frank Landymore reports on a round of pessimism in some parts of the AI world as the scaling approaches used with large language models start encountering difficulties – and explores questions of whether new approaches could rev up improvements again.
- “This study compared the performance of Sequential Multiple Assignment Randomized Trial (SMART) designs to traditional A/B testing in detecting heterogeneous treatment effects (HTEs) and optimizing patient engagement strategies. Our findings reveal distinct advantages and limitations for each method, depending on the timing of HTEs within the sequential design and the complexity of the effects. SMART designs demonstrated superior power in detecting HTEs associated with later stages of randomization, particularly as effect sizes increased. This advantage was most pronounced at larger sample sizes, where SMART designs achieved near-perfect power for detecting large HTEs in later stages, while A/B testing showed only modest improvement.” A research article published in NPJ Digital Medicine by Basu and colleagues reports on a simulation study that compared trial designs for their ability to detect variations in treatment effects.
- “Digital tools can aid early diagnosis of bacterial infectious disease, addressing the need for accurate diagnoses to prevent delays the infection recognition, improper risk stratification, and a poor understanding of local AMR patterns. However, many low-income and middle-income countries (LMICs) face challenges such as poor infrastructure, insufficient investment and human resources, underuse of available health-care data, and inadequate dissemination of information to regulatory bodies.” An editorial appearing in Lancet Digital Health introduces a series of articles addressing the potential role for digital health tools in combating antimicrobial resistance.
- “A panel of 18 experts in health and AI across Canada participated in three rounds of surveys to determine essential AI learning competencies. The study identified key curricular components across ethics, law, theory, application, communication, collaboration, and quality improvement. The findings demonstrate substantial support among medical educators and professionals for the inclusion of comprehensive AI education, with 82 out of 107 curricular competencies being deemed essential to address both clinical and educational priorities. It additionally provides suggestions on methods to integrate these competencies within existing dense medical curricula.” An article published in NPJ Digital Medicine by Singla and colleagues presents a Delphi-method-based study that was used to develop medical training curricula for artificial intelligence.
BASIC SCIENCE, CLINICAL RESEARCH & PUBLIC HEALTH
- “Women are less likely to receive CPR from a bystander than men when stricken with a cardiac arrest outside of a hospital, Duke Health researchers found. This disparity was eliminated, however, when 911 operators guided an emergency caller through CPR steps, helping ease apprehensions about administering chest compressions to women. When guided by a 911 operator, bystanders performed CPR on women 44% of the time and on men 40% of the time. Without this guidance, bystander CPR dropped to 9% for women and 11% for men.” Duke Health’s Stephanie Lopez reports on a recent study that sheds light on factors affecting the willingness of bystanders to perform CPR in emergencies.
- “Historically, though, ‘there’s been a lot of resistance’ to the idea that infections could play a role in dementia, said Nagel, a neurology professor at the University of Colorado School of Medicine. ‘There was a lot of skepticism.’ And yet, she pointed out, before penicillin became available to treat syphilis, that infection was one of the main causes of dementia. But the notion that infections—with shingles as the leader of the pack—might contribute to the development of dementia has been gaining traction as enthusiasm for the conventional wisdom about the etiology of Alzheimer disease wanes.” A perspective article by JAMA’s Rita Rubin examines recent evidence that shingles may affect the risk of developing dementia – and that the shingles vaccine may afford protection from those effects.
- “…by using single-nucleus RNA sequencing, we show that both human and mouse adipose tissues retain cellular transcriptional changes after appreciable weight loss. Furthermore, we find persistent obesity-induced alterations in the epigenome of mouse adipocytes that negatively affect their function and response to metabolic stimuli. Mice carrying this obesogenic memory show accelerated rebound weight gain, and the epigenetic memory can explain future transcriptional deregulation in adipocytes in response to further high-fat diet feeding.” A research article by Hinte and colleagues, published in Nature, presents findings from an RNA sequencing study that suggests epigenetic changes may play a significant role in the biology of weight gain.
- “Stress has previously been observed to induce the generalization of threat/aversive memories,1 but the neurobiological processes mediating this effect have been elusive. Here we used a combination of tools to examine both the molecular and local circuit processes mediating this observation. We found that acute stress drives threat memory generalization at the level of the engram ensemble in the LA.” A research article published in Cell by Lesuis and colleagues presents findings from a mouse study that elucidate how stress can interfere with memory and fuel persistent anxiety.
COMMUNICATION, Health Equity & Policy
- “Social media can be like gasoline on the fires that are burning some teens out. Big tech may deserve the attention it’s getting, particularly as some companies resist changes that would benefit teen well-being at the expense of “time spent” on apps…But to fix mental health trends, we also need to widen our lens beyond Instagram and TikTok: to other technologies, and even to the Strava-fication of school in the form of educational technology platforms that continually pressure students and parents with endless performance updates. We need to contend with the pressures outside of social media, too.” An opinion article by Weinstein and Konrath, published in STAT, expresses skepticism that broad-spectrum bans for particular kinds of tech or social media will be effective in improving the mental health of teens.
- “…publication workload per scientist has increased dramatically. We define this problem as “the strain on scientific publishing.” To analyze this strain, we present five data-driven metrics showing publisher growth, processing times, and citation behaviors. We draw these data from web scrapes, and from publishers through their websites or upon request. Specific groups have disproportionately grown in their articles published per year, contributing to this strain.” A research article published in Quantitative Science Studies by Hanson and colleagues attempts to map patterns of growing stress on academic publishing enterprise – and on the authors and reviewers at the heart of the system.
- “In new research, forthcoming in Management Science, we explore the impact gen AI has already had on the labor market by examining trends in demand for online freelancers. Our findings show significant short-term job replacement after these tools were introduced, and that jobs prone to automation, like writing and coding, were the most affected by ChatGPT. Our research also examines how competition, job requirements, and employer willingness-to-pay have changed to better understand how the online job market is evolving with the rise of gen AI.” An article published in Harvard Business Review by Ozge Demirci, Jonas Hannane and Xinrong Zhu examines how the use of generative AI is changing the labor market.
