In today’s Duke AI Health Friday Roundup: large language model gets pushback from scientists; competition among viruses may blunt effects of feared winter “tripledemic”; oversight for machine learning software in healthcare; fruit fly connectome resembles machine learning architectures; “evidence map” for maternal health risk factors; the importance of trust between patients and physicians; much more:
AI, STATISTICS & DATA SCIENCE
- “A fundamental problem with Galactica is that it is not able to distinguish truth from falsehood, a basic requirement for a language model designed to generate scientific text. People found that it made up fake papers (sometimes attributing them to real authors), and generated wiki articles about the history of bears in space as readily as ones about protein complexes and the speed of light. It’s easy to spot fiction when it involves space bears, but harder with a subject users may not know much about.” At MIT Technology Review, Will Douglas Heaven chronicles the November release (and rapid withdrawal) of Galactica, Meta’s new large language model, which was touted as a tool for science and research in particular.
- “The takeaway is clear: the onus should be on the company selling the AI tool to proactively justify its validity. Without such evidence, we should treat any risk assessment tool as suspect. And that includes most tools on the market today.” A post by Sayash Kappor and Arvind Narayanan at AI Snake Oil describes the phenomenon of predictive algorithmic models being touted for risk prediction but then not performing as advertised.
- “…this paper outlines an ease of social distancing index for sub-Saharan Africa, intended to identify locations where social distancing in urban areas is likely to be very difficult. The index incorporates residential population density and urban form metrics, which are calculated from new geospatial datasets available for sub-Saharan Africa, with index values mapped for small spatial units within urban areas.” A new paper published in Scientific Data by Chamberlain and colleagues describes the use of geolocation data to create high-resolution estimates of the feasibility of social distancing public health efforts in sub-Saharan Africa.
- “[Machine learning-based clinical decision support tools] have the potential to automatically identify patterns and assign health risk in ways that human medical practitioners are not capable of doing. If implemented correctly, this has the potential to provide a more optimized and less expensive health care. However, there is also a downside where end users—who do not necessarily understand how the underlying ML algorithms operate—need to trust the accuracy of such tools.” In a commentary for JAMA Network Open, Goldstein, Mazurowski, and Li present the case for labeling some kinds of machine-learning-based software as medical devices.
BASIC SCIENCE, CLINICAL RESEARCH & PUBLIC HEALTH
- “We found pervasive multisensory and interhemispheric integration, highly recurrent architecture, abundant feedback from descending neurons, and multiple novel circuit motifs. The brain’s most recurrent circuits comprised the input and output neurons of the learning center. Some structural features, including multilayer shortcuts and nested recurrent loops, resembled powerful machine learning architectures.” A preprint by Winding and colleagues, available at bioRxiv, describes a successful attempt to map the entire connectome of a fruit fly larva’s brain.
- “Even with the pandemic still taking a heavy toll, prospects have dimmed for the two most coveted kinds of next-generation vaccines: nasal sprays that can block more infections, and universal coronavirus shots that can defend against a wider array of ever-evolving variants.” An article by the New York Times’ Benjamin Mueller describes headwinds affecting efforts to ensure the availability of “next-generation” COVID vaccines.
- “Identifying risk factors pregnant and birthing people face is vitally important. However, limited depth and quality of available research within each social and structural determinant of health impeded our ability to understand underlying mechanisms, including risk factor interdependence.” The Agency for Healthcare Research and Quality (AHRQ) has published a draft version (currently open for comment) of an “evidence map” of risk factors for maternal morbidity and mortality.
- “…a growing body of epidemiological and laboratory evidence offers some reassurance: SARS-CoV-2 and other respiratory viruses often “interfere” with each other. Although waves of each virus may stress emergency rooms and intensive care units, the small clique of researchers who study these viral collisions say there is little chance the trio will peak together and collectively crash hospital systems the way COVID-19 did at the pandemic’s start.” An article by Science’s Jon Cohen offers some possible good news about the prospects for a winter “tripledemic” as COVID, flu, and RSV are all in circulation.
COMMUNICATION, Health Equity & Policy
- “The health care community is right to be concerned about emerging evidence that patients increasingly distrust the health care system at large, as well as the uptick in violence and other harms perpetrated against health care workers. It’s not hyperbolic to say that trust is what the whole health care delivery system runs on—and where it’s absent, we see major breakdowns. But the same is true about the trust that needs to flow from doctors to patients.” In a viewpoint article for Health Affairs’ Forefront blog, Zink and colleagues describe the importance of bidirectional trust between patients and physicians – and the threat that its absence poses to the entire healthcare system.
- “These out-of-context secondary uses of data—including its sale to and use by data brokers, surveillance advertising firms, and other entities trafficking in consumer profiles—and the overcollection that feeds them are inconsistent with the reasonable expectations of online consumers. These unfair commercial surveillance practices lead to invasive, discriminatory targeting that violates the privacy and autonomy of consumers.” The Electronic Privacy Information Center has responded to a notice of proposed rulemaking regarding “commercial surveillance and data security” by the Federal Trade Commission with a lengthy and detailed report that argues the need for a “data minimization rule.”
- “Advancements in artificial intelligence (AI) are moving faster than the State’s ability to fully govern it, resulting in a need for innovative approaches that also involve non-state actors, or “co-governance” mechanisms. But the question remains as to exactly how co-governance mechanisms can be incorporated into AI governance in the EU.” In a chapter from the 2021 Yearbook of the Digital Ethics Lab, Caitlin C. Corrigan examines potential directions for AI policy development efforts in the European Union.
- “…if an academic journal isn’t validating its content in some way, I’d argue it’s not an academic journal, because the validation element is essential to the act of publishing academic journals. This may seem like splitting hairs, but academic journals have provided validated content to authors — and readers and others in society — for over 350 years, and it’s that history that provides the expectation of a certain type of service.” In an interview with Scholarly Kitchen’s Lisa Janicke Hinchliffe, author Simon Linacre talks about the impetus for his book The Predator Effect, which scrutinizes the impact of predatory publishing practices in academia.