AI Health
Friday Roundup
The AI Health Friday Roundup highlights the week’s news and publications related to artificial intelligence, data science, public health, and clinical research.
December 16, 2022
In today’s Duke AI Health Friday Roundup: tools for disaggregating large datasets for bias evaluation; global deaths from COVID may be substantially undercounted; NIH proposes streamlining peer review for grants; engaging with AI issues is important – for everyone; enormous potential for mRNA platforms; “smart bandages” for healing, monitoring wounds; some telehealth companies funnel data to tech and social media companies; much more:
AI, STATISTICS & DATA SCIENCE
- “Addressing fairness and bias in machine learning models is more important than ever! One form of fairness is equal performance across different groups or features. To measure this, evaluation datasets must be disaggregated across the different groups of interest.” A new resource available from Hugging Face AI’s society-ethics space provides a disaggregation tool for large datasets that will help researchers assess evaluation datasets for bias, together with example modules from LAION, Galactica, and the Big Code Project.
- “In this report, we explore the role that academic and corporate RECs play in evaluating AI and data science research for ethical issues, and also investigate the kinds of common challenges these bodies face….The report draws on two main sources of evidence: a review of existing literature on RECs and research ethics challenges, and a series of workshops and interviews with members of RECs and researchers who work on AI and data science ethics.” A report from the Ada Lovelace Institute scrutinizes the use of research ethics committees in the context of projects involving big data and AI.
- “Here, we identify and investigate the following three major categories of methodological errors in ML and DL model development: (a) violation of the independence assumption, (b) the use of inappropriate performance indicators for model evaluation, and (c) the introduction of batch effect. We also provide guidelines for avoiding these pitfalls.” A special report by Maleki and colleagues published in Radiology: Artificial Intelligence address methodological “pitfalls” that can affect the generalizability of machine learning models used in clinical settings.
- “In contrast, the concept of transformative AI is not based on a comparison with human intelligence. This has the advantage of sidestepping the problems that the comparisons with our own mind bring. But it has the disadvantage that it is harder to imagine what such a system would look like and be capable of. It requires more from us. It requires us to imagine a world with intelligent actors that are potentially very different from ourselves.” At Our World in Data, Max Roser makes a case for widespread societal engagement with issues raised by the rapid progression (and deployment) of AI systems.
- “…if implemented effectively, the October 6 regulation has the potential to move the country closer to the HITECH Act’s ultimate vision: a national capability for health information exchange. Such an outcome would most likely be welcomed by patients and clinicians alike and could both improve the quality of health care and reduce its cost.” In a perspective article for the New England Journal of Medicine, David Blumenthal unpacks the implications of recently enacted regulations that will affect how health data is handled by electronic health record vendors and health information networks.
BASIC SCIENCE, CLINICAL RESEARCH & PUBLIC HEALTH
- “We estimate 14.83 million excess deaths globally, 2.74 times more deaths than the 5.42 million reported as due to COVID-19 for the period. There are wide variations in the excess death estimates across the six World Health Organization regions. We describe the data and methods used to generate these estimates and highlight the need for better reporting where gaps persist.” A research article by Msemburi and colleagues, published this week in Nature, analyzes global patterns of mortality from COVID for 2020-2021 and finds that previous estimates likely reflect a substantial (by nearly a factor 2.75) undercount of COVID mortality.
- “The sheer breadth of disorders with a potential new, fresh approach, many of which have heretofore had no means of therapy or prevention, is staggering. We’ll look back at the pandemic someday as an extraordinary renaissance in biomedicine, giving rise to a validated pluripotent platform with seemingly unlimited refinements and applications.” In a lengthy post at Eric Topol’s Substack, the cardiologist and clinical trialist enumerates the potential of mRNA technologies for an enormous array of applications – ones that go well beyond vaccines for viral diseases.
- “Skin wounds on mice treated with electrical stimulation provided by the smart bandage healed about 25% more quickly than those covered with a standard sterile dressing. The new skin on the mice who got the smart bandage showed an increase in new blood vessels. It was also thicker and stronger than that on mice given standard bandages. Similar results were seen in mouse models of burn healing and diabetic wounds…When the researchers examined cells from mice given the smart bandage, they found that certain types of immune cells had increased activity of genes involved in tissue regeneration.” A post at the National Institutes of Health’s Research Matters blog describes recent research into the use of wireless “smart bandages” to promote healing while also monitoring its progress.
- “Among older adults with subjective cognitive concerns, mindfulness training, exercise, or both did not result in significant differences in improvement in episodic memory or executive function at 6 months. The findings do not support the use of these interventions for improving cognition in older adults with subjective cognitive concerns.” A paper published in JAMA this week by Lenze and colleagues reports on a randomized study of the effects of mindfulness training and exercise (alone or together) on cognitive function in older volunteers with “subjective concerns” about cognitive decline.
COMMUNICATION, Health Equity & Policy
- The National Institutes of Health is soliciting comments on a proposal to streamline the process of peer review for research grant submissions: “NIH is proposing a revised, simplified framework for peer review that will reorganize the major regulatory criteria (42 C.F.R. Part 52h.8) under three factors and reduce the number of non-score driving review considerations that peer reviewers evaluate in research project grant (RPG) applications during the first stage of review. NIH believes that these changes will allow peer reviewers to refocus on the critical task of assessing scientific merit and will improve those assessments by reducing bias.”
- “PLOS has been looking into more than 100 articles by Raoult, but determined that the issues in 49 of the papers, including reuse of ethics approval reference numbers, warrant expressions of concern while the publisher continues its inquiry.” Retraction Watch reports that scientific publisher PLOS is investigating a slew of research articles authored by controversial French scientist Didier Raoult, who advocated hydroxychloroquine as a therapy for COVID.
- “The trackers that STAT and The Markup were able to detect, and what information they sent, is a floor, not a ceiling. Companies choose where to install trackers on their websites and how to configure them. Different pages of a company’s website can have different trackers, and this analysis did not test every page on each company’s site.” An investigation by STAT News and The Markup reveals that multiple telehealth companies that emerged in response to rocketing demand during the COVID pandemic were funneling extremely sensitive personal data to tech and social media companies via web trackers.
- “One of the main things we have learned during the development of the toolkit is that many of the activities that make the most impactful difference are small-scale and low-cost, involve behaviors rather than huge infrastructure shifts, and are changes that organizations of any size can implement. We also recognize that inclusion is a journey, and everybody needs to start somewhere.” At Scholarly Kitchen, a guest post by shares “quick wins” for making a workplace more inclusive for persons with disabilities.
