Generative AI in Health Care and Public Health: Opportunities and Challenges

In both health care and public health — distinct yet intertwined realms — generative AI is beginning to transform operational paradigms. Clinical environments are utilizing generative AI tools for tasks like drafting clinical notes and crafting messages to patients. Meanwhile, the public health sector is exploring how these technologies can tailor health communications to meet the unique needs of various communities.

Adoption with Caution

Despite the promising applications, the integration of generative AI in these fields often surfaces concerns regarding transparency, oversight, and engagement with affected communities. Frequently, developments are spearheaded without meaningful input from the populations most impacted by these technologies, especially those grappling with systemic inequity.

The landscape of health care is notably personal, precarious, and shaped by entrenched exclusions that undermine the well-being of racially minoritized, queer, transgender, and disabled individuals. When generative AI systems are designed without accounting for this nuanced context, the potential for harm escalates significantly.

Historical Precedents of Harm

The risks associated with AI are not merely speculative; they have already manifested in predictive models. For example, one widely used algorithm underestimated the necessary follow-up care for Black patients, resulting in fewer referrals compared to their white counterparts. This disparity arose because the algorithm used health care expenditures as a proxy for medical necessity, neglecting the reality that lower spending among Black patients was often due to structural barriers in care access, including bias from providers.

Similarly, AI-driven risk stratification tools have subjected disabled individuals to compromised treatment options during the Covid-19 pandemic, prioritizing assessments based on flawed assumptions about quality of life. These tools often rely on biased, incomplete data, risking the reinforcement of existing health disparities while lacking transparency and accountability.

Missing Community Voices

Had community members been part of the development process for these AI systems, they would have likely pointed out the flawed assumption that reduced spending indicated less need for services. A community-informed approach could have emphasized more equitable metrics and ongoing monitoring to identify unintentional inequities resulting from the systems’ functioning. Rather than focusing solely on efficiency and cost reduction, such an initiative might have prioritized equitable service access.

Generative AI in Public Health

As generative AI starts to find its place in public health, agencies like the CDC are harnessing these technologies to monitor social media data for school closures or forecast trends in overdose rates. However, as public health systems remain chronically underfunded and overstretched, a rush to adopt readily available generative AI models could sideline community input and governance, stifling opportunities for meaningful accountability.

This slower uptake in public health, however, could serve as a critical window to embed community engagement before these systems are widely implemented. Decision-making processes regarding AI in health must invite the voices of those impacted by these technologies, many of whom have historically been excluded from discussions shaping tech in the health sector.

Community-Centered Approaches

Generative AI must evolve from a community-centered rather than a top-down model. Notable frameworks are already proposing this realignment. For instance, Zainab Garba-Sani’s ACCESS AI framework emphasizes engaging communities, addressing barriers to AI integration, and ensuring equity is interwoven throughout the development and implementation of AI in health care.

Furthermore, initiatives like the Grounded Innovation Lab @ Health Justice aim to bolster accountability, equity, and transparency in generative AI applications. In this model, communities decide what types of training data should be utilized, approving narratives that reflect their experiences and rejecting those that could perpetuate bias or misinformation.

Redefining Success Metrics

To elevate governance standards, outcomes should not be judged merely on technical performance benchmarks but also on community priorities like trust and confidence. Feedback from lived experiences should be considered crucial data in refining AI models over time. Regular meetings involving community members, health AI developers, and stakeholders should prioritize accessibility and emphasize compensating contributions of community expertise.

The Grounded Innovation Lab not only focuses on health care but also acknowledges the broader public health landscape. Given the operational complexities in public health, embracing community governance is not optional; it is imperative. Moreover, the environmental impact of AI, particularly the location of resource-intensive data centers in marginalized communities, underscores the urgent necessity for inclusive governance frameworks.

Environmental and Social Accountability

As generative AI systems evolve, awareness of their environmental costs is vital, especially for populations already facing severe health inequities. These communities often shoulder the dual burdens of health challenges and environmental degradation linked to AI technologies. By actively engaging with local populations, developers can co-create alternative solutions, such as using smaller, domain-specific models that minimize environmental impacts while maximizing accessibility.

The pressing imperative now is to mitigate risks while simultaneously exploiting generative AI’s considerable potential to advance health equity. The recent rejection by the Senate of a proposed ban on state regulation of AI highlights the need for community-led models that shape the design, implementation, and governance of both generative AI and technology in health broadly. By fostering participatory processes and accountability measures, we can avert potential harm and ensure that those most affected dictate the technologies they engage with daily.

The Path Forward

Health care provides an opportunity to cultivate a field marked by equity, transparency, and accountability, shaping the future landscape of AI applications across various sectors. As the discourse around generative AI unfolds, prioritizing community perspectives is not just beneficial but essential for building systems that truly reflect the diverse needs of all populations.

In sum, the journey toward integrating generative AI in health care and public health should spotlight the voices and concerns of those most impacted, crafting a future where technology enhances rather than excludes.

This structured exploration delves into the intricacies surrounding generative AI in health care and public health, emphasizing the importance of community involvement. If you seek deeper insights or specific case studies within this domain, feel free to express your interests.