Title: The Algorithm Keeps The Score: Identity, Marginalization, and Power in the Technology-Mediated Search for Care
Date: Thursday, May 30, 2024
Time: 10:30am - 1:30pm EST
Location (in-person): CODA C1015 Vinings
Location (Zoom): sachin.phd/defense
Sachin Pendse
PhD Candidate in Human-Centered Computing
School of Interactive Computing
Georgia Institute of Technology
Committee:
Dr. Munmun De Choudhury (Co-Advisor) – School of Interactive Computing, Georgia Institute of Technology
Dr. Neha Kumar (Co-Advisor) – School of Interactive Computing, Georgia Institute of Technology
Dr. Michael L. Best – School of Interactive Computing, Georgia Institute of Technology
Dr. Andrea G. Parker – School of Interactive Computing, Georgia Institute of Technology
Dr. Jessica L. Schleider – Feinberg School of Medicine, Northwestern University
Dr. Amit Sharma – Microsoft Research
Abstract
Severe psychological distress and mental illness are widespread, affecting one in two people globally. Identity-based marginalization and power have long played a core role in how individuals navigate their experience of distress, including the search for culturally valid resources and care. Alongside identity, marginalization, and power, technologies play an increasing role in how people engage with care, including mental health helplines, online support communities, and large language model (LLM) chatbots. In turn, the design of these technologies can also influence people's illness experiences as they make meaning from distress and search for care. I understand these support technologies to be technology-mediated mental health support (TMMHS) systems. My work asks the question: what do we gain and lose when technology (and the algorithms that underlie it) keep the score around our experiences with mental health?
To answer this question, I examine how identity, power, and marginalization intersect with the design of TMMHS systems to influence people's experiences with distress and care. Through use of both quantitative and qualitative methods, I highlight how traditional approaches to understanding mental health experiences can often erase the role of identity-based marginalization, both online and offline.
My dissertation begins by outlining the history of mental health support in traditional and technology-mediated contexts, discussing the role of colonial power relations in how mental illness has been understood, measured, and treated across my study contexts in India and the U.S. I then explore four areas where these power relations directly influence engagement with TMMHS systems. First, I investigate the use of Indian mental health helplines, analyzing the role of identity in how individuals in distress engage with care, and where structural and technical gaps prevent access. Next, I shift to resource-constrained areas of the United States, investigating how individuals in mental health professional shortage areas use TMMHS systems to create new identities from their experiences of distress. I examine the role of technical design and marginalization in what I find to often be deeply polarized environments within TMMHS systems. Building on this finding, I then analyze differences in engagement with TMMHS systems between U.S. Republicans and Democrats, given an increasing polarization of partisan identity in the U.S. today. Finally, I investigate identity-based biases within LLM-based chatbots, a new and emergent form of TMMHS support. Across my empirical work, I leverage the language people use around their distress as a tool to analyze identity-based differences.
I conclude by proposing that incorporating considerations of identity, power, and marginalization in the design of TMMHS systems could mitigate the limitations of past approaches to TMMHS system design, ensuring that diverse individuals can leverage technology for their mental health needs. Building on my empirical work, I suggest these considerations include designing technologies that strengthen human support relationships, center an awareness of differences across diverse identities, and keep a constant eye to patterns of historical marginalization.