AI Failure Cards: Understanding and Supporting Grassroots Efforts to Mitigate AI Failures in Homeless Services

Ningjing Tang, Jiayin Zhi, Tzu-Sheng Kuo, Calla Kainaroi, Jeremy J. Northup, Kenneth Holstein, Haiyi Zhu, Hoda Heidari, Hong Shen

Abstract: AI-based decision support tools have been used in a wide range of high-stakes settings. However, many of them have failed. Past literature in FAccT contributes important insights into how to detect and mitigate AI failures from a technical perspective. Recently, there are growing calls to understand AI failures as socio-technical and to support community-centered, grassroots-based mitigations to AI failures, in addition to top-down approaches. In this paper, we present AI Failure Cards, a novel method for both improving communities’ understanding of AI failures and for eliciting their current practices and desired strategies for mitigation, with a goal to better support those efforts in the future. Through a series of workshops with unhoused individuals, frontline workers and service providers, as well as local policy advocates, we conducted an empirical investigation of our method in the context of a locally deployed predictive housing allocation algorithm. Our results suggest that the use of the method helped impacted communities better understand these AI failures. It also surfaced a wide range of existing grassroots practices and desired mitigation strategies. Finally, we discuss both the challenges and opportunities for supporting grassroots efforts in mitigating AI failures.

External IDs:doi:10.1145/3630106.3658935