Missing Pieces: How Framing Uncertainty Impacts Longitudinal Trust in AI Decision Aids - A Gig Driver Case Study

Rex Chen, Ruiyi Wang, Norman Sadeh, Fei Fang

Abstract: Decision aids based on artificial intelligence (AI) induce a wide range of outcomes when they are deployed in uncertain environments. In this paper, we investigate how users' trust in recommendations from an AI decision aid is impacted over time by designs that expose uncertainty in predicted outcomes. Unlike previous work, we focus on gig driving - a real-world, repeated decision-making context. We report on a longitudinal mixed-methods study ($n=51$) where we measured gig drivers' trust as they interacted with an AI-based schedule recommendation tool. Our results show that participants' trust in the tool was shaped by both their first impressions of its accuracy and their longitudinal interactions with it; and that task-aligned framings of uncertainty improved trust by allowing participants to incorporate uncertainty into their decision-making processes. Additionally, we observed that trust depended on their characteristics as drivers, underscoring the need for more in situ studies of AI decision aids.