IS CONFIDENCE ALL YOU NEED? EXPLORING HUMAN-AI JOINT DECISION-MAKING IN SPATIOTEMPORAL ROBOTIC TASKS

An Duc Nguyen, Raunak Bhattacharyya, Clara Colombatto, Stephen M. Fleming, Ingmar Posner, Nick Hawes

Track: long paper (up to 10 pages)

Keywords: joint decision-making; meta-cognition; dynamic human-AI interactions; confidence-calibration; high-stakes domain AI-human interactions

TL;DR: Human-AI joint decision-making in high-stake spatiotemporal task such as robotic is crucial yet largely unexplored. Our research addresses: How do humans benefit from AI recommendations in these tasks?

Abstract: The growing integration of agentic artificial intelligence technologies into human workflows has introduced a new paradigm of AI-assisted decision-making. While previous research has demonstrated that collaboration between humans and AI can lead to higher accuracy than either working alone, such studies have predominantly focused on static and passive tasks, such as price prediction, recidivism risk assessment, conversation and content moderation. In this study, we explore human-AI joint decision-making in a dynamic spatiotemporal robotic task, where humans tele-operate robots. Using human-subject experiments involving 100 participants, we evaluated a teleoperation task in which participants chose between two mobile robots in a simulation, guided by an AI agent providing its confidence level. Our findings reveal that human meta-decisions - particularly in resolving disagreements between humans and AI - are often suboptimal and confidence-driven frameworks such as Maximum Confidence Slating (MCS) can significantly enhance joint decision-making outcomes ($p < 0.001$). To the best of our knowledge, this is the first application of MCS in a human-AI joint decision-making context. Moreover, we discovered that both well-calibrated and poorly-calibrated AI agents influence human decision accuracy. However, a well-calibrated AI agent that effectively represents its confidence can lead to better decision outcomes, while poorly calibrated AI is more likely to steer users toward negative changes in their decision-making process. These results highlighted the importance of well-calibrated AI confidence levels in fostering effective collaboration and enhancing human-AI joint decision-making performance in complex spatiotemporal tasks.

Submission Number: 13