Understanding User Trust and Expectations in Answerable Sociotechnical Systems

Dilara Keküllüoğlu, Yuqi Niu, Michael Rovatsos, Nadin Kökciyan

Abstract: Today, we see an increase in the number of systems where artificial intelligence and autonomous systems (AI/AS) are included in decision-making processes, even in high-stakes domains, such as health care. It is often not clear how to take responsibility appropriately and provide restitution when significant harms happen, especially when organizations use opaque AI/AS to make decisions. We focus on the notion of responsibility as answerability, which demands that individuals have a duty to supply their “reasons and justifications” for an action, or respond to inquiries about why they acted in a particular manner. We investigate the trust toward organizations that use mediator agents, people’s expectations from such agents, and perceived benefits and risks of using this new intervention to enable answerability in organizations. Our findings indicate that providing explanations and offering remedies for harm through mediator agents does not affect the level of trust individuals have in organizations.

External IDs:doi:10.1109/mic.2025.3633484