ARC: A Tool to Rate AI Models for Robustness Through a Causal Lens

Kausik Lakkaraju, Siva Likitha Valluru, Biplav Srivastava, Marco Valtorta

Keywords: AI Robustness, Causal Analysis, Rating

TL;DR: We present a tool for evaluating the robustness of AI models through causal analysis.

Abstract: As Artificial Intelligence (AI) systems become more powerful, concerns about trust issues such as bias hinder their large-scale adoption. Bias may arise with respect to protected attributes, including well-studied factors like gender, race, and age. In this paper, we introduce ARC, a tool to rate AI models for robustness, encompassing both bias and robustness against perturbations, along with accuracy through a causal lens. The main objective of the tool is to assist developers in building better models and aid end-users in making informed decisions based on the available data. The tool is extensible and currently supports four different AI tasks: binary classification, sentiment analysis, group recommendation, and time-series forecasting. It allows users to select data for a task and rate AI models for robustness, assessing their stability against perturbations while also identifying biases related to protected attributes. The rating method is model-independent, and the ratings produced are causally interpretable. These ratings help users make informed decisions based on the data at hand. The demonstration video is available at: https://tinyurl.com/muwujxfv.

Paper Type: New Short Paper

Submission Number: 2