Accounting for AI and Users Shaping One Another: The Role of Mathematical Models
Abstract: As AI systems enter into a growing number of societal domains, these systems increasingly shape and are shaped by user preferences, opinions, and behaviors. However, the design of AI systems rarely accounts for how AI and users shape one another. In this survey paper, we argue for the development of \textit{formal interaction models} which mathematically specify how AI and users shape one another. Formal interaction models can be leveraged to (1) specify interactions for implementation, (2) monitor interactions through empirical analysis, (3) anticipate societal impacts via counterfactual analysis, and (4) control societal impacts via interventions. The design space of formal interaction models is vast, and model design requires careful consideration of factors such as style, granularity, mathematical complexity, and measurability. Using content recommender systems as a case study, we critically examine the nascent literature of formal interaction models with respect to these use-cases and design axes. More broadly, we call for the community to leverage formal interaction models when designing, evaluating, or auditing any AI system which interacts with users.
Submission Length: Regular submission (no more than 12 pages of main content)
Changes Since Last Submission: - Clarifying connection/comparison with RL and causal modelling (introduction, section 2.2)
- Clarifying individual vs. societal level impacts (section 2.1)
- Revised simple example to better illustrate connection between formal model and societal impacts (section 2.1)
- New summary figure on design axes and use cases (section 2.2)
- More detail on game theoretic model style (section 2.3)
- Further comment on measurability and unobservability (section 2.3)
- Illustration of design axes with simple example (section 2.3)
- New summary table on literature review (section 3)
- More detail on inclusion criteria for papers in our analysis (section 3)
- Minor changes to wording, typos, etc (throughout)
Assigned Action Editor: ~Sebastian_Tschiatschek1
Submission Number: 3507
Loading