A Surrogate Policy Model for Auditing Black-Box Recommendation Systems: Application to Change Detection

Marouane Bazzaoui, Matthieu Jonckheere, Erwan Le Merrer, Gilles Tredan

Keywords: Auditing, recommender systems, surrogate models, inverse reinforcement learning.

TL;DR: We propose a method for auditing Black-Box recommender systems using a surrogate model adapted from an IRL-based approach, with formal guarantees in a favorable setting.

Abstract: Recommender systems increasingly shape information exposure. As a result, auditing them has become a growing necessity. A key challenge is to understand what can be inferred about a recommender's behaviour from black-box observations alone, \textit{i.e.}, without access to its internals. In this paper, we propose a method to audit recommender systems using a surrogate policy model. This surrogate policy estimator provides a local approximation of the recommender system’s behaviour with a characterized approximation error. We establish the consistency and asymptotic normality of this estimator, enabling hypothesis testing. We then propose a change detection task for assessing whether or not the recommender has updated its behaviour.

Email Sharing: We authorize the sharing of all author emails with Program Chairs.

Data Release: We authorize the release of our submission and author names to the public in the event of acceptance.

Submission Number: 21