Doubly Calibrated Estimator for Recommendation on Data Missing Not At Random
Keywords: Recommender System, Collaborative Filtering, Missing Not At Random, Doubly Robust, Calibration
TL;DR: We propose a Doubly Calibrated Estimator based on the theoretical insights into how miscalibrated imputation and propensity models may limit the effectiveness of existing doubly robust estimators.
Abstract: Recommender systems often suffer from selection bias as users tend to rate their preferred items.
The datasets collected under such conditions exhibit entries missing not at random and thus are not randomized-controlled trials representing the target population.
To address this challenge, a doubly robust estimator and its enhanced variants have been proposed as they ensure unbiasedness when accurate imputed errors or predicted propensities are provided.
However, we argue that existing estimators rely on miscalibrated imputed errors and propensity scores as they depend on rudimentary models for estimation.
We provide theoretical insights into how miscalibrated imputation and propensity models may limit the effectiveness of doubly robust estimators and validate our theorems using real-world datasets.
On this basis, we propose a Doubly Calibrated Estimator that involves the calibration of both the imputation and propensity models.
To achieve this, we introduce calibration experts that consider different logit distributions across users.
Moreover, we devise a tri-level joint learning framework, allowing the simultaneous optimization of calibration experts alongside prediction and imputation models.
Through extensive experiments on real-world datasets, we demonstrate the superiority of the Doubly Calibrated Estimator in the context of debiased recommendation tasks.
Track: User Modeling and Recommendation
Submission Guidelines Scope: Yes
Submission Guidelines Blind: Yes
Submission Guidelines Format: Yes
Submission Guidelines Limit: Yes
Submission Guidelines Authorship: Yes
Student Author: Yes
Submission Number: 1811
Loading