Aggregating Correlated Estimations with (Almost) no Training

Théo Delemazure, François Durand, Fabien Mathieu

Abstract: Many choice problems cannot be solved exactly and use several estimation algorithms that assign scores to the different available options. The estimation errors can have various correlations, from low (e.g. between two very different approaches) to high (e.g. when using a given algorithm with different hyperparameters). Most aggregation rules would suffer from this diversity of correlations. In this article, we introduce Embedded Voting (EV), an aggregation rule designed to take correlations into account, and we compare it to other aggregation rules in various experiments based on synthetic data. Our results show that when sufficient information about the correlations between errors is available, a maximum likelihood aggregation should be preferred. Otherwise, typically with limited training data, EV outperforms the other approaches.