Combine and Conquer: A Meta-Analysis on Data Shift and Out-of-Distribution Detection
Keywords: out-of-distribution detection, data distribution shift detection
TL;DR: This paper describes a universal approach to combining detectors and compares combination methods for data distribution shift and out-of-distribution detection through meta-analysis tools..
Abstract: This paper describes a universal approach to combining detectors and compares combination methods for data distribution shift and out-of-distribution detection. By aligning each individual detector score's distribution into p-values through a quantile normalization, we transform the problem into a multi-variate hypothesis test that we combine by leveraging established meta-analysis tools. The resulting test can effectively fuse the individual decision boundaries to create a more capable detector. Furthermore, we can obtain a fully interpretable criterion by reshaping the final statistics of the in-distribution score. Our framework is easily extensible to future development of detection scores. Through a comprehensive empirical investigation, we examine diverse kinds of shifts with different magnitudes and fractions of affected data, showing that our framework is advantageous in improving overall robustness and performance across domains and types of shift and out-of-distribution detection.
Supplementary Material: zip
Primary Area: societal considerations including fairness, safety, privacy
Code Of Ethics: I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Submission Guidelines: I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2024/AuthorGuide.
Anonymous Url: I certify that there is no URL (e.g., github page) that could be used to find authors' identity.
No Acknowledgement Section: I certify that there is no acknowledgement section in this submission for double blind review.
Submission Number: 5984
Loading