Risk-limiting Financial Audits via Weighted Sampling without Replacement

Shubhanshu Shekhar, Ziyu Xu, Zachary Chase Lipton, Pierre Jinghong Liang, Aaditya Ramdas

Keywords: Confidence sequences, importance sampling, control variate, risk-limiting audit, sampling without replacement

TL;DR: A general approach to auditing financial transactions using confidence sequences for estimating weighted means under adaptive sampling without replacement.

Abstract: We introduce the notion of risk-limiting financial audits~(RLFA): procedures that manually evaluate a subset of $N$ financial transactions to check the validity of a claimed assertion $\mathcal{A}$ about the transactions. More specifically, RLFA satisfy two properties: (i) if $\mathcal{A}$ is false, they correctly disprove it with probability at least $1-\delta$, and (ii) they validate the correctness of $\mathcal{A}$ with probability $1$, if it is true. We propose a general RLFA strategy, by constructing new confidence sequences~(CSs) for the weighted average of $N$ unknown values, based on samples drawn without replacement from a (randomized) weighted sampling scheme. Next, we develop methods to improve the quality of CSs by incorporating side information about the unknown values. We show that when the side information is sufficiently accurate, it can directly drive the sampling. For the case where the accuracy is unknown \emph{a priori}, we introduce an alternative approach using control variates. Crucially, our construction adapts to the quality of side information by strongly leveraging the side information if it is highly predictive, and learning to ignore it if it is uninformative. Our methods also recover the state-of-the-art bounds for the special case of uniformly sampled observations with no side information, which has already found applications in election auditing. The harder weighted case with general side information solves the more challenging problem of AI-assisted financial auditing.