Which Wilcoxon should we use? An interactive rank test and other alternatives

Boyan Duan, Aaditya Ramdas, Larry Wasserman

Abstract:

Classical nonparametric tests to compare multiple samples, such as the Wilcoxon test, are often
based on the ranks of observations. We design an interactive rank test called i-Wilcoxon—an analyst is
allowed to adaptively guide the algorithm using observed outcomes, covariates, working models and prior
knowledge—that guarantees type-I error control using martingales. Numerical experiments demonstrate
the advantage of (an automated version of) our algorithm under heterogeneous treatment effects. The
i-Wilcoxon test is first proposed for two-sample comparison with unpaired data, and then extended to
paired data, multi-sample comparison, and sequential settings, thus also extending the Kruskal-Wallis
and Friedman tests. As alternatives, we numerically investigate (non-interactive) covariance-adjusted
variants of the Wilcoxon test, and provide practical recommendations based on the anticipated population
properties of the treatment effects.