The Cross-Context Threshold Test: Detecting Discrimination Under Environmental Shifts

Jun Yuan, Xinyue Ye

Abstract: We study how threshold tests for detecting discrimination under environmental shifts, focusing on the Veil-of-Darkness (VoD) setting where visibility changes between daylight and darkness. We show that standard threshold tests, when applied separately to daylight and darkness data, violate key assumptions: risk distributions drift across contexts and thresholds fluctuate arbitrarily. We propose a cross-context threshold test that enforces distributional invariance and monotonic threshold decay. Using New York City stop-and-frisk data and synthetic experiments, we demonstrate that this model yields more reliable thresholds, improves bias detection, and aligns with the counterfactual logic of the VoD test. Our framework generalizes to fairness auditing whenever environmental context influences decisions.

Code Dataset Promise: Yes

Code Dataset Url: https://github.com/junyuan-ai/cctt-threshold-test

Format Confirmation: I agree that I have read and followed the formatting instructions for the camera ready version.

Submission Number: 1894