FAIR Universe HiggsML Uncertainty Dataset and Competition

Wahid Bhimji, Ragansu Chakkappai, Po-Wen Chang, Yuan-Tang Chou, Sascha Diefenbacher, Jordan Dudley, Ibrahim Elsharkawy, Steven Farrell, Aishik Ghosh, Cristina Giordano, Isabelle Guyon, Chris Harris, Yota Hashizume, Shih-Chieh Hsu, Elham E Khoda, Claudius Krause, Ang Li, Benjamin Nachman, David Rousseau, Robert Schöfbeck et al. (5 additional authors not shown)

Keywords: competition, uncertainty quantification, simulation based inference, particle physics, adversarial training, normalising flows

TL;DR: Fair Universe HiggsML was a NeurIPS 2024 competition where participants dealt with a biased dataset and estimated a confidence interval on a parameter of interest in fundamental physics. This paper is about the dataset and the competition findings.

Abstract: The FAIR Universe – HiggsML Uncertainty Challenge focused on measuring the physical properties of elementary particles with imperfect simulators. Participants were required to compute and report confidence intervals for a parameter of interest regarding the Higgs boson while accounting for various systematic (epistemic) uncertainties. The dataset is a tabular dataset of 28 features and 280 million instances. Each instance represents a simulated proton-proton collision as observed at CERN’s Large Hadron Collider in Geneva, Switzerland. The features of these simulations were chosen to capture key characteristics of different types of particles. These include primary attributes, such as the energy and three-dimensional momentum of the particles, as well as derived attributes, which are calculated from the primary ones using domain-specific knowledge. Additionally, a label feature designates each instance’s type of proton-proton collision, distinguishing the Higgs boson events of interest from three background sources. As outlined in this paper, the permanent dataset release allows long-term benchmarking of new techniques. The leading submissions, including Contrastive Normalising Flows and Density Ratios estimation through classification, are described. Our challenge has brought together the physics and machine learning communities to advance our understanding and methodologies in handling systematic uncertainties within AI techniques.

Dataset URL: https://zenodo.org/records/15131565

Code URL: https://github.com/FAIR-Universe/FAIR_Universe_dataset

Primary Area: AL/ML Datasets & Benchmarks for physics (e.g. climate, health, life sciences, physics, social sciences)

Submission Number: 1318