Responsible AI in the OSS: Reconciling Innovation with Risk Assessment and Disclosure

Mahasweta Chakraborti, Bert Joseph Prestoza, Nicholas Vincent, Seth Frey, Vladimir Filkov

Keywords: Open Source, Responsible AI, AI Safety, AI Evaluation, Model Cards

Abstract: Ethical concerns around AI have led to increased emphasis on model auditing and reporting requirements. We analyze OSS projects to understand if model evaluation is associated with documentation of limitations, biases, and other risks. Our analysis of 7902 Hugging Face projects found that while risk documentation is strongly associated with evaluation practices, high performers from the platform’s largest competitive leaderboard (N=788) were less accountable. Recognizing these delicate tensions in performance incentives may guide providers in revisiting evaluation objectives and legal scholars to formulate platform interventions and policies that balance innovation and responsibility.

Submission Number: 18