Sean McGregor

Founder, Responsible AI Collaborative

Joined

September 2022

Names

Sean McGregor (Preferred)

Emails

****@seanbmcgregor.com (Confirmed)

****@ul.org (Confirmed)

Personal Links

Homepage

Google Scholar

ORCID

Career & Education History

Founder

Responsible AI Collaborative (incidentdatabase.ai)

2022 – Present

Principal Researcher

UL Research Institutes (ul.org)

2023 – 2025

Research Engineer

Syntiant Corp (syntiant.com)

2017 – 2022

PhD student

Oregon State University (oregonstate.edu)

2010 – 2017

Advisors, Relations & Conflicts

PhD Advisor

Thomas Dietterich

2010 – 2017

Expertise

Benchmarks

2022 – Present

AI Safety

2017 – Present

Edge Computing

2018 – 2022

Reinforcement Learning

2010 – 2017

Publications

Risk Management for Mitigating Benchmark Failure Modes: BenchRisk
Sean McGregor, Vassil Tashev, Armstrong Foundjem, Aishwarya Ramasethu, Sadegh AlMahdi Kazemi Zarkouei, Chris Knotz, Kongtao Chen, Alicia Parrish, Anka Reuel, Heather Frase
- NeurIPS 2025 Datasets and Benchmarks Track poster
- Readers: Everyone
In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI
Shayne Longpre, Kevin Klyman, Ruth E. Appel, Sayash Kapoor, Rishi Bommasani, Michelle Sahar, Sean McGregor, Avijit Ghosh, Borhane Blili-Hamelin, Nathan Butters, Alondra Nelson, Dr. Amit Elazari, Andrew Sellars, Casey John Ellis, Dane Sherrets, Dawn Song, Harley Geiger, Ilona Cohen, Lauren McIlvenny, Madhulika Srikumar et al. (14 additional authors not shown)
- ICML 2025 Workshop TAIG Poster
- Readers: Everyone
Introducing v0.5 of the AI Safety Benchmark from MLCommons
Bertie Vidgen, Sean McGregor, Peter Mattson, Joaquin Vanschoren
- arXiv
- Readers: Everyone
To Err Is AI: A Case Study Informing LLM Flaw Reporting Practices
Sean McGregor, Allyson Ettinger, Nick Judd, Paul Albee, Liwei Jiang, Kavel Rao, William H. Smith, Shayne Longpre, Avijit Ghosh, Christopher Fiorelli, Michelle Hoang, Sven Cattell, Nouha Dziri
- Innovative Applications of Artificial Intelligence Conference
- Readers: Everyone
Malicious and Unintentional Disclosure Risks in Large Language Models for Code Generation
Rafiqul Rabin, Sean McGregor, Nick Judd
- CoRR 2025
- Readers: Everyone
In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI
Shayne Longpre, Kevin Klyman, Ruth E. Appel, Sayash Kapoor, Rishi Bommasani, Michelle Sahar, Sean McGregor, Avijit Ghosh, Borhane Blili-Hamelin, Nathan Butters, Alondra Nelson, Amit Elazari, Andrew Sellars, Casey John Ellis, Dane Sherrets, Dawn Song, Harley Geiger, Ilona Cohen, Lauren McIlvenny, Madhulika Srikumar et al. (14 additional authors not shown)
- CoRR 2025
- Readers: Everyone
AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons
Shaona Ghosh, Heather Frase, Adina Williams, Sarah Luger, Paul Röttger, Fazl Barez, Sean McGregor, Kenneth Fricklas, Mala Kumar, Quentin Feuillade--Montixi, Kurt Bollacker, Felix Friedrich, Ryan Tsang, Bertie Vidgen, Alicia Parrish, Chris Knotz, Eleonora Presani, Jonathan Bennion, Marisa Ferrara Boston, Mike Kuniavsky et al. (81 additional authors not shown)
- CoRR 2025
- Readers: Everyone
Position: In-House Evaluation Is Not Enough. Towards Robust Third-Party Evaluation and Flaw Disclosure for General-Purpose AI
Shayne Longpre, Kevin Klyman, Ruth Elisabeth Appel, Sayash Kapoor, Rishi Bommasani, Michelle Sahar, Sean McGregor, Avijit Ghosh, Borhane Blili-Hamelin, Nathan Butters, Alondra Nelson, Dr. Amit Elazari, Andrew Sellars, Casey John Ellis, Dane Sherrets, Dawn Song, Harley Geiger, Ilona Cohen, Lauren McIlvenny, Madhulika Srikumar et al. (14 additional authors not shown)
- ICML 2025 Position Paper Track spotlightposter
- Readers: Everyone
To Err Is AI: A Case Study Informing LLM Flaw Reporting Practices
Sean McGregor, Allyson Ettinger, Nick Judd, Paul Albee, Liwei Jiang, Kavel Rao, William H. Smith, Shayne Longpre, Avijit Ghosh, Christopher Fiorelli, Michelle Hoang, Sven Cattell, Nouha Dziri
- AAAI 2025
- Readers: Everyone
Lessons for Editors of AI Incidents from the AI Incident Database
Kevin Paeth, Daniel Atherton, Nikiforos Pittaras, Heather Frase, Sean McGregor
- AAAI 2025
- Readers: Everyone

View all 37 publications

Co-Authors

View all 278 co-authors