HAC: The Hacker-Cup AI Competition

Weiwei Yang, Mark Saroufim, Joe Issacson, Luca Antiga, Greg Bowyer, Christian Puhrsch, Driss Guessous, Supriya Rao, Geeta Chauhan, Margaret Li, David Harmeyer, Wesley May

Keywords: Generative AI, Code Generation, AI Agent, AI system integration, Efficiency

TL;DR: Launching the inaugural AI track in the Meta Hacker Cup programming competition and releasing all historic Hacker Cup problems with solutions.

Abstract:

We are launching the first AI track for the popular Meta Hacker Cup programming
competition, designed to assess the capabilities of Generative AI in performing
autonomous code generation tasks. We aim to test the limits of AI in complex
coding challenges and measure the performance gap between AI systems and
human programmers. We will provide access to all Hacker Cup problems since 2011
alongside their respective solutions in a multimodal (image and text) format, and
utilize the existing Hacker Cup infrastructure for competitor evaluation. Featuring
both "open evaluation, open model" and "open evaluation, closed model" tracks,
this competition invites diverse participation from research institutions of varied
interests and resource constraints, including academic labs, AI startups, large
technology companies, and AI enthusiasts. Our goal is to develop and democratize
meaningful advancements in code automation with the very first open evaluation
process for competitive AI programmers.

Competition Timeline:

• Early July 2024: Hacker Cup 2024 schedule to be released, finalize Hacker Cup questions.

• July-Aug 2024: Tutorials and lectures on Discord

• Late September 2024: Practice Round

• Early October 2024: Round 1

• End of October 2024: Round 2, Notify top k-teams

• Early November 2024: Round 3

• Mid-November 2024: Vet top-k solution in open track.

• Early December 2024: Final round, announce winners.

• December 8th - 12th 2024: in-person workshop

• February 29th, 2025: Draft of competition paper

• March 2025: Next iteration of the competition proposal

• May 2025: Submit competition retrospective paper to NeurIPS 2025 D&B trac

Website: https://hackercupai.github.io/

Primary Contact Email: https://discord.gg/wWeN9hTH32

Participant Contact Email: https://discord.gg/wWeN9hTH32

Workshop Format: Hybrid (Vancouver + some online speakers)

Preferred Timezone: PST

Submission Number: 8