Model%ResolvedOrgDateLogsTrajsSite
Isoform55.00https://avatars.githubusercontent.com/u/4956703?s=200&v=42025-01-14✓✓�
BlackboxAIAgent49.00-2024-12-20✓✓�
Gru(2024-12-08)48.67
https://pbs.twimg.com/profile_images/1832709320896917504/FB5KojGT_400x400.png2024-12-08--
�
GlobantCodeFixerAgent48.33https://ai.globant.com/wp-content/uploads/2024/06/favicon.png
2024-11-27✓✓�
devlo47.33https://devlo.ai/static/media/devlo.ef99a045eba5ea0bcada.png2024-11-22✓✓�
Kodu-v1+Claude-3.5Sonnet(20241022)44.67
https://raw.githubusercontent.com/kodu-ai/claude-coder/main/extension/assets/kodu.png2024-12-07
✓✓�
OpenHands+CodeActv2.1(claude-3-5-sonnet-20241022)41.67-2024-10-25✓✓�
PatchKitty-0.9+Claude-3.5Sonnet(20241022)41.33-2024-12-20✓✓-
ComposioSWE-Kit(2024-10-30)41.00-2024-10-30✓✓�
Agentless-1.5+Claude-3.5Sonnet(20241022)40.67-2024-12-02✓✓�
OpenCSGStarshipAgenticCoder+GPT4(0806)39.67-2025-01-13✓✓�
BytedanceMarsCodeAgent39.33-2024-09-12✓✓�
MoatlessTools+Claude3.5Sonnet(20241022)39.00-2025-01-14✓✓�
MoatlessTools+Claude3.5Sonnet(20241022)38.33-2024-11-17✓✓�
Honeycomb38.33-2024-08-20✓✓�
Patched.CodesPatchwork37.00-2025-01-04✓✓�
AppMapNaviev236.00-2024-11-13✓✓�
CodeFuse-AAIS35.67-2025-01-04✓✓�
Gru(2024-08-11)35.67-2024-08-11✓✓�
Isoform35.00-2024-08-29✓✓�
SuperCoder2.034.00-2024-08-06✓✓�
BytedanceMarsCodeAgent+GPT4o(2024-05-13)34.00-2024-07-23✓-�
AlibabaLingmaAgent33.00-2024-06-22✓✓�
Agentless-1.5+GPT4o(2024-05-13)32.00-2024-10-28--�
CodeShellTester+GPT4o(2024-05-13)31.33-2024-11-11✓✓�
AutoCodeRover(v20240620)+GPT4o(2024-05-13)30.67-2024-06-21✓✓�
AIGCodeInfant-Coder(2024-08-30)30.00-2024-09-08✓✓�
KortixAI(claude-3-5-sonnet-20241022)30.00-2024-12-03✓✓�
AmazonQDeveloperAgent(v20240719-dev)29.67-2024-07-21✓✓�
Agentless+RepoGraph+GPT-4o29.67-2024-08-08✓✓�
CodeR+GPT4(1106)28.33-2024-06-04✓✓�
reproducedRG28.00-2024-11-17✓✓-
SIMA+GPT4o(2024-05-13)27.67-2024-07-06✓✓�
MASAI+GPT4o(2024-05-13)27.33-2024-06-12✓✓�
Agentless+GPT4o(2024-05-13)27.33-2024-06-30✓-�
MoatlessTools+Claude3.5Sonnet26.67-2024-06-23✓✓�
OpenHands+CodeActv1.826.67-2024-07-25✓✓�
IBMResearchAgent-10126.67-2024-06-12✓-�
Aider+GPT4o&Claude3Opus26.33-2024-05-23✓-�
HyperAgent25.33-2024-09-25✓✓�
MoatlessTools+GPT4o(2024-05-13)24.67-2024-06-17✓✓�
IBMAIAgentSWE-1.0(withopenLLMs)23.67-2024-10-16✓✓�
SWE-Fixer(Qwen2.5-7bretriever+Qwen2.5-72beditor)2024112823.33-2024-11-28✓✓�SWE-agent+Claude3.5Sonnet23.00-2024-06-20✓✓�
AppMapNavie+GPT4o(2024-05-13)21.67-2024-06-15✓-�
BytedanceAutoSE(basedonSWE-Agent)+GPT4/GPT4oMixed(20240828)21.67-2024-08-28✓✓-
AmazonQDeveloperAgent(v20240430-dev)20.33-2024-05-09✓-�
AutoCodeRover(v20240408)+GPT4(0125)19.00-2024-05-30✓-�
SWE-agent+GPT4o(2024-05-13)18.33-2024-07-28✓✓�
SWE-agent+GPT4(1106)18.00-2024-04-02✓✓�
SWE-agent+Claude3Opus11.67-2024-04-02✓✓�
RAG+Claude3Opus4.33-2024-04-02✓-�
RAG+Claude23.00-2023-10-10✓✓-
RAG+GPT4(1106)2.67-2024-04-02✓--
RAG+SWE-Llama7B1.33-2023-10-10✓--
RAG+SWE-Llama13B1.00-2023-10-10✓--
RAG+ChatGPT3.50.33-2023-10-10✓--
MoatlessTools+DeepseekV30.00-2025-01-11✓✓�