Code Generation On Codecontests
Metrics
Test Set pass@1
Test Set pass@5
Val Set pass@1
Results
Performance results of various models on this benchmark
Model Name | Test Set pass@1 | Test Set pass@5 | Val Set pass@1 | Paper Title | Repository |
|---|---|---|---|---|---|
| MapCoder (GPT-4) | 28.5 | 35.2 | 28.5 | MapCoder: Multi-Agent Code Generation for Competitive Problem Solving | |
| LPW (GPT-4o) | 34.7 | - | - | Planning-Driven Programming: A Large Language Model Programming Workflow | |
| WizardCoder-15B | 1.11 | 3.18 | 1.98 | WizardCoder: Empowering Code Large Language Models with Evol-Instruct | |
| MoTCoder-7B-v1.5 | 20.77 | - | 16.72 | MoTCoder: Elevating Large Language Models with Modular of Thought for Challenging Programming Tasks | |
| CodeSim (GPT4) | 29.1 | - | - | CODESIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debugging | |
| CodeChain + WizardCoder-15B | 2.35 | 3.29 | 2.48 | CodeChain: Towards Modular Code Generation Through Chain of Self-revisions with Representative Sub-modules | - |
| MoTCoder-15B | 26.34 | - | 20.35 | MoTCoder: Elevating Large Language Models with Modular of Thought for Challenging Programming Tasks |
0 of 7 row(s) selected.