Code Generation On Pecc
评估指标
Pass@3
评测结果
各个模型在此基准测试上的表现结果
模型名称 | Pass@3 | Paper Title | Repository |
|---|---|---|---|
| Llama-3-8B-Instruct | 3.1 | PECC: Problem Extraction and Coding Challenges | - |
| Claude 3 Haiku | 27.67 | PECC: Problem Extraction and Coding Challenges | - |
| chat-bison | 8.48 | PECC: Problem Extraction and Coding Challenges | - |
| GPT-3.5 Turbo | 23.75 | PECC: Problem Extraction and Coding Challenges | - |
| WizardLM-2-7B | 3.72 | PECC: Problem Extraction and Coding Challenges | - |
| Mixtral-8x7B-Instruct | 8.35 | PECC: Problem Extraction and Coding Challenges | - |
| codechat-bison | 11.39 | PECC: Problem Extraction and Coding Challenges | - |
| Phi-3-mini-128k-instruct | 7.18 | PECC: Problem Extraction and Coding Challenges | - |
0 of 8 row(s) selected.