HyperAI超神经
首页资讯最新论文教程数据集百科SOTALLM 模型天梯GPU 天梯顶会
全站搜索
关于
HyperAI超神经

Code Generation On Res Q

评估指标

pass@1

评测结果

各个模型在此基准测试上的表现结果

模型名称
pass@1
Paper TitleRepository
QurrentOS-coder + Gemini 1.5 Pro30.0RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale
QurrentOS-coder + Claude 3.5 Sonnet58.0RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale
QurrentOS-coder + Llama 3 70b20.0RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale
QurrentOS-coder + Qwen-72B-Instruct18.0RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale
QurrentOS-coder + GPT-430.0RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale
QurrentOS-coder + Claude 3 Opus36.0RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale
QurrentOS-coder + GPT-4o46.0RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale
QurrentOS-coder + DeepSeek-Coder-V229.0RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale
QurrentOS-coder + GPT-4 Turbo37.0RES-Q: Evaluating Code-Editing Large Language Model Systems at the Repository Scale
0 of 9 row(s) selected.