HyperAI超神经

首页算力平台文档资讯论文教程数据集百科 SOTA LLM 模型天梯 GPU 天梯顶会

中文

HyperAI超神经

Code Generation On Conala

评估指标

BLEU

Exact Match Accuracy

评测结果

各个模型在此基准测试上的表现结果

			Paper Title	Repository
PanGu-Coder-FT-I	44.32	-	Fine-Tuning Large Language Models for Answering Programming Questions with Code Snippets	-
RoBERTaMarian	35.74	13.8	Leveraging pre-trained language models for code generation	-
MarianCG	34.43	10.2	MarianCG: a code generation transformer model inspired by machine translation	-
TranX + BERT w/mined	34.2	5.8	The impact of lexical and grammatical processing on generating code from natural language
BERT + TAE	33.41	-	Code Generation from Natural Language with Less Prior and More Monolingual Data
BERTMarian	32.46	12.40	Leveraging pre-trained language models for code generation	-
External Knowledge With API + Reranking	32.26	-	Incorporating External Knowledge through Pre-training for Natural Language to Code Generation
External Knowledge With API	30.69	-	Incorporating External Knowledge through Pre-training for Natural Language to Code Generation
BART W/ Mined	30.55	-	Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation
ELECTRAMarian	30.18	10.0	Leveraging pre-trained language models for code generation	-
Reranker	30.11	2.8	Reranking for Neural Semantic Parsing	-
LUKEMarian	29.83	7.6	Leveraging pre-trained language models for code generation	-
BART Base	26.24	-	Reading StackOverflow Encourages Cheating: Adding Question Text Improves Extractive Code Generation
TranX	24.30	-	TRANX: A Transition-based Neural Abstract Syntax Parser for Semantic Parsing and Code Generation

0 of 14 row(s) selected.

Code Generation On Conala | SOTA | HyperAI超神经