Question Answering On Openbookqa

评估指标

Accuracy

评测结果

各个模型在此基准测试上的表现结果

模型名称	Accuracy	Paper Title	Repository
UnifiedQA 11B	87.2	UnifiedQA: Crossing Format Boundaries With a Single QA System
FLAN-T5-Large 783M	31.2	LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
GenMC 11B	89.8	Clues Before Answers: Generation-Enhanced Multiple-Choice QA
BLOOM 176B (2-shot)	47.2	BloombergGPT: A Large Language Model for Finance	-
GPT-4 + knowledge base	95.9	-	-
PaLM 540B (Standard-Prompting)	84.4	Large Language Models Can Self-Improve	-
PaLM 540B (CoT Prompting)	86.4	Large Language Models Can Self-Improve	-
GrapeQA: PEGA+CANP	90	GrapeQA: GRaph Augmentation and Pruning to Enhance Question-Answering	-
LLaMA-2 7B + MixLoRA	81.6	MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts
LLaMA-3 8B+MoSLoRA	86.8	Mixture-of-Subspaces in Low-Rank Adaptation
BiLSTM max-out question-match (WordNet + science fact)	56.3	Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
MVP-Tuning (ensemble)	95.2	-	-
PaLM 540B (Self Improvement, Self Consistency)	94.4	Large Language Models Can Self-Improve	-
GPT-3 175B (few-shot, k=32)	65.4	Language Models are Few-Shot Learners
GrapeQA: PEGA	82	GrapeQA: GRaph Augmentation and Pruning to Enhance Question-Answering	-
LaMini-F-T5 783M	34	LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
AristoRoBERTa + MVP-Tuning	87.6	-	-
GrapeQA: CANP	66.2	GrapeQA: GRaph Augmentation and Pruning to Enhance Question-Answering	-
LaMini-T5 738M	36	LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
LLaMA-3 8B + MixLoRA	84.8	MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts

0 of 45 row(s) selected.