Question Answering On Storycloze

评估指标

Accuracy

评测结果

各个模型在此基准测试上的表现结果

模型名称	Accuracy	Paper Title	Repository
Switch Transformer 9B	73.3	Efficient Language Modeling with Sparse all-MLP	-
BLOOMZ	96.3	Crosslingual Generalization through Multitask Finetuning
Gshard 9B	67.9	Efficient Language Modeling with Sparse all-MLP	-
SparseGPT (175B, 2:4 Sparsity)	76.19	SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot
T0-3B (CoT fine-tuned)	94.5	The CoT Collection: Improving Zero-shot and Few-shot Learning of Language Models via Chain-of-Thought Fine-Tuning	-
FLAN 137B (few-shot, k=10)	94.7	Finetuned Language Models Are Zero-Shot Learners
SparseGPT (175B, 50% Sparsity)	78.87	SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot
GPT-3 Large 760M (zero-shot)	72.4	Language Models are Few-Shot Learners
FLAN 137B (zero-shot)	93.4	Finetuned Language Models Are Zero-Shot Learners
SparseGPT (175B, 4:8 Sparsity)	77.02	SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot
Memory chains and semantic supervision	78.7	-	-
Finetuned Transformer LM	86.5	Improving Language Understanding by Generative Pre-Training
KiC-770M	94.40	Knowledge-in-Context: Towards Knowledgeable Semi-Parametric Language Models	-
val-LS-skip	76.5	A Simple and Effective Approach to the Story Cloze Test	-
sMLP – deterministic 9.4B (0-shot)	74.7	Efficient Language Modeling with Sparse all-MLP	-
Reading Strategies Model	88.3	Improving Machine Reading Comprehension with General Reading Strategies
Hidden Coherence Model	77.6	Story Comprehension for Predicting What Happens Next	-
OPT-175B	79.82	SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot
HASH Layers 10B (0-shot)	64.7	Efficient Language Modeling with Sparse all-MLP	-
Base Layers 10B (0-shot)	61.4	Efficient Language Modeling with Sparse all-MLP	-

0 of 23 row(s) selected.