Question Answering On Squad11 Dev

Metrics

Results

Performance results of various models on this benchmark

Model Name	EM	F1	Paper Title	Repository
RASOR	66.4	74.9	Learning Recurrent Span Representations for Extractive Question Answering	-
FG fine-grained gate	59.95	71.25	Words or Characters? Fine-grained Gating for Reading Comprehension	-
R.M-Reader (single)	78.9	86.3	Reinforced Mnemonic Reader for Machine Reading Comprehension	-
Match-LSTM with Bi-Ans-Ptr (Boundary+Search+b)	64.1	64.7	Machine Comprehension Using Match-LSTM and Answer Pointer	-
DCN (Char + CoVe)	71.3	79.9	Learned in Translation: Contextualized Word Vectors	-
MPCM	66.1	75.8	Multi-Perspective Context Matching for Machine Comprehension	-
KAR	76.7	84.9	Explicit Utilization of General Knowledge in Machine Reading Comprehension	-
DistilBERT-uncased-PruneOFA (90% unstruct sparse, QAT Int8)	75.62	83.87	Prune Once for All: Sparse Pre-Trained Language Models	-
BART Base (with text infilling)	-	90.8	BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension	-
DensePhrases	78.3	86.3	Learning Dense Representations of Phrases at Scale	-
BERT-Large-uncased-PruneOFA (90% unstruct sparse, QAT Int8)	83.22	90.02	Prune Once for All: Sparse Pre-Trained Language Models	-
FABIR	65.1	75.6	A Fully Attention-Based Information Retriever	-
BERT-Base-uncased-PruneOFA (85% unstruct sparse)	81.1	88.42	Prune Once for All: Sparse Pre-Trained Language Models	-
T5-3B	88.53	94.95	Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer	-
FusionNet	75.3	83.6	FusionNet: Fusing via Fully-Aware Attention with Application to Machine Comprehension	-
TinyBERT-6 67M	79.7	87.5	TinyBERT: Distilling BERT for Natural Language Understanding	-
Ruminating Reader	70.6	79.5	Ruminating Reader: Reasoning with Gated Multi-Hop Attention	-
BiDAF + Self Attention + ELMo	-	85.6	Deep contextualized word representations	-
SAN (single)	76.235	84.056	Stochastic Answer Networks for Machine Reading Comprehension	-
DCN	65.4	75.6	Dynamic Coattention Networks For Question Answering	-

0 of 55 row(s) selected.