Question Answering On Social Iqa

Metrics

Accuracy

Results

Performance results of various models on this benchmark

Model Name	Accuracy	Paper Title	Repository
LLaMA 13B (zero-shot)	50.4	LLaMA: Open and Efficient Foundation Language Models	-
LLaMA 7B (zero-shot)	48.9	LLaMA: Open and Efficient Foundation Language Models	-
UnifiedQA 3B	79.8	UnifiedQA: Crossing Format Boundaries With a Single QA System	-
Chinchilla (zero-shot)	51.3	Training Compute-Optimal Large Language Models	-
LLaMA 65B (zero-shot)	52.3	LLaMA: Open and Efficient Foundation Language Models	-
DeBERTa-Large 304M	80.2	Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering	-
RoBERTa-Large 355M (fine-tuned)	76.7	RoBERTa: A Robustly Optimized BERT Pretraining Approach	-
DeBERTa-Large 304M (classification-based)	79.9	Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering	-
LLaMA-3 8B + MixLoRA	78.8	MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts	-
CompassMTL 567M with Tailor	82.2	Task Compass: Scaling Multi-task Pre-training with Task Prefix	-
LLaMA-3 8B+MoSLoRA (fine-tuned)	81.0	Mixture-of-Subspaces in Low-Rank Adaptation	-
Random chance baseline	33.3	SocialIQA: Commonsense Reasoning about Social Interactions	-
Gopher (zero-shot)	50.6	Scaling Language Models: Methods, Analysis & Insights from Training Gopher	-
BERT-base 110M (fine-tuned)	63.1	SocialIQA: Commonsense Reasoning about Social Interactions	-
ExDeBERTa 567M	79.6	Task Compass: Scaling Multi-task Pre-training with Task Prefix	-
BERT-large 340M (fine-tuned)	64.5	SocialIQA: Commonsense Reasoning about Social Interactions	-
GPT-1 117M (fine-tuned)	63	SocialIQA: Commonsense Reasoning about Social Interactions	-
LLaMA-2 13B + MixLoRA	82.5	MixLoRA: Enhancing Large Language Models Fine-Tuning with LoRA-based Mixture of Experts	-
LLaMA 33B (zero-shot)	50.4	LLaMA: Open and Efficient Foundation Language Models	-
CompassMTL 567M	81.7	Task Compass: Scaling Multi-task Pre-training with Task Prefix	-

0 of 24 row(s) selected.