Question Answering On Pubmedqa

评估指标

Accuracy

评测结果

各个模型在此基准测试上的表现结果

模型名称	Accuracy	Paper Title	Repository
MediSwift-XL	76.8	MediSwift: Efficient Sparse Pre-trained Biomedical Language Models	-
PaLM (8B, Few-shot)	34	Large Language Models Encode Clinical Knowledge	-
BioGPT(345M)	78.2	BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining
PaLM (62B, Few-shot)	57.8	Large Language Models Encode Clinical Knowledge	-
PubMedBERT uncased	55.84	Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing
Claude 3 Opus (5-shot)	75.8	The Claude 3 Model Family: Opus, Sonnet, Haiku	-
Flan-T5-XXL	76.80	Evaluation of large language model performance on the Biomedical Language Understanding and Reasoning Benchmark	-
GAL 120B (zero-shot)	77.6	Galactica: A Large Language Model for Science
Human Performance (single annotator)	78.0	PubMedQA: A Dataset for Biomedical Research Question Answering
BioELECTRA uncased	64.2	BioELECTRA:Pretrained Biomedical text Encoder using Discriminators
BioLinkBERT (base)	70.2	LinkBERT: Pretraining Language Models with Document Links
BLOOM (zero-shot)	73.6	Galactica: A Large Language Model for Science
Flan-PaLM (540B, Few-shot)	79	Large Language Models Encode Clinical Knowledge	-
Med-PaLM 2 (CoT + SC)	74.0	Towards Expert-Level Medical Question Answering with Large Language Models
Med-PaLM 2 (ER)	75.0	Towards Expert-Level Medical Question Answering with Large Language Models
Flan-PaLM (62B, Few-shot)	77.2	Large Language Models Encode Clinical Knowledge	-
BioMedGPT-10B	76.1	BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicine
PaLM (540B, Few-shot)	55	Large Language Models Encode Clinical Knowledge	-
Med-PaLM 2 (5-shot)	79.2	Towards Expert-Level Medical Question Answering with Large Language Models
BioLinkBERT (large)	72.2	LinkBERT: Pretraining Language Models with Document Links

0 of 29 row(s) selected.