Semantic Parsing On Wikitablequestions

Metrics

Accuracy (Dev)

Accuracy (Test)

Results

Performance results of various models on this benchmark

Model Name	Accuracy (Dev)	Accuracy (Test)	Paper Title	Repository
Tab-PoT	/	66.78	Efficient Prompting for LLM-based Generative Internet of Things	-
CABINET	/	69.1	CABINET: Content Relevance based Noise Reduction for Table Question Answering
Chain-of-Table	/	67.31	Chain-of-Table: Evolving Tables in the Reasoning Chain for Table Understanding
ReasTAP-Large	59.7	58.7	ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples	-
TabSQLify (col+row)	-	64.7	TabSQLify: Enhancing Reasoning Capabilities of LLMs Through Table Decomposition	-
SynTQA (RF)	/	71.6	SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA
Binder	65.0	64.6	Binding Language Models in Symbolic Languages
TAPAS-Large (pre-trained on SQA)	/	48.8	TAPAS: Weakly Supervised Table Parsing via Pre-training
T5-3b(UnifiedSKG)	50.65	49.29	UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models
MAPO + TABERTLarge (K = 3)	52.2	51.8	TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data
NormTab (Targeted) + SQL	-	61.20	NormTab: Improving Symbolic Reasoning in LLMs Through Tabular Data Normalization
Structured Attention	43.7	44.5	Learning Semantic Parsers from Denotations with Latent Structured Alignments and Abstract Programs
Dater	64.8	65.9	Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoning
TAPEX-Large	57.0	57.5	TAPEX: Table Pre-training via Learning a Neural SQL Executor
TabLaP	/	76.6	Accurate and Regret-aware Numerical Problem Solver for Tabular Question Answering
SynTQA (GPT)	-	74.4	SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA
SynTQA (Oracle)	-	-	SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA
ARTEMIS-DA	-	80.8	ARTEMIS-DA: An Advanced Reasoning and Transformation Engine for Multi-Step Insight Synthesis in Data Analytics	-
LEVER	64.6	65.8	LEVER: Learning to Verify Language-to-Code Generation with Execution
OmniTab-Large	62.5	63.3	OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering

0 of 21 row(s) selected.