HyperAIHyperAI

Command Palette

Search for a command to run...

5 months ago

SynTQA: Synergistic Table-based Question Answering via Mixture of Text-to-SQL and E2E TQA

Zhang Siyue ; Luu Anh Tuan ; Zhao Chen

SynTQA: Synergistic Table-based Question Answering via Mixture of
  Text-to-SQL and E2E TQA

Abstract

Text-to-SQL parsing and end-to-end question answering (E2E TQA) are two mainapproaches for Table-based Question Answering task. Despite success on multiplebenchmarks, they have yet to be compared and their synergy remains unexplored.In this paper, we identify different strengths and weaknesses throughevaluating state-of-the-art models on benchmark datasets: Text-to-SQLdemonstrates superiority in handling questions involving arithmetic operationsand long tables; E2E TQA excels in addressing ambiguous questions, non-standardtable schema, and complex table contents. To combine both strengths, we proposea Synergistic Table-based Question Answering approach that integrate differentmodels via answer selection, which is agnostic to any model types. Furtherexperiments validate that ensembling models by either feature-based orLLM-based answer selector significantly improves the performance overindividual models.

Code Repositories

siyue-zhang/SynTableQA
Official
Mentioned in GitHub

Benchmarks

BenchmarkMethodologyMetrics
semantic-parsing-on-wikitablequestionsSynTQA (RF)
Accuracy (Dev): /
Accuracy (Test): 71.6
semantic-parsing-on-wikitablequestionsSynTQA (GPT)
Accuracy: 65.2
Accuracy (Test): 74.4
semantic-parsing-on-wikitablequestionsSynTQA (Oracle)
Test Accuracy: 77.5

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp