Visual Question Answering On Iconqa

Metrics

Reasoning (Alg.)

Reasoning (Com.)

Reasoning (Cou.)

Reasoning (Est.)

Reasoning (Fra.)

Reasoning (Geo.)

Reasoning (Mea.)

Reasoning (Pat.)

Reasoning (Pro.)

Reasoning (Sce.)

Reasoning (Sen.)

Reasoning (Spa.)

Reasoning (Tim.)

Sub-tasks (Blank)

Sub-tasks (Img.)

Sub-tasks (Txt.)

Results

Performance results of various models on this benchmark

																	Paper Title
Patch-TRM	56.73	87.00	77.81	98.24	82.13	81.87	97.98	68.75	95.73	62.39	92.49	55.62	77.98	83.62	82.66	75.19	IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
ViT	51.10	82.12	70.84	98.95	77.41	82.60	98.76	58.46	86.07	68.80	84.72	54.64	68.66	78.92	79.15	72.34	IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
ViLBERT	50.62	75.60	71.05	99.22	74.09	80.05	99.07	62.78	70.94	58.52	81.78	49.46	66.72	77.08	76.66	70.47	IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
ViLT	50.55	84.95	71.13	99.02	75.81	82.61	98.91	59.22	87.65	66.72	86.10	53.38	69.99	79.27	79.67	72.69	IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
DFAF	50.27	81.69	70.68	99.02	77.60	81.80	98.83	56.60	85.70	67.01	84.11	51.42	67.72	78.28	77.72	72.17	IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
Top-Down	50.00	80.65	65.01	99.54	72.43	80.07	99.46	55.01	83.75	58.22	84.54	45.78	68.28	73.03	75.92	68.51	IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
UNITER	49.18	83.67	71.01	99.41	78.37	81.31	99.38	60.81	87.84	61.25	86.10	48.34	69.77	78.53	78.71	72.39	IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
BAN	47.46	82.12	67.56	97.06	73.77	79.99	96.50	55.67	82.45	66.92	82.12	53.20	66.50	75.54	76.33	70.82	IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
MCAN	47.32	82.73	68.94	99.08	76.20	79.86	98.99	54.79	84.87	62.49	83.25	49.70	68.00	74.52	77.36	71.25	IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
I-Only	31.73	45.26	37.64	62.29	32.48	38.71	64.02	36.29	37.51	35.47	45.25	37.52	47.37	46.65	41.56	36.02	IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
Q-Only	28.02	48.19	33.63	40.46	33.06	38.03	38.07	33.66	40.76	35.37	45.25	37.14	48.09	28.45	41.64	36.86	IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning
Random	11.12	41.20	18.38	3.62	34.84	30.30	0.36	34.81	38.81	34.25	45.16	36.49	35.82	0.29	41.70	36.87	IconQA: A New Benchmark for Abstract Diagram Understanding and Visual Language Reasoning

0 of 12 row(s) selected.

Command Palette

Visual Question Answering On Iconqa

Metrics

Results