Audio Visual Speech Recognition On Lrs3 Ted

Metrics

Word Error Rate (WER)

Results

Performance results of various models on this benchmark

		Paper Title	Code
TM-seq2seq	7.2	Deep Audio-Visual Speech Recognition
EG-seq2seq	6.8	Discriminative Multi-modality Speech Recognition
RNN-T	4.5	Recurrent Neural Network Transducer for Audio-Visual Speech Recognition
Hyb-Conformer	2.3	End-to-end Audio-visual Speech Recognition with Conformers
Zero-AVSR	1.5	Zero-AVSR: Zero-Shot Audio-Visual Speech Recognition with LLMs by Learning Language-Agnostic Speech Representations
AV-HuBERT Large	1.4	Robust Self-Supervised Audio-Visual Speech Recognition
RAVEn Large	1.4	Jointly Learning Visual and Auditory Speech Representations from Raw Data
DistillAV	1.3	Audio-Visual Representation Learning via Knowledge Distillation from Speech Foundation Models
CTC/Attention	0.9	Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels
Llama-AVSR	0.77	Large Language Models are Strong Audio-Visual Speech Recognition Learners
Whisper-Flamingo	0.76	Whisper-Flamingo: Integrating Visual Features into Whisper for Audio-Visual Speech Recognition and Translation
MMS-LLaMA	0.74	MMS-LLaMA: Efficient LLM-based Audio-Visual Speech Recognition with Minimal Multimodal Speech Tokens

0 of 12 row(s) selected.

Audio Visual Speech Recognition On Lrs3 Ted

Metrics

Word Error Rate (WER)

Results

Performance results of various models on this benchmark

		Paper Title	Code
TM-seq2seq	7.2	Deep Audio-Visual Speech Recognition
EG-seq2seq	6.8	Discriminative Multi-modality Speech Recognition
RNN-T	4.5	Recurrent Neural Network Transducer for Audio-Visual Speech Recognition
Hyb-Conformer	2.3	End-to-end Audio-visual Speech Recognition with Conformers
Zero-AVSR	1.5	Zero-AVSR: Zero-Shot Audio-Visual Speech Recognition with LLMs by Learning Language-Agnostic Speech Representations
AV-HuBERT Large	1.4	Robust Self-Supervised Audio-Visual Speech Recognition
RAVEn Large	1.4	Jointly Learning Visual and Auditory Speech Representations from Raw Data
DistillAV	1.3	Audio-Visual Representation Learning via Knowledge Distillation from Speech Foundation Models
CTC/Attention	0.9	Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels
Llama-AVSR	0.77	Large Language Models are Strong Audio-Visual Speech Recognition Learners
Whisper-Flamingo	0.76	Whisper-Flamingo: Integrating Visual Features into Whisper for Audio-Visual Speech Recognition and Translation
MMS-LLaMA	0.74	MMS-LLaMA: Efficient LLM-based Audio-Visual Speech Recognition with Minimal Multimodal Speech Tokens

0 of 12 row(s) selected.

Audio Visual Speech Recognition On Lrs3 Ted | SOTA | HyperAI超神经