Language Modelling On Penn Treebank Character

Metrics

Bit per Character (BPC)

Number of params

Results

Performance results of various models on this benchmark

			Paper Title	Repository
Bipartite Flow	1.38	-	Discrete Flows: Invertible Generative Models of Discrete Data
TCN	1.31	5.9M	Seq-U-Net: A One-Dimensional Causal U-Net for Efficient Sequence Modelling
Temporal Convolutional Network	1.31	-	An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling
Seq-U-Net	1.3	5.9M	Seq-U-Net: A One-Dimensional Causal U-Net for Efficient Sequence Modelling
STAR	1.30	-	Gating Revisited: Deep Multi-layer RNNs That Can Be Trained
R-Transformer	1.24	-	R-Transformer: Recurrent Neural Network Enhanced Transformer
2-layer Norm HyperLSTM	1.219	14.4M	HyperNetworks
NAS-RL	1.214	16.3M	Neural Architecture Search with Reinforcement Learning
FS-LSTM-2	1.193	27M	Fast-Slow Recurrent Neural Networks
IndRNN	1.19	-	Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNN
FS-LSTM-4	1.190	27M	Fast-Slow Recurrent Neural Networks
6-layer QRNN	1.187	13.8M	An Analysis of Neural Language Modeling at Multiple Scales
Dense IndRNN	1.18	-	Deep Independently Recurrent Neural Network (IndRNN)
3-layer AWD-LSTM	1.175	13.8M	An Analysis of Neural Language Modeling at Multiple Scales
Past Decode Reg. + AWD-LSTM-MoS + dyn. eval.	1.169	13.8M	Improved Language Modeling by Decoding the Past	-
Feedback Transformer	1.160	10.7M	Addressing Some Limitations of Transformers with Feedback Memory
Trellis Network	1.158	13.4M	Trellis Networks for Sequence Modeling
GAM-RHN-5	1.147	16.0M	Recurrent Highway Networks with Grouped Auxiliary Memory	-
Mogrifier LSTM	1.120	24M	Mogrifier LSTM
Mogrifier LSTM + dynamic eval	1.083	24M	Mogrifier LSTM

0 of 20 row(s) selected.

Command Palette

Language Modelling On Penn Treebank Character

Metrics

Results