An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling

Shaojie Bai, J. Zico Kolter and Vladlen Koltun

arxiv | code* | reddit

Problematic:

How to choose between CNN and RNN based sequence models

Contributions:

Benchmarking convolutional and recurrent neural networks on a range of standard tasks.

Experiments:

Describe a generic temporal convolutional network (TCN) including practices such as dilations and residual connections and compare it to a baseline RNN architecture in three variants (LSTM, GRU, Vanilla).

Issues & comments:

No attention used with the RNN, although the TCN is boosted with the recent updates

Focusing mainly sequence modelling tasks.

Check this for another comparative study.