Comment by georgehill

I am not sure this article will answer your question, but Karpathy has an article about RNNs.

https://karpathy.github.io/2015/05/21/rnn-effectiveness