Comment by ma2rten

Neural networks often perform worse than bag of words for text classification. They compared to a bag of words baseline, but they didn't tune the regularization parameter, which is important.

They should probably have also tried tf-idf and n-grams.