That's a fair criticism. Sentiment analysis is quite hard to get right on social media messages because of diversity, subtlety, and many other aspects. From my experience with similar commercial and (very!) expensive products, their accuracy is far from perfect too.
Also consider the lack of labeled data for HN and Reddit messages: I had to use Twitter messages to train the classifiers.
This is the reason why I tried to play with BERT to see if I could get a model to generalize well from only Twitter messages. From my experiments, if you activate BERT (which makes the app much slower), you should be able to get 60~70% accuracy.
It's not perfect, but not too bad as well if you are getting averages over a large amount of messages.
Overall it's still a work in progress, I expect to greatly improve the accuracy over the following weeks!
Also consider the lack of labeled data for HN and Reddit messages: I had to use Twitter messages to train the classifiers.
This is the reason why I tried to play with BERT to see if I could get a model to generalize well from only Twitter messages. From my experiments, if you activate BERT (which makes the app much slower), you should be able to get 60~70% accuracy.
It's not perfect, but not too bad as well if you are getting averages over a large amount of messages.
Overall it's still a work in progress, I expect to greatly improve the accuracy over the following weeks!