Comment by khandelwal

I've used NLTK (http://www.nltk.org) to do something similar. And there's also http://scikit-learn.org/stable/. Both libraries are for Python.

As far as my experience with NLTK and machine learning in general goes: you'll need to spend a fair bit of time figuring out which features to extract.