Comment by jefferson123

Yes. Sounds like you'll have to deal with getting the data from different sources (either by crawling or APIs) hopefully in a distributed manner (storm, for example) and then analyze the inputs using something like spark or hadoop. If you like, let me know if you end up building it. Email in profile.