kkkkk11
Joined 3 years ago
Karma
2
Submissions
2
Member since
Jun 16
Joined 3 years ago
I'd like to ask: how do you scrape news articles and the forums that have discussion on them? Do you use RSS feeds or do you use a custom web scraper? How do you group them together when they are related (i.e., clusters)? Have you considered making it open source or do you prefer to keep the secret "sauce" secret?
I regularly use Techmeme and I've always wondered about these things.