The current state of the art is hiQ vs LinkedIn:
https://www.eff.org/cases/hiq-v-linkedin
Basically: if it's publicly visible, you can scrape it.
Caveat: the case is still making its way to the Supreme Court.
Edit: There's also Sandvig v. Sessions, which establishes that scraping publicly available data isn't a computer crime:
https://www.eff.org/deeplinks/2018/04/dc-court-accessing-pub...
Edit2: Two extra common sense caveats:
- Don't hammer the site you're scraping, which is to say don't make it look like you're doing a denial of service attack.
- Don't sell or publish the data wholesale, as is -- that's basically guaranteed to attract copyright infringement lawsuits. Consume it, transform it, use it as training data, etc. instead.
https://www.eff.org/cases/hiq-v-linkedin
Basically: if it's publicly visible, you can scrape it.
Caveat: the case is still making its way to the Supreme Court.
Edit: There's also Sandvig v. Sessions, which establishes that scraping publicly available data isn't a computer crime:
https://www.eff.org/deeplinks/2018/04/dc-court-accessing-pub...
Edit2: Two extra common sense caveats:
- Don't hammer the site you're scraping, which is to say don't make it look like you're doing a denial of service attack.
- Don't sell or publish the data wholesale, as is -- that's basically guaranteed to attract copyright infringement lawsuits. Consume it, transform it, use it as training data, etc. instead.