My point is that researchers and academics really have been doing this for decades - it's the reason projects like Common Crawl and LAION exist.
I think it's notable that nobody was calling out those researchers for their lack of integrity, because the systems they were building did not seem like a threat to anyone.
OpenAI etc get accused of a lack of integrity on this precisely because the systems they are building work, and are profitable.
My personal opinion here is that integrity is more about what you build with the data. I think saying "scraping means you lack integrity" is a simplification.
I think it's notable that nobody was calling out those researchers for their lack of integrity, because the systems they were building did not seem like a threat to anyone.
OpenAI etc get accused of a lack of integrity on this precisely because the systems they are building work, and are profitable.
My personal opinion here is that integrity is more about what you build with the data. I think saying "scraping means you lack integrity" is a simplification.