Comment Comment by gpm > The vast, vast, VAST majority of the original datasets were from pirated books and the likeAnd there's been significant legal consequences as a result> Also, arguably a robots.txt is the exact mechanism to follow to do the mass GET-ingYou're free to argue this of course, but the courts have largely rejected it already pre LLMs. See for example hiQ Labs v. LinkedIn parent gpm 3 months ago
And there's been significant legal consequences as a result
> Also, arguably a robots.txt is the exact mechanism to follow to do the mass GET-ing
You're free to argue this of course, but the courts have largely rejected it already pre LLMs. See for example hiQ Labs v. LinkedIn