The paper mentions "... similar to the ones used in GPT-2’s training data, i.e. documents linked to from Reddit [25], plus Wikipedia and a collection of books". Does anyone know what collection of books they are talking about?
I tried following the chain of references but ended up at a pay-walled source. Is it based on project gutenberg? Also, does Google train their models on the contents of all the books they scanned for Google Books or are they not allowed to because of copyright right issues?
I tried following the chain of references but ended up at a pay-walled source. Is it based on project gutenberg? Also, does Google train their models on the contents of all the books they scanned for Google Books or are they not allowed to because of copyright right issues?