Comment by minimaxir

It's worth noting that there's a surprising difference between the full model (1542M hyperparameters) and the released smaller model in the repo (117M): the full model can account for narrative structure much better (https://twitter.com/JanelleCShane/status/1097652984316481537), while the smaller model tends to go on trainwrecks, especially if you give it a bespoke grammatical structure (https://twitter.com/JanelleCShane/status/1097656934180696064)

The arguments OpenAI have given about people using the model to create fake news are IMO not good; bad actors will still create fake news anyways (or with the paper/repo, they might even create more bespoke models trained on a specific type of dataset for even better fake news). With the open model, we would know how to better fight such attacks.