Ask HN: How much would it cost to replicate GPT-2's model?

Cost 1: Scrape 8 Million articles from Reddit posts.

http://files.pushshift.io/reddit/

Cost 2: Training a 1.5 B parameter transformer network.

I'm assuming it could be trained with google's TPUs. And the model would just be a bigger version of the model open ai released.