Except the energy costs for these models is such a tiny fraction of crypto. Cambridge researchers estimate annual energy consumption of just bitcoin at 119 terawatt hours (TWh) [1] or 60 megatons of CO2 [2], just below all of Portugal.
Meanwhile, even a very critical paper on this from some Scandinavian researchers [3] says GPT-3 cost 190,000 kWh (0.00019 TWh) to train. ChatGPT/GPT-3.5 is allegedly an order of magnitude larger in terms of data and cost to train, so let's say it is 0.0019 TWh.
When The Register reported on that paper, you can see how they tried as hard as they could to make it sound big: the cost to train GPT-3 was the same as driving a car to the moon and back (435k miles). They could have said it cost the same amount of carbon as 25 US drivers emit each year. In the grand scheme of things, that's nothing. That's one long-haul flight per trained model. And you just need to train them once. Querying the models cost far less.
And the electricity generated for US-based server farms is way cleaner than cars, planes, or the coal mines powering Chinese bitcoin mines.
[1] https://ccaf.io/cbeci/index
[2] https://ccaf.io/cbeci/ghg/comparisons
[3] https://arxiv.org/pdf/2007.03051.pdf
[4] https://www.theregister.com/2020/11/04/gpt3_carbon_footprint...
Meanwhile, even a very critical paper on this from some Scandinavian researchers [3] says GPT-3 cost 190,000 kWh (0.00019 TWh) to train. ChatGPT/GPT-3.5 is allegedly an order of magnitude larger in terms of data and cost to train, so let's say it is 0.0019 TWh.
When The Register reported on that paper, you can see how they tried as hard as they could to make it sound big: the cost to train GPT-3 was the same as driving a car to the moon and back (435k miles). They could have said it cost the same amount of carbon as 25 US drivers emit each year. In the grand scheme of things, that's nothing. That's one long-haul flight per trained model. And you just need to train them once. Querying the models cost far less.
And the electricity generated for US-based server farms is way cleaner than cars, planes, or the coal mines powering Chinese bitcoin mines.
[1] https://ccaf.io/cbeci/index
[2] https://ccaf.io/cbeci/ghg/comparisons
[3] https://arxiv.org/pdf/2007.03051.pdf
[4] https://www.theregister.com/2020/11/04/gpt3_carbon_footprint...