Let's say 1 joule per token, 500B token => 500 GJ => 150 MWh
If we assume 1/3 of a ton of CO2 per MWh (typical in most of the world, including the US), you get 50 tons of CO2 for that experiment.
This is an order of magnitude approximation, but it is enough to conclude that that 18 tons number make sense.
The unknowns are the energy per token but 1 joule per token should get you in the correct order of magnitude. If you want to verify it independently, run a local LLM, measure the power at the plug (in watts), divide it by the number of tokens/second your LLM runs at and you get a rough approximation of the energy use per token in joules. Luna is smallish model, and there is some caching going on, so it may be less, which may explain the 18 tons number GP gave instead of my 50 tons.
The second unknown is the tons of CO2 per MWh. Some datacenters claim they are near zero, because they use renewables or nuclear, but that clean energy they are using is clean energy that is not used to reduce CO2 emissions elsewhere, so I prefer to think in terms of averages.
This is an order of magnitude approximation, but it is enough to conclude that that 18 tons number make sense.
The unknowns are the energy per token but 1 joule per token should get you in the correct order of magnitude. If you want to verify it independently, run a local LLM, measure the power at the plug (in watts), divide it by the number of tokens/second your LLM runs at and you get a rough approximation of the energy use per token in joules. Luna is smallish model, and there is some caching going on, so it may be less, which may explain the 18 tons number GP gave instead of my 50 tons.
The second unknown is the tons of CO2 per MWh. Some datacenters claim they are near zero, because they use renewables or nuclear, but that clean energy they are using is clean energy that is not used to reduce CO2 emissions elsewhere, so I prefer to think in terms of averages.