I'm curious how you got the energy consumption for 8 hours of use.
I would imagine there's a huge spread there too. Depending on hardware, size of the model, requests being batched and queued rather than occupying a whole GPU or TPU, or if you are running an agent loop for 8 hours straight or spend most of the time reviewing results.
I would imagine there's a huge spread there too. Depending on hardware, size of the model, requests being batched and queued rather than occupying a whole GPU or TPU, or if you are running an agent loop for 8 hours straight or spend most of the time reviewing results.