If your prognosis is true I'm not sure it's fully positive. Inference without batching is just massive less efficient, even if local hardware can be more energy efficient per operation and can skip out on cooling.
Open weights, obviously beneficial. Local compute when not necessary, seems like it'd be significantly worse for the environment?
Open weights, obviously beneficial. Local compute when not necessary, seems like it'd be significantly worse for the environment?