While I am 100% with you that we've arrived at the place local GPUs can do real, good work and that it helps combat the energy arguments against certain LLM usage, Drew is not likely to agree with you. I can't find it on a quick search which tells me I'm probably remembering it from his prior Mastodon account, but I distinctly remember Drew once talking about how gaming GPUs were part of the (climate) problem and that we should all take notes from the Nintendo Switch in terms of gaming joy delivered per watt.
This is not to say I disagree with Drew's desire for efficiency, even in gaming. This is simply to say: from my recollection of that blog/toot thread/whatever it was, even home LLM usage won't meet what I recall his energy bar to be, until we've got planning and orchestrator models that fit on a non-flagship phone.
This is not to say I disagree with Drew's desire for efficiency, even in gaming. This is simply to say: from my recollection of that blog/toot thread/whatever it was, even home LLM usage won't meet what I recall his energy bar to be, until we've got planning and orchestrator models that fit on a non-flagship phone.