It seems likely that the more efficient our processors become, the larger share of the world's energy we'll devote to them [0]. Not that that's necessarily a bad thing, if we're getting more than proportionally more utility out of the processors, but I worry about that too [1].
[0] https://en.wikipedia.org/wiki/Jevons_paradox
[1] https://en.wikipedia.org/wiki/Wirth%27s_law