Compared to other microcontrollers: ESP32 is very power hungry. Shiny displays are very power hungry, Wi-Fi is power hungry. So expect to draw about 5 watts/hour continuously while in operation with all bells and whistles.
With this said (I'm also using them for off-grid) you will need to put them to sleep and only use the display when absolutely needed for most scenarios. I've recently started using devices with e-paper display which at least solve that nuisance of the display power draw: https://www.waveshare.com/wiki/ESP32-S3-ePaper-1.54
The last thing to keep in mind is heating. They will warm quite a bit and you should consider a way to either keep them cooled or make them sleep enough to cooldown, otherwise they will reboot or stop working until they are cooled again.
With this said (I'm also using them for off-grid) you will need to put them to sleep and only use the display when absolutely needed for most scenarios. I've recently started using devices with e-paper display which at least solve that nuisance of the display power draw: https://www.waveshare.com/wiki/ESP32-S3-ePaper-1.54
The last thing to keep in mind is heating. They will warm quite a bit and you should consider a way to either keep them cooled or make them sleep enough to cooldown, otherwise they will reboot or stop working until they are cooled again.