There's no reason you have to run ESPHome on your Home Assistant server.
It's offered as a HA a̵d̵d̵o̵n̵ app for ease of use (makes it a one-click install), but you can also just `pip install esphome` or use the provided Docker image and get the exact same UI, but with everything (including compilation) running on your much beefier laptop.
So your binaries get compiled quickly and you can still do the OTAs directly from your laptop. HA needn't be involved.
It's offered as a HA a̵d̵d̵o̵n̵ app for ease of use (makes it a one-click install), but you can also just `pip install esphome` or use the provided Docker image and get the exact same UI, but with everything (including compilation) running on your much beefier laptop.
So your binaries get compiled quickly and you can still do the OTAs directly from your laptop. HA needn't be involved.