Comment by m_sahaf

Caddy doesn't reuse the private key by default. The user has to explicitly configure it in that broken way.