Hey guys I got this, I speak cloudonaut. Here I'll translate it to sysadmin:
An admin was doing a rolling restart that triggered a bug in the loadbalancer software. The auto restart script turned out to just make things worse by restarting it over and over (they always do), so we thought we'd just quick throw spare capacity at it, but turns out that never works in a panicked rush either. Also, our system designed to handle outage notifications wasn't capacity planned, like, at all.
An admin was doing a rolling restart that triggered a bug in the loadbalancer software. The auto restart script turned out to just make things worse by restarting it over and over (they always do), so we thought we'd just quick throw spare capacity at it, but turns out that never works in a panicked rush either. Also, our system designed to handle outage notifications wasn't capacity planned, like, at all.