fractalcat

Joined 15 years ago

Karma
773
Submissions
59
Member since
Feb 15
[email protected] http://tesser.org

[ my public key: https://keybase.io/fractalcat; my proof: https://keybase.io/fractalcat/sigs/beXR3OnjFCyPZfdMHt2Snvk-XP2ov1Og4npyqFUpOQo ]

View on Hacker News

Recent submissions

Comment · 10 years ago
Comment · 10 years ago
Last time I was in a role which involved on-call rotation:

> expected duties (only answer on-call, do other work, etc)

Only answer pages. My employer did shifts a bit differently from most companies - only six hours per shift, no fixed schedule (decided a week in advance) and only outside of work hours (pages during work hours were handled by whichever sysadmins were on duty), which worked quite well to avoid burning out sysadmins. On-call shifts were paid, and shortages of volunteers were rare.

I'd expect to spend maybe fifteen minutes per shift fixing things, on average (this is in managed hosting, so a page could be any of our customers' services).

> how deep does your on-call dive into code to fix, vs triaging, patching, and creating follow up work to fix permanently?

In my case (sysadmin for a managed hosting company) the code involved was often not under our control; the standard practice was to escalate to the customer if the cause of the outage was a bug in the application. The usual process when suspecting a bug was to track it down if possible (the codebases were usually unfamiliar, so this wasn't always the case), work around it as best we could (e.g., temporarily disable a buggy search indexer which was leaking memory, et cetera), and then get in touch with the customer (by email if the workaround was expected to last until work hours, by phone if not). Occasionally I'd fix the bug in-place and send the customer the patch, but this was technically outside scope.

> priority order of work (on-call tickts vs regular work vs existing on-call tickets vs special queue of simple tasks)

The only priorities were resolving the pages at hand and arranging followup where needed (usually raising a ticket to be followed up during work hours).

> what happens when the current on-call can't complete in time?

Generally the on-call sysadmin would resolve whichever pages they had acknowledged; in the event of an extended outage the acking sysadmin was expected to brief and hand over to the person on the next shift.

> how do you manage for other teams' risk? (ie their api goes down, you can't satisfy your customers)

In practice, we could escalate to anyone in the company for a serious outage we were unable to handle ourselves. This was pretty rare, as a small ops-heavy company, but everyone had access to everyone else's cell phone number and an outage-inducing bug was usually sufficient cause to wake someone up if it couldn't be worked around.

Comment · 10 years ago
I've run barebones hypervisors in production for years; KVM is head and shoulders above everything else I've tried. libvirt and virsh provide a reasonably nice interface if you don't need web-based tooling, and of course config management makes everything a lot easier to maintain. I haven't used salt, but if it has mature tooling for what you need, KVM would be a no-brainer for me.
Comment · 11 years ago
I'm not sure what the author thinks 'corollary' means here. (Something to do with correlation? Not the definition I'm used to, anyway.)
Comment · 11 years ago
Loading this webpage I downloaded ~12KB. For comparison, I went to the NYT homepage and downloaded ~1.2 MB. For people with shitty downlinks like you and I, the absence of exorbitant quantities of Javascript and images makes a lot of difference - a lot more than time-to-first-byte.
Comment · 11 years ago
Comment · 11 years ago
Comment · 11 years ago
Comment · 11 years ago
I didn't realize this was ever in question - of course you can't trust Tor exit nodes not to snoop on your traffic. You can't trust your ISP or friendly local intelligence agency not to snoop on your traffic either; this is why end-to-end authentication and encryption is a useful thing. (Not meant as a criticism of Chloe's research, it's certainly valuable data).
Comment · 11 years ago
Comment · 12 years ago
What's "enterprise security"? Is it like regular security? Does it involve client-side encryption?

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton