Recent submissions

Comment · 9 months ago
My mother-in-law fell for a love scam last year. Luckily we caught it early and no money was lost, but I was surprised how difficult it was to convince her it was a scam despite the obvious signs and family and friends all telling her it was. The scammer claimed to be the actor Jack Wagner (popular on Hallmark channel and soap operas). Out of desperation we looked him up on cameo and paid him for a video message telling her it wasn't him texting her, and that finally snapped her out of it.
Comment · 10 months ago
Ah, good to know, I hadn't seen that V2 update. Looks like a min 5m inactivity to auto-pause (i.e., scale to 0), and any connection attempt (valid or not) resumes the DB.
Comment · 10 months ago
This was true for RDS serverless v1 which scaled to 0 but is no longer offered. V2 requires a minimum 0.5 ACU hourly commit ($40+ /mo).
Comment · 1 year ago
Comment · 1 year ago
Yes, we've been testing volume attachments every 5m since start of the year, and have experienced 100-150 attachment failures per volume type in that time frame during multiple events (most recently last week):

https://cloudlooking.glass/dashboard/#aws.ebs.us-east-1--cp-...

Another interesting bit, is last March AWS changed something in the control plane which both triggered a multi-day LSE, and ultimately increased attachment times from 2-3s to 10-20s (also visible in the graphs).

Comment · 1 year ago
Comment · 1 year ago
Comment · 1 year ago
For networking, the site only reports uptime % for zonal, regional, cross-region or cross-cloud tests. It excludes last mile network tests as those fail frequently due to many hops and endpoint unreliability (we use Ripe Atlas and Globalping.io endpoints which are not always reliable even with redundant probes per test).
Comment · 1 year ago
For data and control plane I can determine the issue from the API request/response logs (i.e., network timeout, 5xx, etc.). Network tests are trickier and we don't have a great way to validate failure cause each of those events (i.e., we don't capture a traceroute on failure), other than to evaluate results from multiple endpoint combinations (e.g., AWS us-east-1 to us-west-1 fails while us-east-2 to us-west-1 succeeds).
Comment · 1 year ago
Happy to answer any questions also.
Comment · 1 year ago
$8549 with 1TB storage
Comment · 1 year ago
There is lot of statistical and empirical data on this topic - MTBF estimates from vendors (typically 100k - 1m+ hours), Backblaze and Google drive failure data (~1-2% annual failure rate), IEEE and others. With N+1 redundancy (backup servers/RAID + spare drives) and proper design and change control processes, operational failures should be very rare.

With cloud hardware issues are just the start - yes you MUST "plan for failure", leveraging load balancers, auto scaling, cloudwatch, and dozens of other proprietary dials and knobs. However, you must also consider control plane, quotas, capacity, IAM, spend, and other non-hardware breaking points.

You're autoscaling isn't working - is the AZ out of capacity, did you hit a quota limit, run out of IPv4s, or was an AMI inadvertently removed? Your instance is unable to write to S3 - is the metadata service being flakey (for your IAM role), or is it due to an IAM role / S3 policy change? Your Lambda function is failing - did it hit a timeout, or exhaust the (512MB) temp storage? Need help diagnosing an issue - what is your paid support tier - submit a ticket and we'll get back to you sometime in the 24 hours.

Comment · 1 year ago
> Any usecase that requires interacting with a service is automatically excluded

This isn't correct. Many applications consist of a mix of static and dynamic content. Even dynamic content is often cacheable for a time. All of this can be served by a CDN (using TTLs) which is a much simpler and more cost effective solution than multi-region cloud infra, with the same performance benefits.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton