sksjvsla
Joined 1 year ago
Joined 1 year ago
On the topic of Hetzner and account risks, I completely agree: this is not just a technical issue, and that's why we built a multi-cloud setup spanning Hetzner and OVH in Europe. The architecture was designed from the start to absorb a full platform-level outage or even a unilateral account closure. Recovery and failover have been tested specifically with these scenarios in mind — it's not a "we'll get to it later" plan, it's baked into the ops process now.
I’ve also engaged Hetzner directly about the reported shutdown incidents — here’s one of the public discussions where I raised this: https://www.reddit.com/r/hetzner/comments/1lgs2ds/comment/mz...
What I got in a private follow-up from Hetzner support helped clarify a lot about those cases. Without disclosing anything sensitive, I’ll just say the response gave me more confidence that they are aware of these issues and are actively working to handle abuse complaints more responsibly. Of course, it doesn't mean the risk is zero — no provider offers that — but it did reduce my level of concern.
Regarding Cloudflare, I actually agree with your point: vendor contract structure and incentives matter. But that’s also why I find the AWS argument interesting. While it’s true that AWS is incentivized to keep accounts alive to keep billing, they also operate at a scale where mistakes (and opaque actions) still happen — especially with automated abuse handling. Cloudflare, for its part, has consistently been one of the most resilient providers in terms of DNS, global routing, and mitigation — at least in my experience and according to recent data. Neither platform is perfect, and both require backup plans when they become uncooperative or misaligned with your needs.
The broader point you make — about counterparty risk, legal ambiguity, and the illusions of control in tech stacks — is one I think deserves more attention in technical circles. You're absolutely right that these risks aren't visible in logs or Grafana dashboards, and can't always be solved by code. It's exactly why we're investing in process-level failovers, not just infrastructure ones.
Again, thank you for sharing your insights here. I don’t think we’re on opposite sides — I think we’re simply looking at the same risks through slightly different lenses of experience and mitigation.
Being able to stay compliant and protect revenue is worth far more than quibbling over which cloud costs a little less or much a monthly salary for an employee is in various countries.
The real ratio to look at is cloud spend vs. the revenue.
For me, switching from AWS to European providers wasn’t just about saving on cloud bills (though that was a nice bonus). It was about reducing risk and enabling revenue. Relying on U.S. hyperscalers in Europe is becoming too risky — what happens if Safe Harbor doesn’t get renewed? Or if Schrems III (or whatever comes next) finally forces regulators to act?
If you want to win big enterprise and governmental deals, Then you got to do whatever it takes and being compliant and in charge is a huge part of that.
The ISO 27001 has me audited for just that (disaster recovery and monitoring) so that settles it, no?
Also worth noting that these are the two things you don't really get from the hyperscalers. If you want to count on more than their uptime guarantees, you have to roll some DR yourself and while you might think that this is easy, it is not easier than doing it with Terraform and Ansible on other clouds.
I have had my DR and monitoring audited in its AWS and EU version. One was no easier or harder than the other.
But the EU setup gave me a clear answer to clients on CLOUD act, Shrems II, GDPR, Safe Harbor, which is a competitive advantage.
The defining conditions is my current setup and business requirement. It works well and we've resisted pretending that we know where we will be in 5 years.
I am reminded of the 2023 story of the surprisingly simple infra of Stack Overflow[1] and the 2025 story of that Stack overflow is almost dead[2]
Given that the setup works now, one can't add that it is only working "for now". I see no client demand in the foreseeable future leading me to think that this has been fundamentally architected incorrectly.
[1] https://x.com/sahnlam/status/1629713954225405952
[2] https://blog.pragmaticengineer.com/stack-overflow-is-almost-...
A thing we learned in this process is that there's many levels of abstraction which you can think of rollback and locking down SSH and so on and so forth.
If your abstraction level is AWS and the big hyperscalers, it would be to use Kubernetes, but peeling layers of complexity off that, you could also do it with Docker Compose or even Linux programs that are really battle tested for decades.
Most ISO certified companies are not at hyperscale so here is a fun one: Instead of Grafana Agent from 2020, you could most likely get away better with rsyslog from 2004.
And if you want your EKS cluster to give you insights you have configure CloudWatch yourself so does what hands-off is there comparing that setup to Ubuntu+Grafana Agent?
Being able to stay compliant and protect revenue is worth far more than quibbling over which cloud costs a little less.
The real ratio to look at is cloud spend vs. the revenue.
For me, switching from AWS to European providers wasn’t just about saving on cloud bills (though that was a nice bonus). It was about reducing risk and enabling revenue. Relying on U.S. hyperscalers in Europe is becoming too risky — what happens if Safe Harbor doesn’t get renewed? Or if Schrems III (or whatever comes next) finally forces regulators to act?
When you rely heavily on U.S. hyperscalers in Europe, you’re exposed to potential disruptions — what if data transfer agreements break down or new rulings force major changes? The value of cloud spend, in my view, isn’t just in engineering convenience, but in how it helps sustain the business and unlock growth. That’s why I prioritized compliance and risk reduction — even if that means stepping a little outside the comfort of the big providers’ managed services.
This could be a stack that could be parametrised with sound defaults just requiring some terraform provider credentials as well as a path to an executable web app and a choice of database engine.
ISO readiness built-in and abstracted at the OS level rather than programming language level.
If anyone wants to "assetize" what I built, reach out at [email protected]. I bring a battle-tested setup that has been ISO certified by independent auditors.
You bring clients directly or indirectly with marketing/growth hacking mindset.
Also, the idea that using VPS or non-hyperscaler clouds means “poorly integrated and insecure mess” feels like AWS marketing talking. Good ops doesn’t mean gatekeepers — it means understanding your system so you don’t need to swap out components every quarter because the last choice didn’t scale as promised.
I’d rather spend time building something stable that aligns with my compliance and revenue goals, than chasing the latest AWS feature set. And by the way, someone still has to keep all that AWS software up to date — you’ve just outsourced it and locked yourself into their way of doing it.
For me, switching from AWS to European providers wasn’t just about saving on cloud bills (though that was a nice bonus). It was about reducing risk and enabling revenue. Relying on U.S. hyperscalers in Europe is becoming too risky — what happens if Safe Harbor doesn’t get renewed? Or if Schrems III (or whatever comes next) finally forces regulators to act?
Being able to stay compliant and protect revenue is worth far more than quibbling over which cloud costs a little less.
1. Dumping a backup every so often?
2. Exporting its performance via Prometheus, and displaying in a dashboard?
3. Machine disk usage via Prometheus?
4. An Ansible playbook for recovery? Maybe kicking that into effect with an alert triggered from bullet 2 and 3.
5. Restoring the database that you backed up into your staging env, so you get a recurring, frequent check of its integrity.
This would be around 100 to 500 lines of code of which an LLM can do for you.
What am I missing?
Our logging server will switch primary DB in case the original primary DB server is down. Since we are counting on downtime, the monitoring server is by default not hosted in the same places as the primary DB but in the same place as the secondary DB.
We assume that each clod will go down but not at the same time.
https://datapult.dk/
https://knobel-dk.github.io/landingpage-iso-playbooks/