> 50 machines at hetzner
- install machines with ansible (using hetzner scripts for OS install)
- machines communicate over vswitch/vlans, external interfaces disabled whenever possible. Pay attention to the custom mtu trick.
- harden machines, unattended-upgrades mandatory on each machine
- ssh open with IP whitelists from iptables on gateways
- machines organized as k8s clusters, took ~1 year to have everything working cleanly
- everything deployed as k8s resources (kustomize, fluxcd, gitops)
- use keepalived for external IPs with floating IPs for ingress on 3 machines per cluster
Machines are managed as cattle, it takes <1h+ hetzner provisioning time to add as many machines as we need.
- install machines with ansible (using hetzner scripts for OS install)
- machines communicate over vswitch/vlans, external interfaces disabled whenever possible. Pay attention to the custom mtu trick.
- harden machines, unattended-upgrades mandatory on each machine
- ssh open with IP whitelists from iptables on gateways
- machines organized as k8s clusters, took ~1 year to have everything working cleanly
- everything deployed as k8s resources (kustomize, fluxcd, gitops)
- use keepalived for external IPs with floating IPs for ingress on 3 machines per cluster
Machines are managed as cattle, it takes <1h+ hetzner provisioning time to add as many machines as we need.