Kubernetes

A Kubernetes cluster spread across four countries (Netherlands, Switzerland, Norway, Greece) with my own ASN and BGP anycast routing. The excuse was disaster recovery. The real reason was wanting to understand how the internet works at the routing level. Overview Aspect Details Countries Netherlands, Switzerland, Norway, Greece Architecture Hybrid (on-premises + cloud VPS) Routing BGP anycast with personal ASN IPv6 /48 prefix via RIPE LIR Nodes 13 across 2 operational sites (NL, GR) Key Technical Components BGP & Networking Personal ASN registration via RIPE LIR with /48 IPv6 prefix eBGP peering with two upstream transit providers for anycast redundancy iBGP mesh over IPsec/WireGuard tunnels using Cilium BGP control plane NAT64 edge translation enabling IPv6 ingress to IPv4 core infrastructure Site Connectivity Full mesh encrypted tunnels between on-premises (Cisco ASA) and cloud sites (strongSwan) Geo-distributed edge nodes for latency optimization and DDoS resilience IPsec tunnels with automatic failover Kubernetes Platform Cilium CNI with eBPF dataplane BGP control plane for LoadBalancer services Cross-cluster service mesh via Cilium Cluster Mesh Architecture ┌─────────────────────────────────┐ │ IPv6 Anycast Traffic │ │ (Personal ASN + /48) │ └───────────────┬─────────────────┘ │ ┌───────────────┴───────────────┐ │ Transit Providers │ │ (eBGP - Redundant Path) │ └───────────────┬───────────────┘ │ ┌────────────────┬───────────────┼───────────────┬────────────────┐ │ │ │ │ │ ▼ ▼ │ ▼ ▼ ┌───────────────┐ ┌───────────────┐ │ ┌───────────────┐ ┌───────────────┐ │ Switzerland │ │ Norway │ │ │ Netherlands │ │ Greece │ │ (iFog VPS) │ │(Gigahost VPS) │ │ │ (On-Prem) │ │ (On-Prem) │ │ strongSwan │ │ strongSwan │ │ │ Cisco ASA │ │ Cisco ASA │ │ NAT64+eBGP │ │ NAT64+eBGP │ │ │ 5508-X │ │ 5508-X │ └───────┬───────┘ └───────┬───────┘ │ └───────┬───────┘ └───────┬───────┘ │ │ │ │ │ │ │ │ │ │ └─────────────────┴──────────────┴──────────────┴─────────────────┘ │ ┌────────────────────┴────────────────────┐ │ IPsec Full Mesh (All 4 Sites) │ │ CH ←→ NO ←→ NL ←→ GR ←→ CH ←→ NL... │ └────────────────────┬────────────────────┘ │ ┌──────────────────────────┴──────────────────────────┐ │ │ ▼ ▼ ┌─────────────────────────────────┐ ┌─────────────────────────────────┐ │ Netherlands (NL) │ │ Greece (GR) │ │ nllei01k8s │ │ grskg01k8s │ │ ┌───────────────────────────┐ │ │ ┌───────────────────────────┐ │ │ │ 3x Control Plane (HA) │ │ │ │ 3x Control Plane (HA) │ │ │ │ 4x Worker Nodes │ │ │ │ 4x Worker Nodes │ │ │ │ Cilium CNI + iBGP │ │ │ │ Cilium CNI + iBGP │ │ │ │ Proxmox VE │ │ │ │ Proxmox VE │ │ │ └───────────────────────────┘ │ │ └───────────────────────────┘ │ │ PRIMARY SITE │◄───────►│ DR/HA SITE │ │ 192.168.85.0/24 │ Cluster │ 192.168.58.0/24 │ └─────────────────────────────────┘ Mesh └─────────────────────────────────┘ Technology Stack Networking BGP: Personal ASN with /48 IPv6 prefix Transit: Dual upstream providers for redundancy Tunneling: IPsec (Cisco ASA) + strongSwan (cloud) Edge: NAT64 for IPv6→IPv4 translation Kubernetes Version: v1.34.2 CNI: Cilium with eBPF dataplane Mesh: Cilium Cluster Mesh for cross-site connectivity Ingress: NGINX with BGP-advertised VIPs Infrastructure On-premises: Proxmox VE, Cisco ASA 5508-X Cloud: iFog (Switzerland), Gigahost (Norway) Storage: SeaweedFS with cross-site replication Status Current: Netherlands and Greece operational (primary + DR) Transit: Switzerland (iFog) and Norway (Gigahost) operational as edge/transit nodes ...

Everything in the homelab — network devices, VMs, Kubernetes workloads, Docker services, backups — is managed through GitLab CI/CD, Atlantis, and Argo CD. Nothing gets configured via SSH anymore. What It Covers Platform vs. application split — Atlantis/OpenTofu handles the infrastructure layer (nodes, networking, storage), Argo CD handles application workloads. Both triggered by merge requests, both with auto-sync. Kubernetes — 7-node HA cluster (v1.34.2) with Cilium CNI replacing kube-proxy via eBPF. BGP peering with the Cisco ASA for LoadBalancer services. Dual storage: NFS for shared volumes, Synology iSCSI CSI for block storage. ...

The restlessness I had a perfectly stable single-site Kubernetes cluster in the Netherlands. It worked, it was reliable, and by any reasonable measure, it was enough. But I have learned over the years that when things become too stable, too predictable, I start looking for the next challenge; this pattern has followed me throughout my career, moving on whenever circumstances headed toward a stale that was beyond my control to influence. ...