Managed Kubernetes Cluster (HA) for Side Projects
If you want to orchestrate and scale containers you get in touch with Kubernetes someday. Kubernetes allows you to run complex architectures in a reliable and reproducible manner. For almost all problems you can choose between different solutions in the ecosystem.
- Networking (Pod-to-Pod communication, Policy) => Calico, Canal, Flannel
- Ingress / Engress (Inbound and outbound traffic handling) => Nginx, Traefik, Istio
- Storage (Persistent storage for your containers) => Host, Cloud
- Secret Management (Key rotation, Identity integration) => Vault, Sops
- Monitoring (Logging, tracing) => ELK-Stack, LOKI-Stack, Kiali, Jaeger
- Certificate Management (Let’s encrypt) => Cert-manager
- Databases (Encryption, State Management) => Kubedb, Postgres, Mysql
If you stick with some good defaults you can provide a solid foundation in a few clicks. If you don’t want to deal with these topics you can fall back to a fully-managed Kubernetes cluster like Google KE or Amazon EKS. They are deeply integrated into the cloud provider infrastructure. In the best case, you only need to define your deployment definition everything else is offloaded to the cloud. This is the recommended approach if you deal with business-critical applications. If you want to create a playground for your side-projects you can create your own cluster with considerably fewer resources (€). We will do this in the next chapter.
Cloud Provider selection
It’s essential to pick up a provider that provides Kubernetes support. Kubernetes support means your provider has developed the following extensions:
- Cloud-Controller-Manager: The cloud-controller-manager is an extension to the Kubernetes API which allows external cloud providers to integrate Kubernetes in their infrastructure (e.g hcloud-cloud-controller-manager). This includes support for private-networking, load-balancing.
- CSI-Driver: The Container-Storage-Interface (CSI) is installed in your cluster and makes it possible to provision persistent storage of your cloud-provider (e.g csi-driver) and use it in Kubernetes as volumes.
Without the support of these extensions, the dynamic provisioning of clusters can be very exhaustive. In my projects, I don’t want to deal with such time-consuming tasks.
Cluster Management Tools
There are a lot of great tools to manage Kubernetes Cluster. I’d like to focus on Rancher.
Rancher makes it possible to manage multiple clusters with a lean UI. In order to use Rancher, your cloud provider must provide a rancher driver. There are drivers for Digital-Ocean, Google, AWS, Azure, Hetzner (Unofficial), and more. I use Hetzner because it’s cheap and I live in Germany.
Install Rancher
JM Robles did already a great job to explain the process to install Rancher on Hetzner Cloud. However, you must do some changes. If you create a cluster ensure that you configure your cluster in the following way.
- Select “Canal” as a network provider. Calico doesn’t work.
- Disable Nginx as an ingress-controller because we use the cloud load-balancer.
- Select the In-Tree Cloud Provider from type “external” because the cloud-controller-manager should be activated. This results in unschedulable cluster nodes but don’t worry the next step will make them healthy again.
- Install hcloud-cloud-controller-manager. After some minutes your nodes become ready.
- Install csi-driver. This will create a default storage class in your cluster.
- Ensure that you install the latest versions.
I installed Rancher on a dedicated server in a different region than the worker nodes. A custom subdomain https://rancher.domain.de points to the server. In that way, rancher is failure-tolerant to node and region failures. In the future, you could disable any public traffic to your cluster nodes.
Hetzner load-balancer
There are some annotations to add when creating service from type “load-balancer” on Hetzner Cloud. This should be very similar to other cloud providers.
Other improvements
- Disable the public network of you cluster nodes so that only rancher is publicly accessible.
- Encrypt the traffic between your nodes.
- Backup your rancher cluster configs and etcd snapshots on external storage.
- High-availability setup of Rancher
Closing words
That’s it! You are free to install any software on your cluster. You have native support for the Kubernetes service type “load-balancer”, private networking, and persistent volumes of your cloud-provider. Additionally, you can create multiple clusters of any size and separate them for different projects. In my current project, I pay 35€ for a high-available Kubernetes cluster with:
- 1x dedicated master node.
- 2x worker/master nodes of type CX-21 (3 vCPU, 4GB RAM, 80GB nVME SSD, 20TB Traffic).
- 1x dedicated Rancher server
- Load Balancer
- Private networking
- 7x10GB SSD Storage (Kubernetes Volumes)
- GDPR compliant, DDOS protection, 3 regions in EU
- Bonus: Great user-experience on hetzner-cloud.
