Skip to content

Bootstrap MicroK8s cluster on DigitalOcean with Terraform

Notifications You must be signed in to change notification settings

balchua/do-microk8s

Repository files navigation

DigitalOcean Terraform MicroK8s

Verfied using terraform v1.2.6

Does not work when modifying the module after it is created. Adding a new node now works

Warning Reducing nodes still does not leave the cluster

Support for worker only node, which means it will not run control plane components such as the api-server, scheduler and controller manager, available from MicroK8s v1.22

Bootstrap a Highly Available MicroK8s cluster in DigitalOcean with Terraform.

For example to bootstrap a 3 control plane nodes and 2 worker nodes cluster.

module "microk8s" {
  source                       = "../"
  cluster_name                 = "hades"
  node_count                   = "3"
  worker_node_count            = "2"
  os_image                     = "ubuntu-20-04-x64"
  node_size                    = "s-2vcpu-4gb"
  worker_node_size             = "s-4vcpu-8gb"
  node_disksize                = "30"
  region                       = "sgp1"
  dns_zone                     = "geeks.sg"
  microk8s_channel             = "latest/stable"
  cluster_token_ttl_seconds    = 3600
  digitalocean_ssh_fingerprint = var.digitalocean_ssh_fingerprint
  digitalocean_private_key     = var.digitalocean_private_key
  digitalocean_token           = var.digitalocean_token
  digitalocean_pub_key         = var.digitalocean_pub_key
}

Fields Description Default values
source The source of the terraform module none
node_count The number of MicroK8s nodes to create 3
os_image DigitalOcean OS images.
To get the list OS images doctl compute image list-distribution
ubuntu-20-04-x64
node_size DigitalOcean droptlet sizes
To get the list of droplet sizes doctl compute size list
s-4vcpu-8gb
node_disksize Additional volume to add to the droplet. Size in GB 50
region DigitalOcean region
To get the list of regions doctl compute region list
sgp1
dns_zone The DNS zone representing your site. Need to register your domain. geeks.sg
microk8s_channel Specify the MicroK8s channel to use. Refer here stable
cluster_token_ttl_seconds How long the token validity (in seconds) 3600
digitalocean_ssh_fingerprint Your DigitalOcean SSH fingerprint to use, so you can seemlessly ssh into your nodes Refer to TF environment variables
digitalocean_private_key The private key location to use when connecting to your droplets Refer to TF environment variables
digitalocean_token Your DigitalOcean token Refer to TF environment variables
digitalocean_pub_key The public key to use to connect to the droplet Refer to TF environment variables
worker_node_size The worker node size example: s-4vcpu-8gb s-4vcpu-8gb
worker_node_count The number of MicroK8s worker nodes 2
worker_node_disksize Additional volume to add to the droplet. Size in GB 100

DigitalOcean TF environment variables

You must have these environment variables present.

TF_VAR_digitalocean_token=<your DO access token>
TF_VAR_digitalocean_ssh_fingerprint=<Your private key fingerprint>
TF_VAR_digitalocean_private_key=<location of your private key>
TF_VAR_digitalocean_pub_key=<location of your public key>

Creating the cluster

Simply run the terraform plan and then terraform apply

Once terraform completes, you should be able to see the cluster.

Login to one of the node using ssh root@node, then issue the command below.

root@microk8s-node-cetacean-0:~# microk8s kubectl get no
NAME                       STATUS   ROLES    AGE     VERSION
microk8s-node-cetacean-0   Ready    <none>   6m36s   v1.20.4-38+85035ca77e3c6e
microk8s-node-cetacean-2   Ready    <none>   4m33s   v1.20.4-38+85035ca77e3c6e
microk8s-node-cetacean-6   Ready    <none>   4m47s   v1.20.4-38+85035ca77e3c6e
microk8s-node-cetacean-5   Ready    <none>   4m21s   v1.20.4-38+85035ca77e3c6e
microk8s-node-cetacean-4   Ready    <none>   4m15s   v1.20.4-38+85035ca77e3c6e
microk8s-node-cetacean-3   Ready    <none>   4m12s   v1.20.4-38+85035ca77e3c6e
microk8s-node-cetacean-1   Ready    <none>   4m6s    v1.20.4-38+85035ca77e3c6e

Downloading Kube config file

The module automatically downloads the kubeconfig file to your local machine in /tmp/client.config In order to access the Kubernetes cluster from your local machine, simple do export KUBECONFIG=/tmp/client.config

This will connect using the load balancer fronting the api servers. The dns entry will be microk8s-cluster.<domain name>

Example: microk8s-cluster.geeks.sg

MicroK8s High Availability

It requires node counts to be greater than or equal to 3 to form a majority. Each node can be a control plane, hence there is really no concept of control plane.

Check documentation on MicroK8s Discuss HA

Digitalocean attached volume

This terraform also creates and attach a volume to each droplet. This will let you setup Rook + Ceph storage. This way you can freely create volumes that you can share to your pods.

Persistent Volumes

The following sections describes how to install Rook/Ceph, Longhorn and OpenEBS with MicroK8s

Using Rook / Ceph

Some instructions on how to use Rook

Using Longhorn storage

Some instructions on how to use Longhorn

Using OpenEBS

Instructions on how to install OpenEBS

Worker node only node

If you want to create a worker node only node, refer to these instructions