Creating a k3s cluster

Kubernetes

Kubernetes is an open-source container-orchestration platform for automating application deployment, scaling, and management.

Requirements

1. Create at least 4 machines, 2 of these will be the masters and the other 2 will be the worker nodes. This is the minimum for a High Availability cluster. These can either be virtual machines or physical machines like raspberry pi. I will be using ubuntu for my machines.
2. Install docker if you have not yet onto the 4 machines as this is the base that kubernetes needs to run on.
3. Create another server or use one of your 4 machine for nginx and mysql. Nginx will be used as a loadbalancer and mysql will be for the backend.

Mysql set up

• create user for local and remote

    CREATE USER 'newuser'@'localhost' IDENTIFIED BY 'password';
  

  For remote, you just need to change the localhost to the network you are on. For example if your ip scheme is 192.168.1.X then you would replace localhost with ‘192.168.1.%’. the ‘%’ is known as a wildcard, meaning anything on that 4th section will be allowed.

• You will now need to allow the server with nginx and mysql to allow remote connections to mysql. Change /etc/mysql/mysql.conf.d/mysqld.cnf with nano or vim to allow remote on bind-address 0.0.0.0 then restart mysql.

• Then we will need to alter the account password so the remote connection will work.

    ALTER USER 'yourusername'@'localhost' IDENTIFIED WITH mysql_native_password BY 'youpassword';
  

• Create the database and give the user you created access to it.

    CREATE DATABASE k3s;
    GRANT ALL PRIVILEGES ON k3s.* TO 'username'@'localhost';
  

  Make sure you replace localhost with your IP Address as well.

NGINX set up

1. Now to create the nginx load balancer. It is possible to run NGINX as a docker container. I will not be running it as a container but I included what you need if you are or not. Just replace the X.X.X.X with the IP Address of your servers.

    #uncomment this next line if you are NOT running nginx in docker
    load_module /usr/lib/nginx/modules/ngx_stream_module.so;
   
    events {}
   
    stream {
    upstream k3s_servers {
        server X.X.X.X:6443;
        server X.X.X.X:6443;
    }
   
    server {
        listen 6443;
        proxy_pass k3s_servers;
    }
    }
   

   K3s Install

2. Now if you have not installed docker on to your nodes, then do so now. If you are unsure if you have docker, then use the command docker --version. Once that is done, we can download k3s on the masters.

    curl -sfL https://get.k3s.io | sudo sh -s - server \
    --datastore-endpoint='mysql://USERNAME:PASSWORD@tcp(X.X.X.X:3306)/k3s' \
    --node-taint CriticalAddonsOnly=true:NoExecute --tls-san X.X.X.X
   

   The datastore-endpoint parameters USERNAME and PASSWORD will be replaced with the respective account credentials you made for mysql. The X.X.X.X:3306 should be replaced with the ip of the machine running mysql.

   The tls-san will be the IP of your loadbalancer.

   Note: If you mess up to uninstall k3s

   /usr/local/bin/k3s-uninstall.sh

3. Check that the nodes are seeing each other.

   sudo k3s kubectl get node

4. Get the token from one of the masters

   sudo cat /var/lib/rancher/k3s/server/node-token

5. on each of the worker nodes, install the agent

   curl -sfL https://get.k3s.io | K3S_URL=https://X.X.X.X:6443 K3S_TOKEN={token} sh -

   replace the X.X.X.X with the ip of your load balancer.

6. once all added, on the one of the masters get the config

   sudo cat /etc/rancher/k3s/k3s.yaml

7. On either your machine or some other machine install kubectl (ubuntu).

   curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
   sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
   kubectl version --client
   

8. Then take your configuration and paste it on your main machine.

   mkdir ~/.kube && nano ~/.kube/config

9. Change the localhost ip 127.0.0.1 to your load balancer IP.

10. Now you should be able to run kubectl get nodes on your machine and see your nodes!