Separated Deployment with Docker and K8s

Here we use the lightweight K3s as an example. If you have already deployed a K8s cluster, you can refer to the subsequent configuration for similar settings.

⚠️

Please read the entire tutorial before proceeding, otherwise the deployment result may not meet expectations.

Install Docker

You can find the installation instructions on the Docker official website (opens in a new tab), and use the basic configuration in Quick Start to run GZCTF.

Install K3s

K3s is a lightweight k8s distribution that can be quickly deployed on single and multiple machines. Official documentation: https://docs.k3s.io/ (opens in a new tab)

If you only have one machine and want to run GZCTF through Docker, you can specify the Docker backend by adding the following parameters during installation:

INSTALL_K3S_EXEC="--docker"

And install k3s in the following way. For more information, please refer to k3s installation configuration (opens in a new tab):

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="..." sh -

But this is not recommended, using Docker as a container backend may cause some functions to be unavailable.

If you don't want to use Docker as a container backend, but need to deploy on a single machine, you can refer to the method in K8s Cluster Deployment.

curl -sfL https://get.k3s.io | sh -
# Check for Ready node, takes ~30 seconds
sudo k3s kubectl get node

Chinese users can use a mirror site to speed up installation.

curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh -

For multi-node installation and cluster setup, please refer to the official documentation (opens in a new tab).

Configure GZCTF

The connection configuration file for k3s is located at /etc/rancher/k3s/k3s.yaml, and it can be exported using the following command:

sudo cat /etc/rancher/k3s/k3s.yaml

Use the following command to obtain the IP address of the k3s control-panel machine:

sudo k3s kubectl cluster-info

If it shows 127.0.0.1, it means that the k3s control-panel is on the current machine. Please use ip a to check the IP address of the current machine.

You can directly use the IP address or use a domain name, but make sure the domain name resolves to the machine where the k3s control-panel is located, and ensure that the machine running GZCTF can access port 6443 on it.

Save the above output as kube-config.yaml and change the server field to the IP address of the machine where the k3s control-panel is located, for example:

apiVersion: v1
clusters:
  - cluster:
      certificate-authority-data: # ...
      server: https://127.0.0.1:6443 # change this to your k3s control-panel's IP or domain
    name: default
# ...

Save it to the machine where GZCTF is deployed, in the same folder as docker-compose.yml, for example, kube-config.yaml. Then modify the mount configuration in docker-compose.yml:

gzctf:
  image: gztime/gzctf:latest
  restart: always
  ports:
    - "80:8080"
  networks:
    default:
  volumes:
    - "./data/files:/app/files"
    - "./appsettings.json:/app/appsettings.json:ro"
    - "./kube-config.yaml:/app/kube-config.yaml:ro" # this is required for k8s deployment
    # - "/var/run/docker.sock:/var/run/docker.sock" # this is required for docker deployment
  depends_on:
    - db

Also, change the appsettings.json file and set the ContainerProvider field:

{
  "Type": "Kubernetes",
  "PublicEntry": "ctf.example.com" // change this to your k3s control-panel's IP or domain
}

Restart GZCTF, and then you can use k3s as the container backend. Users who have already used k8s can also refer to the above configuration process to integrate GZCTF into an existing k8s cluster.

Change NodePort Port Range

The default NodePort port range for k3s is 30000-32767, which may not meet your requirements. Therefore, you can modify the NodePort port range of k3s according to your needs.

Run the following commands on the machine where the k3s control-panel is located:

sudo nano /etc/systemd/system/k3s.service

Edit the ExecStart setting below to specify service-node-port-range

ExecStart=/usr/local/bin/k3s \
    server \
    --kube-apiserver-arg service-node-port-range=20000-50000

sudo systemctl daemon-reload
sudo systemctl restart k3s

Change the container limit of K3s

The default container limit of K3s is 110, which may not be suitable for a large number of small containers in a competition. Therefore, you can change the container limit of K3s according to your needs.

Reference: How do you increase maximum pods per node in K3S? (opens in a new tab)

Run the following commands on the machine where the k3s control-panel is located:

sudo nano /etc/rancher/k3s/kubelet.config

Edit the maxPods setting as follows

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
maxPods: 500

sudo nano /etc/systemd/system/k3s.service

Edit the ExecStart setting below to specify kubelet-arg

ExecStart=/usr/local/bin/k3s \
    server \
    --kubelet-arg=config=/etc/rancher/k3s/kubelet.config

sudo systemctl daemon-reload
sudo systemctl restart k3s

If you need to run more than 255 Pods on a single k3s instance, it may exceed the default subnet size. You need to specify INSTALL_K3S_EXEC during k3s installation and change node-cidr-mask-size to the desired subnet size.

INSTALL_K3S_EXEC="--kube-controller-manager-arg=node-cidr-mask-size=16"

Install k3s using the following format, for more information please refer to k3s installation configuration (opens in a new tab):

curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_EXEC="..." sh -

Add Container Image Registry

Using an external container image registry directly is not supported in k3s. You need to add the image registry to k3s.

Run the following commands on the machine where the k3s control-panel is located:

sudo nano /etc/rancher/k3s/registries.yaml

Edit the mirrors setting below to specify the address of the image registry you need

mirrors:
  "container.ctf.example.com": # change this to your registry's domain
    endpoint:
      - "https://container.ctf.example.com" # change this to your registry's domain

sudo systemctl restart k3s

appsettings.json Deploy with K8s Only