Separated Deployment with Docker and K8s
Here we use the lightweight K3s as an example. If you have already deployed a K8s cluster, you can refer to the subsequent configuration for similar settings.
Please read the entire tutorial before proceeding, otherwise the deployment result may not meet expectations.
Install Docker
You can find the installation instructions on the Docker official website (opens in a new tab), and use the basic configuration in Quick Start to run GZCTF.
Install K3s
K3s is a lightweight k8s distribution that can be quickly deployed on single and multiple machines. Official documentation: https://docs.k3s.io/ (opens in a new tab)
If you only have one machine and want to run GZCTF through Docker, you can specify the Docker backend by adding the following parameters during installation:
INSTALL_K3S_EXEC="--docker"
If you need to run more than 255 challenge containers on a single k3s instance, you need to specify INSTALL_K3S_EXEC
during k3s installation and change node-cidr-mask-size
to the desired subnet size.
INSTALL_K3S_EXEC="--kube-controller-manager-arg=node-cidr-mask-size=16"
The above configuration CANNOT BE CHANGED after installation. Changing the CIDR to /16
can support 65535 Pods.
The above configuration only changes the IP address range used by the node Pods. If you need to change the node's Pod limit, please refer to the following.
And install k3s in the following way. For more information, please refer to k3s installation configuration (opens in a new tab):
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="..." sh -
But this is not recommended, using Docker as a container backend may cause some functions to be unavailable.
If you don't want to use Docker as a container backend, but need to deploy on a single machine, you can refer to the method in K8s Cluster Deployment.
curl -sfL https://get.k3s.io | sh -
# Check for Ready node, takes ~30 seconds
sudo k3s kubectl get node
Chinese users can use a mirror site to speed up installation.
curl -sfL https://rancher-mirror.rancher.cn/k3s/k3s-install.sh | INSTALL_K3S_MIRROR=cn sh -
For multi-node installation and cluster setup, please refer to the official documentation (opens in a new tab).
Configure GZCTF
The connection configuration file for k3s is located at /etc/rancher/k3s/k3s.yaml
, and it can be exported using the following command:
sudo cat /etc/rancher/k3s/k3s.yaml
Use the following command to obtain the IP address of the k3s control-panel machine:
sudo k3s kubectl cluster-info
If it shows 127.0.0.1
, it means that the k3s control-panel is on the current machine. Please use ip a
to check the IP address of the current machine.
You can directly use the IP address or use a domain name, but make sure the domain name resolves to the machine where the k3s control-panel is located, and ensure that the machine running GZCTF can access port 6443 on it.
Save the above output as kube-config.yaml
and change the server
field to the IP address of the machine where the k3s control-panel is located, for example:
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: # ...
server: https://127.0.0.1:6443 # change this to your k3s control-panel's IP or domain
name: default
# ...
Save it to the machine where GZCTF is deployed, in the same folder as compose.yml
, for example, kube-config.yaml
.
Then modify the mount configuration in compose.yml
:
gzctf:
image: gztime/gzctf:latest
restart: always
ports:
- "80:8080"
networks:
default:
volumes:
- "./data/files:/app/files"
- "./appsettings.json:/app/appsettings.json:ro"
- "./kube-config.yaml:/app/kube-config.yaml:ro" # this is required for k8s deployment
# - "/var/run/docker.sock:/var/run/docker.sock" # this is required for docker deployment
depends_on:
- db
Also, change the appsettings.json
file and set the ContainerProvider
field:
{
"Type": "Kubernetes",
"PublicEntry": "ctf.example.com" // change this to your k3s control-panel's IP or domain
}
Restart GZCTF, and then you can use k3s as the container backend. Users who have already used k8s can also refer to the above configuration process to integrate GZCTF into an existing k8s cluster.
Change NodePort Port Range
The default NodePort port range for k3s is 30000-32767, which may not meet your requirements. Therefore, you can modify the NodePort port range of k3s according to your needs.
Run the following commands on the machine where the k3s control-panel is located:
-
sudo nano /etc/systemd/system/k3s.service
-
Edit the
ExecStart
setting below to specifyservice-node-port-range
ExecStart=/usr/local/bin/k3s \ server \ --kube-apiserver-arg=service-node-port-range=20000-50000
-
sudo systemctl daemon-reload
-
sudo systemctl restart k3s
Change the container limit of K3s
The default container limit of K3s is 110, which may not be suitable for a large number of small containers in a competition. Therefore, you can change the container limit of K3s according to your needs.
Run the following commands on the machine where the k3s control-panel is located:
-
sudo nano /etc/rancher/k3s/kubelet.config
-
Edit the
maxPods
setting as followsapiVersion: kubelet.config.k8s.io/v1beta1 kind: KubeletConfiguration maxPods: 500
-
sudo nano /etc/systemd/system/k3s.service
-
Edit the
ExecStart
setting below to specifykubelet-arg
ExecStart=/usr/local/bin/k3s \ server \ --kubelet-arg=config=/etc/rancher/k3s/kubelet.config
-
sudo systemctl daemon-reload
-
sudo systemctl restart k3s
Add Container Image Registry
Using an external container image registry directly is not supported in k3s. You need to add the image registry to k3s.
Run the following commands on the machine where the k3s control-panel is located:
-
sudo nano /etc/rancher/k3s/registries.yaml
-
Edit the
mirrors
setting below to specify the address of the image registry you needmirrors: "container.ctf.example.com": # change this to your registry's domain endpoint: - "https://container.ctf.example.com" # change this to your registry's domain
-
sudo systemctl restart k3s