That's a bad idea. Every VM you start from that prebuilt cluster will have the same CAs which means a kubeconfig or auth token from one will be valid on all the others as well.
A better idea is to just preinstall k3s and stage manifests for everything you want to deploy in /var/lib/rancher/k3s/server/manifests and set k3s to start up for the first time when you boot the image. DO NOT start k3s on the node before imaging it.
12/03/2022, 9:31 PM
Yeah I do agree regarding putting everything in manifests , but then i will need to wait for the resources to be deployed, and that is what I am trying to avoid. Like save the waiting time needed to deploy the k8s resources.
Regarding the CAs , that is fine. Because this VM will be up for 5 minutes in the CI pipeline , run the tests and then deleted.
There will be a builder vm that build the iso every night ( nightly)
But my question was more focused on what paths I should include in the iso builder scripts that k3s uses
For example you mentioned /var/lib/rancher/k3s
Where can I find all paths that k3s uses?
12/03/2022, 11:20 PM
I think you’ll find that it’s more maintainable if you just package k3s and the deployment manifests. Maybe preload the images in an airgap tarball too if your pulls take a while. Just starting things up shouldn’t take very long at all. There are a bunch of folks using t K3s that way. We do all our CI tests that way, it starts up in seconds.
12/03/2022, 11:23 PM
I will try that and compare both approaches,
Thanks for your help.