https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# k3d
a
This message was deleted.
w
I'd volume mount the config and put the 
k3d cluster create
command as the command of the DinD container. Though you'll have to add something there that keeps the container running then.
What's your use case for k3d DinD even?
t
put the 
k3d cluster create
command as the command of the DinD container
was my first intention, but it seems to override default command for container, and 
docker-entrypoint-initdb.d
is not started then (docker agent becomes inaccessible and container dies after few tries).
What's your use case for k3d DinD even?
I need a local development environment for multi-layer api-gateway configurations (with kong). So intention is to start k8s cluster in container and have another container to apply helm charts against this cluster, with helm chart values being provided from mounted volume for this container. As I want it to be portable and reusable, the only requirement for host machine is working docker/docker-compose, without platform-specific k3d binaries, so I decided to go with k3d-dind, and so far it seems promising, just want to make setup less manual.
w
So the only pain point is the k3d binary, right? K3s has auto-deploy manifests and a HelmChart controller built-in, so a volume mount for your charts, etc. would work there. Then use k3d without DinD and use the host docker instead of nested docker. Less layers and probably simpler after all.
WDYT?
t
that would add the requirement to have platform-specific k3d binary at host machine, right?
w
Nope, a container that has the k3d binary
Just need to mount the docker socket into the k3d container so it can use that.
t
Perhaps got your idea. So cluster would be created in host machine, and 
docker ps
on host machine would show running agents/servers containers?
w
Exactly, just like you normally use k3d, but the k3d binary would be inside a container.
t
ok, thnx for idea, will play with that. But I guess original question about auto-apply of k3d config would be still actual?
w
So e.g. 
alias k3d='docker run -v docker.sock:docker.sock k3d:v5
(adjust socket paths and image, then as usual do 
k3d cluster create
and either you have the config mounted as well or pass it on via stdin. I'm on my phone so I didn't test anything of this
🙌 1
t
roger, thank you. Will see how to adapt it for better devexp
fun times:
Copy code
docker run -v /var/run/docker.sock:/var/run/docker.sock -v "$(pwd)/k3d:/k3d" <http://ghcr.io/k3d-io/k3d:5.5.1|ghcr.io/k3d-io/k3d:5.5.1> cluster create --config /k3d/k3d-cluster-config.yml
FATA[0000] error creating temp copy of configfile /k3d/k3d-cluster-config.yml for variable expansion: open /tmp/k3d-config-tmp-k3d-cluster-config.yml3060607825: no such file or directory
Error comes from here. I am pretty sure it is related to the fact that in scratch image there is no temp dir and powers to write to it. I made a workaround by mounting my volume to 
/tmp/
directory, as upon container creation it can be written, and it worked. I didn't find similarities under github issues. Should I open a PR?
@wide-garage-9465 sorry to bother, what do you think about this last one?
w
Ooooh
Yeah.. I think it's a 
scratch
image without a filesystem 😅
So 
/tmp
does not exist inside the k3d image, as it only contains the k3d binary and nothing else.
That's an easy fix though
Pushed 85110aef which will be in the next release, so the binary-only image will have an empty /tmp
🙌 1
t
When having this server running inside cluster, I can see periodic error in the log, saying
Copy code
W0802 08:32:15.439695       8 sysinfo.go:203] Nodes topology is not available, providing CPU topology
E0802 08:32:15.441704       8 info.go:114] Failed to get system UUID: open /etc/machine-id: no such file or directory
W0802 08:32:15.442084       8 info.go:53] Couldn't collect info from any of the files in "/etc/machine-id,/var/lib/dbus/machine-id"
which I believe comes for similar reason. Mb there are more cases.
But at least you know that nature of scratch container is not playing well with k3d binary, as it relies on being run on "real" OS env
thank you for quick update
w
Running k3d without access to the host system resources in general. E.g. memory-limits, etc. have no chance to work without it. The logs you see there are from K3s though, so that's the general "issue" (or not) of running K3s in a container.
👍 1
99 Views
Open in Slack
PreviousNext
Powered by