https://rancher.com/ logo
Title
m

melodic-hamburger-23329

08/08/2022, 12:39 PM
Does Rancher Desktop’s k3s include embedded stargz snapshotter, and if so, how to enable it?
f

freezing-airplane-30363

08/08/2022, 12:51 PM
does not include, it is pure upstream k3s - currently trying to make it run
m

melodic-hamburger-23329

08/08/2022, 12:52 PM
hmm, I see. I would have assumed it did (https://github.com/k3s-io/k3s/issues/2918)
f

freezing-airplane-30363

08/08/2022, 12:53 PM
aha, maybe it is 🙂
m

melodic-hamburger-23329

08/08/2022, 12:55 PM
I feel it is included there somewhere, but no idea how to enable :S (
ctr plugins ls
doesn’t show it)
f

freezing-airplane-30363

08/08/2022, 12:59 PM
could you please try this? ~/Library/Application\ Support/rancher-desktop/lima/_config/override.yaml
env:
  ADDITIONAL_ARGS: --snapshotter=stargz
the arg have been appended to
k3s server
command but not sure what else it should do, here is some docu https://rancher.com/docs/k3s/latest/en/advanced/#enabling-lazy-pulling-of-estargz-experimental
the sample pod from docu ^ is running so I guess it pulled the stargz formatted image correcty
(at least on x86 Linux)
m

melodic-hamburger-23329

08/08/2022, 1:15 PM
Does that work if I have kubernetes disabled?
i.e., only running containerd
f

freezing-airplane-30363

08/08/2022, 1:16 PM
I don't think so, as it is argument for k3s/kubernetes
here is some info how to enable it for nerdctl https://github.com/containerd/nerdctl/blob/master/docs/stargz.md
m

melodic-hamburger-23329

08/09/2022, 2:17 AM
I ended up compiling it inside the VM as the default release from github didn’t seem to work out of the box ^^; Then set up in same way via OpenRC as containerd. tbh, I would prefer to switch to Ubuntu-based image with everything configured via systemd.
Need to set up some hack to make everything persist RD restarts..
f

freezing-airplane-30363

08/09/2022, 8:15 AM
Hi, I ended up with this
override.yaml
and it is persistent, but the config is probably not correct/complete, anyway it might be usefull for you.
containerd:
  system: true # official provisioning system script will be triggered just once on initial run so cusom config.toml will be used
provision:
  - mode: system
	script: |
	  #!/bin/sh
	  apk update
	  apk add mkcert --force-broken-world --repository <https://dl-cdn.alpinelinux.org/alpine/edge/testing>
	  apk add libc6-compat
	  wget <https://github.com/containerd/stargz-snapshotter/releases/download/v0.12.0/stargz-snapshotter-v0.12.0-linux-amd64.tar.gz>
	  tar xf stargz-snapshotter-v0.12.0-linux-amd64.tar.gz
	  ./containerd-stargz-grpc &
  - mode: system
	script: |
	  #!/bin/sh
	  cat <<'EOF' >> /etc/containerd/config.toml
	  version = 2

	  # Plug stargz snapshotter into containerd
	  # Containerd recognizes stargz snapshotter through specified socket address.
	  # The specified address below is the default which stargz snapshotter listen to.
	  [proxy_plugins]
		[proxy_plugins.stargz]
		  type = "snapshot"
		  address = "/run/containerd-stargz-grpc/containerd-stargz-grpc.sock"

	  # Use stargz snapshotter through CRI
	  [plugins."io.containerd.grpc.v1.cri".containerd]
		snapshotter = "stargz"
		disable_snapshot_annotations = false
	  EOF
...
apk add libc6-compat
makes the upstream stargz binary run under limavm and
containerd.system: true
makes the changes "permanent".
m

melodic-hamburger-23329

08/09/2022, 8:18 AM
hmm, I see. I’ll try that one
f

freezing-airplane-30363

08/09/2022, 8:19 AM
but the config is wrong and RD report an error
but maybe you will figure it out
m

melodic-hamburger-23329

08/09/2022, 8:57 AM
containerd:
  system
seems to require systemd. It seems there are quite many things that need changing in order to get this to work. I’ll take a look later if I can get this approach work by using Ubuntu image containing systemd (alpine doesn’t seem to support systemd out of the box..?).
f

freezing-airplane-30363

08/09/2022, 9:09 AM
alpine is using openRC instead of systemd
m

melodic-hamburger-23329

08/09/2022, 9:10 AM
yes, so the above approach doesn’t work with the default image
f

freezing-airplane-30363

08/09/2022, 9:11 AM
why? there is nothing systemd specific
m

melodic-hamburger-23329

08/09/2022, 9:11 AM
I tried switching to Ubuntu 22.04 image but getting error:
/Applications/Rancher <http://Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl|Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl> shell --workdir=. 0 sudo /sbin/rc-service --ifstarted k3s stop
Apparently RD tries to install k3s using openrc even though the OS is systemd-based
what error did you get?
f

freezing-airplane-30363

08/09/2022, 9:12 AM
containerd.system
is about running provisioning scripts for containerd - it has nothing to do with systemd
m

melodic-hamburger-23329

08/09/2022, 9:12 AM
btw, is it possible to pass to RD/k3s some parameters to make it use systemd?
f

freezing-airplane-30363

08/09/2022, 9:14 AM
k3s itself supports systemd but alpine limavm doesn't - you'd need to rewrite lot of stuff (image, provisioning scripts, etc...)
here are the initd scripts https://github.com/rancher-sandbox/rancher-desktop/tree/main/src/assets/scripts which would need to be rewritten
m

melodic-hamburger-23329

08/09/2022, 9:22 AM
hmm, i see. I’ll try with openrc again. maybe had wrong conf somewhere.
is it possible to override those scripts via yaml? I wouldn’t like to compile whole RD..
f

freezing-airplane-30363

08/09/2022, 9:26 AM
no idea
m

melodic-hamburger-23329

08/10/2022, 3:49 AM
@freezing-airplane-30363 It doesn’t seem to work.
containerd.system: true
seems to require systemd:
`time="2022-08-10T12:46:07+09:00" level=fatal msg="degraded, status={Running:true Degraded:true Exiting:false Errors:[1 error occurred:\\n\\t* failed to satisfy the optional requirement 1 of 2 \\"systemd must be available\\": systemd is required to run containerd, but does not seem to be available.\\nMake sure that you use an image that supports systemd. If you do not want to run\\ncontainerd, please make sure that both 'container.system' and 'containerd.user'\\nare set to 'false' in the config file.\\n; skipping further checks: stdout=\\"\\", stderr=\\"+ command -v systemctl\\\\n+ echo 'systemd is not available on this OS'\\\\nsystemd is not available on this OS\\\\n+ exit 1\\\\n\\": failed to execute script \\"systemd must be available\\": stdout=\\"\\", stderr=\\"+ command -v systemctl\\\\n+ echo 'systemd is not available on this OS'\\\\nsystemd is not available on this OS\\\\n+ exit 1\\\\n\\": exit status 1\\n\\n] SSHLocalPort:59256}"\n`,
  code: 1,
  command: [
    '/Applications/Rancher <http://Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl|Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl>',
    'start',
    '--tty=false',
    '/Users/.../Library/Application Support/rancher-desktop/lima/_config/0.yaml'
  ]
}
f

freezing-airplane-30363

08/10/2022, 8:13 AM
Hm, seems you are right