https://rancher.com/ logo
Title
c

cool-pillow-1781

11/03/2022, 4:42 PM
Afternoon I was looking for partitioning guidance on RKE2 and came across this issue: https://github.com/rancher/rke2/issues/2479 Looking at my own cluster I found the following areas to be larger: • /var/lib/kubelet ? ◦ Not sure what the growth potential is • /var/lib/rancher ! ◦ contianerd uses a sub folder as the root storage so container layers should be here • /var/run/k3s ? ◦ Not sure on growth rate • /var/run/containerd ? ◦ Not sure on growth rate • /var/log/pods ! ◦ pod logs • /var/log ! ◦ bane of my existence always seems to fill up 😄 • /run X - seems to be the tmpfs for containerd sockets as well as where the overlayFS is setup ◦ doesn't seem like I want to create a separate partition ▪︎ as this would take me off a RAM FS and put me onto spinners (sadly 😞 ) ? = should I partition ; ! = planning on partitioning ; X = don't plan on partitioning
This was my really cheap way of looking at what was in use in my little play cluster:
This is the mount points for /run and it looks like containerd wizardry I want to stay away from 😄
Note this is just a little test VM setup from my home not real numbers just getting an idea for hardware considerations. Hence my very tiny /run which if more was needed it would have to page to disc I'm guessing.
s

sparse-fireman-14239

11/03/2022, 4:59 PM
If you looked at the mountpoint you’d see that /run is tmpfs and /var/run is a symlink to /run
c

cool-pillow-1781

11/03/2022, 5:01 PM
Thank you totally missed that
Darn can't edit the original but that puts an X by /var/run/*
s

sparse-fireman-14239

11/03/2022, 5:07 PM
There’s a lot going on in /var/lib/rancher - this is where your containers live while they run and until they are removed, also where (at least by default) etcd snapshots are stored, rke2 container images if you run an air-gapped install. In my case containers take up 90-98% of the space.
c

cool-pillow-1781

11/03/2022, 5:08 PM
ok that's what my quick investigation looked like so def a good candidate to get it some extra space 👍
s

sparse-fireman-14239

11/03/2022, 5:09 PM
If it’s a VM it’s easy to add more later, but yeah always try to aim for something sensible to begin with 🙂
On a random worker node I see 99% is used by containers. /var/log/pods is ~100-200MB in my case at the moment, obviously depends on what you’re running, how much they log and such.
.. not 99% of available space, but 99% of used space.
👍 1
If you get into PVs with i.e. Longhorn then space mgmt is a whole other level of wtf.
c

cool-pillow-1781

11/03/2022, 5:17 PM
I'm leaving that as a headache for another day 😄 for now it's just looking at a decent initial baseline for some baremetal nodes
s

sparse-fireman-14239

11/03/2022, 5:18 PM
ah baremetal, yeah you’ll want to get it semi right from the start, unless you can easily add space 🙂
c

cool-pillow-1781

11/03/2022, 5:19 PM
thankfully for me its like a proof of concept type of deal and I'll pass it off to the real experts 😛artyparrot:
but wanna make sure I document well and do my due diligence
s

sparse-fireman-14239

11/03/2022, 5:21 PM
In /var/lib/kubelet I see 1-2GB being used and it appears to be related to PVs which looks odd now that I’m diving into it
Gotta talk to a dev about that 😣
👍 1