I'm having an issue on starting k3s. I thought it ...
# k3s
h
I'm having an issue on starting k3s. I thought it could be because of limited local storage space, so I changed the
--default-local-storage-path
and
--data-dir
to a location that has more space. After a restart, I'm getting the same issue.
Copy code
garbagecollector.go:835] "failed to discover some groups" groups="<internal error: json: unsupported type: map[schema.GroupVersion]error>" 
eviction_manager.go:366] "Eviction manager: attempting to reclaim" resourceName="ephemeral-storage" 
eviction_manager.go:377] "Eviction manager: must evict pod(s) to reclaim" resourceName="ephemeral-storage" 
eviction_manager.go:395] "Eviction manager: pods ranked for eviction" pods=["kube-system/traefik-7d5f6474df-bt446"] 
eviction_manager.go:598] "Eviction manager: cannot evict a critical pod" pod="kube-system/traefik-7d5f6474df-bt446" 
eviction_manager.go:427] "Eviction manager: unable to evict any pods from the node"
c
What do your partitions look like? How full are they?
h
The path I'm using for both above flags has plenty of space.
Ideally I would cleanup the root partition, but that is not an option at the moment.
c
Did you reinstall with a new --data-dir or did you just try to change it on an existing node and then restart the service?
h
Both
c
it’s complaining about insufficient ephemeral-storage, not space in the image store. so changing the data-dir won’t help with that, it’s out of space wherever /tmp or /var/tmp (I can’t remember which) is
you need to make more space available on the root fs.
ephemeral-storage = temp files. not data or images.
h
Okay that makes more sense to me, thanks. How can I check if it's /tmp or /var/tmp?
c
oh actually I guess it’s /var/lib/kubelet
so you could put that on a different fs
h
Yeah I'm just going to do that for now.
How could I have found the location?
c
https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet --root-dir string Default: /var/lib/kubelet Directory path for managing kubelet files (volume mounts, etc).
👍 1
I would recommend not trying to change that path, a bunch of stuff assumes the default value
just mount another partition at that path
h
Yeah I was going to ask, but figured it would break a lot of stuff.
thanks again for your help.
313 Views