#CGGQEHPPW I need help with an NodeDiskPressure Issue. I have installed k3s on single node ubuntu server with below specs CPU: 28 cores Memory: 64 GB Storage: 1 TB. Despite these ample resources, I’m encountering frequent DiskPressure warnings. The kubelet logs show errors similar to:

Warning  FreeDiskSpaceFailed    58m   kubelet  Failed to garbage collect required amount of images. Attempted to free 6681749913 bytes, but only found 0 bytes eligible to free. Warning  ImageGCFailed          58m   kubelet  Failed to garbage collect required amount of images. Attempted to free 6681749913 bytes, but only found 0 bytes eligible to free. Warning  FreeDiskSpaceFailed    53m   kubelet  Failed to garbage collect required amount of images. Attempted to free 8402172313 bytes, but only found 301463 bytes eligible to free. ...

The errors repeat periodically with varying byte counts, and eventually, the node status sometimes reports NodeHasNoDiskPressure before the errors recur. kubelet configuration :

"evictionHard": {

"imagefs.available": "5%",

"nodefs.available": "5%"

},

"evictionPressureTransitionPeriod": "5m0s",

"evictionMinimumReclaim": {

"imagefs.available": "10%",

"nodefs.available": "10%"

},

Queries : What might be causing k3s to fail at garbage collecting images even though there appears to be ample disk space? Are there specific k3s configuration options or known issues that could lead to these DiskPressure warnings? How can I further diagnose or resolve this issue in my k3s single-node environment?

you have provided no information on your disk layout or how much is actually used/free on any of the partitions. Have you looked at that yet?

Screenshot 2025-03-25 at 12.54.07 PM.png

https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet/ > --image-gc-high-threshold int32 Default: 85 > The percent of disk usage after which image garbage collection is always run. Values must be within the range [0, 100], To disable image garbage collection, set to 100. > --image-gc-low-threshold int32 Default: 80 > The percent of disk usage before which image garbage collection is never run. Lowest disk usage to garbage collect to. Values must be within the range [0, 100] and should not be larger than that of --image-gc-high-threshold. The kubelet is trying to gc down to 80% disk free. You are at 81% and apparently it cannot find any more unused images to remove to free space. You can change that setting, or find some other way to make more space available.

please note that you have a 1TB drive, but your / is only 100GB, and everything is there, logs, os, kubernetes, container images, you're not using /data at all (which is 800GB)

Any reference on how can I change / increase the ephmeral storage for this kubelet or increase the eviction threshold ?

Yes I linked you to the docs on how to change the thresholds. But a better idea would be to increase the size of the correct partition instead of allocating your space to a location that is not used by Kubernetes