https://rancher.com/ logo
#k3os
Title
# k3os
m

magnificent-airplane-36357

08/08/2022, 8:16 AM
Hi! I currently run into a few issues with k3os that I can’t seem to solve. A few questions regarding this: • is there someone or some support we could pay to resolve these issues? • the first issue: after a few days of a cluster node being active, I can see etcd logs complaining about requests taking too long (apply request took too long), which results in the node getting NotReady after a long time (hours) and k3s within k3os not recovering from the condition. After that happens the only two log messages are basically “compact tree index” and “finished scheduled compaction” but no other output (neither positive nor negative) ◦ restarting the node recovers the issue (but not very long) ◦ stopping k3s, deleting the etcd folder, restarting k3s recovers the issue (not sure if it will fail again, since this was the last thing I tried) ◦ I can see a LOT of connections to port 2379 although this cluster currently has only 4 nodes (should have 5, but one is not available right now) ◦ the issue happens randomly on arbitrary nodes (no logic to see right now) ◦ the servers are not fully loaded, are bare metal and should be fast enough not to run into etcd performance issues ◦ I’m running k3OS v0.22.2-k3s2r0 (unfortunately, the cluster was incorrectly set up) • second issue: I cannot see etcd metrics (curl http://localhost:2379/metrics returns an empty response) ◦ this happens on all cluster nodes
13 Views