Dec 21 19:11:16 prod-rke-server-1 rke2[733]: time="2025-12-21T19:11:16Z" level=info msg="Failed to test etcd connection: failed to get etcd status: rpc error: code = Unavailable desc = connection error: desc = \"transport: authentication handshake failed: context deadline exceeded\""

could it be a certificate issue?

also the cli does not work

etcdctl member list
{"level":"warn","ts":"2025-12-21T19:13:46.260964Z","logger":"etcd-client","caller":"v3/retry_interceptor.go:63","msg":"retrying of unary invoker failed","target":"<etcd-endpoints://0xc0000283c0/127.0.0.1:2379>","attempt":0,"error":"rpc error: code = DeadlineExceeded desc = context deadline exceeded"}
Error: context deadline exceeded

looking at the etcd container logs

{"level":"info","ts":"2025-12-21T19:15:53.287673Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8d85e5c6c784841 is starting a new election at term 250"}
{"level":"info","ts":"2025-12-21T19:15:53.287700Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8d85e5c6c784841 became pre-candidate at term 250"}
{"level":"info","ts":"2025-12-21T19:15:53.287711Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8d85e5c6c784841 received MsgPreVoteResp from b8d85e5c6c784841 at term 250"}
{"level":"info","ts":"2025-12-21T19:15:53.287730Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8d85e5c6c784841 [logterm: 250, index: 894172459] sent MsgPreVote request to 71bf100e144b2cac at term 250"}
{"level":"warn","ts":"2025-12-21T19:15:57.881377Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_RAFT_MESSAGE","remote-peer-id":"71bf100e144b2cac","rtt":"0s","error":"dial tcp 10.3.1.100:2380: i/o timeout"}
{"level":"warn","ts":"2025-12-21T19:15:57.881430Z","caller":"rafthttp/probing_status.go:68","msg":"prober detected unhealthy status","round-tripper-name":"ROUND_TRIPPER_SNAPSHOT","remote-peer-id":"71bf100e144b2cac","rtt":"0s","error":"dial tcp 10.3.1.100:2380: i/o timeout"}
{"level":"info","ts":"2025-12-21T19:15:58.287289Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8d85e5c6c784841 is starting a new election at term 250"}
{"level":"info","ts":"2025-12-21T19:15:58.287320Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8d85e5c6c784841 became pre-candidate at term 250"}
{"level":"info","ts":"2025-12-21T19:15:58.287334Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8d85e5c6c784841 received MsgPreVoteResp from b8d85e5c6c784841 at term 250"}
{"level":"info","ts":"2025-12-21T19:15:58.287357Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"b8d85e5c6c784841 [logterm: 250, index: 894172459] sent MsgPreVote request to 71bf100e144b2cac at term 250"}
{"level":"warn","ts":"2025-12-21T19:15:58.858907Z","caller":"etcdserver/server.go:2167","msg":"failed to publish local member to cluster through raft","local-member-id":"b8d85e5c6c784841","local-member-attributes":"{Name:prod-rke-server-1-50d2191e ClientURLs:[<https://10.10.5.50:2379>]}","request-path":"/0/members/b8d85e5c6c784841/attributes","publish-timeout":"15s","error":"etcdserver: request timed out"}

seems like there is a vote issue

tried to run cluster-reset but getting

WARN[0032] failed to list etcd members: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:2379: connect: connection refused" 
INFO[0035] Failed to test etcd connection: failed to get etcd status: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:2379: connect: connection refused"

even there

Why are you doing a cluster reset? Just start the other nodes. Now you'll have to remove the DB from the others and rejoin them to the cluster.

the reset helped me getting up the cluster again