hi all, recently I faced an error after restarting k3s.service on my on-premise environment. Since I was not able to find out the root cause, immediately restored from snapshot. My environment consists of 2 master servers with local etcd database. The first server came up and running after the restore. But another one is not able to join the cluster since restore. I read the github documentation and issues, and found that "restore from snapshot" option restores cluster with only one server and others will stay outside of the cluster. So I deleted -data-dir on the second server and tried to rejoin, but with no luck. It says the below, do you have any suggestions on this? Thanks.

INFO[0005] Tunnel server egress proxy waiting for runtime core to become available
INFO[0005] Waiting to retrieve kube-proxy configuration; server is not ready: <https://127.0.0.1:6443/v1-k3s/readyz>: 500 Internal Server Error
INFO[0010] Tunnel server egress proxy waiting for runtime core to become available
INFO[0010] Waiting to retrieve kube-proxy configuration; server is not ready: <https://127.0.0.1:6443/v1-k3s/readyz>: 500 Internal Server Error
{"level":"info","ts":"2023-01-23T15:28:45.167+0100","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"ef75232eb2491d72 switched to configuration voters=(5985293894695459910 17254736230806527346)"}
{"level":"info","ts":"2023-01-23T15:28:45.167+0100","caller":"membership/cluster.go:535","msg":"promote member","cluster-id":"96831dd7fb68f941","local-member-id":"ef75232eb2491d72"}
INFO[0015] Tunnel server egress proxy waiting for runtime core to become available
INFO[0015] Waiting to retrieve kube-proxy configuration; server is not ready: <https://127.0.0.1:6443/v1-k3s/readyz>: 500 Internal Server Error
{"level":"warn","ts":"2023-01-23T15:28:50.015+0100","caller":"etcdserver/server.go:2065","msg":"failed to publish local member to cluster through raft","local-member-id":"ef75232eb2491d72","local-member-attributes":"{Name:server-002.e636bb1a ClientURLs:[<https://10.31.194.2:2379>]}","request-path":"/0/members/ef75232eb2491d72/attributes","publish-timeout":"15s","error":"etcdserver: request timed out, possibly due to connection lost"}
{"level":"info","ts":"2023-01-23T15:28:50.015+0100","caller":"etcdserver/server.go:2044","msg":"published local member to cluster through raft","local-member-id":"ef75232eb2491d72","local-member-attributes":"{Name:aserver-002.e636bb1a ClientURLs:[<https://10.31.194.2:2379>]}","request-path":"/0/members/ef75232eb2491d72/attributes","cluster-id":"96831dd7fb68f941","publish-timeout":"15s"}