I tried to install my first home cluster. I used 3x Raspberry with 8GB RAM and M2 NVMe disks and built a k3s cluster with three masters (etcd).
I created several pods (influxdb, nodered, ...). But I am surprised that when I physically disconnect node #3, it takes a long time for the cluster to detect it. Then the pod running on node #3 starts to terminate, but gets stuck in "Terminating" state. A new pod starts on node #2, but gets stuck in "pending" state because it is waiting for the persistent volume to be released.
I would like that when one node is down, all pods start immediately on another.
What am I doing wrong? Where should I look for the error?