Hey guys,
I'm having a little bit of trouble with a k3s external datastore.
Here are the details
It seems that every day or so k3s is generating a new authentication key and is trying to write it into my external mysql database.
The problem is that there is apparently a lot of writings when he does that because my sql provider indicate that there is a cpu Outage and the mysql server becomes unavalable during theses writes.
This cause all my nodes to become unavailble because they don't have the new k3s key to authenticate until I physically reboot one of the nodes.
Therefore I was trying migrate my external mysql datastore to an etcd datastore but I don't have any clue on how to do that and even if it's possible, at the end of the k3s documentation for ha-embedded there is a line indicating that it should be possible, but the trys I've made have miserably failed and created new empty k3s clusters 🙂 (source :
https://docs.k3s.io/datastore/ha-embedded)
If any of you have some ideas on how I could resolve the issue I would really be glad because for now i'm forced to restart my nodes every day and my cluster is not reliable in that state 😕
here is the cluster creation script i've been using :
curl -sfL <https://get.k3s.io> | INSTALL_K3S_EXEC="--node-external-ip 192.168.0.10 --service-node-port-range 7777-32767" \
sh -s - --disable=traefik --disable=local-storage --kubelet-arg system-reserved=cpu=500m,memory=1Gi,ephemeral-storage=1Gi \
--kubelet-arg system-reserved=cpu=300m,memory=500Mi --kubelet-arg eviction-hard='memory.available<500Mi,nodefs.available<10%' \
--token ${NODE_TOKEN} \
--datastore-endpoint="mysql://${MYSQL_USER}:${MYSQL_PWD}/tcp(<${MYSQL_URI}:${MYSQL_PORT}>)/${MYSQL_DATABASE}"