Channels
academy
amazon
arm
azure
cabpr
chinese
ci-cd
danish
deutsch
developer
elemental
epinio
espanol
events
extensions
fleet
français
gcp
general
harvester
harvester-dev
hobbyfarm
hypper
japanese
k3d
k3os
k3s
k3s-contributor
kim
kubernetes
kubewarden
lima
logging
longhorn-dev
longhorn-storage
masterclass
mesos
mexico
nederlands
neuvector-security
office-hours
one-point-x
onlinemeetup
onlinetraining
opni
os
ozt
phillydotnet
portugues
rancher-desktop
rancher-extensions
rancher-setup
rancher-wrangler
random
rfed_ara
rio
rke
rke2
russian
s3gw
service-mesh
storage
submariner
supermicro-sixsq
swarm
terraform-controller
terraform-provider-rancher2
terraform-provider-rke
theranchcast
training-0110
training-0124
training-0131
training-0207
training-0214
training-1220
ukranian
v16-v21-migration
vsphere
windows
Title
r
rich-cartoon-70161
01/03/2023, 2:03 PMIs it possible to downgrade a HA control-plane setup to a single server one?
For some reason when I stop 2 of the 3 the last control plane node also becomes also unhappy because of health checks to the others
s
sticky-summer-13450
01/03/2023, 2:23 PMOnce your HA cluster doesn't have quorum any more (i.e less than 50% of the control-plane nodes are available) it cannot continue.
r
rich-cartoon-70161
01/03/2023, 2:24 PMJeah, the question is… can I somehow reconfigure it to allow it to only have 1 control-plane node?
s
sticky-summer-13450
01/03/2023, 2:24 PMI don't know whether you can remove the two nodes from the cluster, but it feels like you're in a catch-22 situation
r
rich-cartoon-70161
01/03/2023, 2:50 PMyou read my issue in the post before? jeah… it pretty much feels like it. and I thought downgrading again to single-control-plane would make things easier… and then adding server nodes again later.
s
sticky-summer-13450
01/03/2023, 4:07 PMI had skimmed your issue previously, but it wasn't an issue I'd noticed.
I've tried looking for FAQs about not being able to scale down from a 3 node HA to a single node, but I haven't found anything yet. Maybe it was said in Slack and has disappeared.
c
creamy-pencil-82913
01/03/2023, 6:37 PMstarting a node with --cluster-reset will make it the only member of the etcd cluster. At that point you could delete and rejoin the other nodes as agents.
r
rich-cartoon-70161
01/03/2023, 6:40 PMSo I would stop all three server instances and then restart one with cluster-reset and it’ll still have the data but as single control plane node? That sounds good and might make me solve the issue I posted as GitHub issue above :D
Would I have to recreate all agents again or would they automatically be joined again if the token stays the same?
I’ll try this out tomorrow in a test Cluster :) but thanks that’s a good lead
c
creamy-pencil-82913
01/03/2023, 6:47 PMstop the k3s service on all three nodes, run
k3s server --cluster-resetthen reinstall the other servers as agents
and yes, this keeps all the datastore data
s
sticky-summer-13450
01/03/2023, 6:49 PMWow - that really is magic.
c
creamy-pencil-82913
01/03/2023, 6:58 PMThat flag is documented in passing at https://docs.k3s.io/backup-restore#options
r
rich-cartoon-70161
01/04/2023, 1:38 PMThis seems to be a feasible way out of my checkmate-situation 🙂 downscaling the control-plane to 1 server, changing the important config flags and then adding more servers again seems to work fine.
@creamy-pencil-82913 thanks again for the hint with the
--cluster-reset