Channels
extensions
rancher-extensions
supermicro-sixsq
ozt
os
one-point-x
rio
onlinemeetup
v16-v21-migration
events
onlinetraining
training-1220
training-0110
training-0124
k3s-contributor
submariner
storage
k3os
windows
ci-cd
theranchcast
rfed_ara
nederlands
ukranian
danish
training-0131
masterclass
training-0207
training-0214
terraform-controller
epinio
phillydotnet
swarm
rancher-wrangler
deutsch
russian
chinese
mesos
français
japanese
arm
longhorn-dev
terraform-provider-rke
service-mesh
azure
gcp
academy
random
elemental
espanol
lima
harvester-dev
portugues
kubernetes
kim
hypper
kubewarden
opni
logging
neuvector-security
rancher-setup
developer
office-hours
mexico
cabpr
rke
vsphere
longhorn-storage
k3s
k3d
terraform-provider-rancher2
fleet
amazon
harvester
rke2
s3gw
hobbyfarm
rancher-desktop
general
Title
a
adorable-toddler-38962
05/24/2023, 3:57 AMHey all I’m having a super weird issue after upgrading my cluster to HA embedded etcd… I still just connect to a single server node for kubectl but it seems pretty often that I’ll get no response or server not responding errors when I do normal operations like listing pods. I’m not sure how to debug this… any ideas? I’ve got a 6 node cluster, 3 of which are server nodes. Centos stream 9 vms on proxmox
c
creamy-pencil-82913
05/24/2023, 4:24 AMYou check the logs for errors?
If they're all vms on the same backing disk it's highly likely your disk doesn't have the iops for your workload + etcd
Etcd writes are highly latency sensitive
a
adorable-toddler-38962
05/24/2023, 5:53 AMWhich logs should I check and what should I look for to diagnose that?
The vms are split across 2 separate machines each running ssds as the backing disks
c
creamy-pencil-82913
05/24/2023, 7:33 AMJust the k3s journald logs
a
adorable-toddler-38962
05/24/2023, 1:54 PMSo maybe something that would help would be to make my control plane non scheduleable? how do i add the taints to my control plane node to do that? I see this
--node-taint CriticalAddonsOnly=true:NoExecute