https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# general
a
This message was deleted.
w
heh 
failed to start cluster controllers
r
w
no, i'm on rancher running on k3s via helm,
not sure if this matters
Copy code
=== <http://rkeclusters.rke.cattle.io|rkeclusters.rke.cattle.io> ===
      host: localhost
r
Were you requesting the DNS name of the Rancher Server UI when you encountered the 404? Is Rancher Server installed in the same K3S cluster where the node was replaced?
w
yes, it i went from k3s 1.21.x all the way up to 1.25.x version by version
I did it by first joining a new node, 1.22, and then shutting down original node..then upgraded 1 version at a time.
I've since switched the IP back so everything is as it was, except for a new hostname on the node
r
Is this an excerpt from the Ingress resource for the Rancher Server UI?
Copy code
=== <http://rkeclusters.rke.cattle.io|rkeclusters.rke.cattle.io> ===
      host: localhost
w
output from 
sudo  kubectl get crd | grep -v ^NAME | awk '{print $1}' | while read CRD; do echo === $CRD ===; sudo kubectl get $CRD -A -o yaml | grep 'localhost'; done
the (now former) new IP 172.16.1.95 is nowhere to be found in the crds. the 172.16.1.167 (old ip, now the current IP):
Copy code
=== <http://nodes.management.cattle.io|nodes.management.cattle.io> ===
      - address: 172.16.1.167
      <http://flannel.alpha.coreos.com/public-ip|flannel.alpha.coreos.com/public-ip>: 172.16.1.167
      <http://k3s.io/internal-ip|k3s.io/internal-ip>: 172.16.1.167
so only thing that has really changed is the hostname of the k8s node, but i would hope that would not matter.
ahh Error: UPGRADE FAILED: chart requires kubeVersion: < 1.25.0-0 which is incompatible with Kubernetes v1.25.7+k3s1
may have something to do with it
👍 1
hmm downgraded and tried to re-upgrade the chart, but i suspect no changes so idempotent
kube-system                 svclb-traefik-415191bb-rwfqw
is pending and helm traefik installs failed.. hmm
didn't traefik upgrade to v2 or something in iirc 1.21?
r
I recall something about k3s moving to Traefik v2.
w
yea
traefik install failed because crds aren't there
Copy code
+ helm_v3 install traefik-crd <https://10.43.0.1:443/static/charts/traefik-crd-20.3.1+up20.3.0.tgz>                                                                                │
│ Error: INSTALLATION FAILED: cannot re-use a name that is still in use                                                                                                            │
│ Stream closed EOF for kube-system/helm-install-traefik-crd-xnsr7 (helm)
ok, problem was the upgrading of the k3s applied new crds versions for traefik and broke them. the helm install/upgrade failed. Had to force delete (remove finalizers) and delete history manually then 
kubectl create -f ./manifests/traefik.yaml
and that fixed it
Dashboard works now
r
Nice work figuring that out.
800 Views
Open in Slack
PreviousNext
Powered by