https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# vsphere
a
This message was deleted.
b
Hi @bitter-shoe-85930 I don’t know the details of your system, but it might not be supported. Please refer to https://www.suse.com/suse-rancher/support-matrix/all-supported-versions/rancher-v2-7-5/ to check. 🙏
👍 2
a
Can you grab the 
rancher-system-agent
service logs from one of the deployed nodes please?
b
Hi @agreeable-oil-87482, the logs attached. As always, any hints are much appreciated.
rancher_system_agent_logs_2508.txt
I was looking at the cattle-system pods and I see the below.
Copy code
k get pods -n cattle-system
NAME                                                              READY   STATUS      RESTARTS   AGE
apply-system-agent-upgrader-on-cluster-c5a7ef-controller-a9lr5q   0/1     Completed   0          11m
apply-system-agent-upgrader-on-cluster-c5a7ef-controller-cnkbl9   0/1     Completed   0          11m
apply-system-agent-upgrader-on-cluster-c5a7ef-worker-93bc-wx6ww   0/1     Completed   0          11m
cattle-cluster-agent-58b67c8658-dg6pt                             1/1     Running     0          15m
cattle-cluster-agent-58b67c8658-m7t4s                             1/1     Running     0          13m
helm-operation-2ktfs                                              1/2     Error       0          13m
helm-operation-8892z                                              1/2     Error       0          14m
helm-operation-dphlp                                              0/2     Completed   0          12m
rancher-webhook-74c9bd4d6-x7gbf                                   1/1     Running     0          11m
system-upgrade-controller-68d57657cb-nswh9                        1/1     Running     0          11m
Copy code
k logs helm-operation-8892z -n cattle-system
Defaulted container "helm" out of: helm, proxy
helm upgrade --force-adopt=true --history-max=5 --install=true --namespace=cattle-system --reset-values=true --timeout=5m0s --values=/home/shell/helm/values-rancher-webhook-2.0.5-up0.3.5.yaml --version=2.0.5+up0.3.5 --wait=true rancher-webhook /home/shell/helm/rancher-webhook-2.0.5-up0.3.5.tgz
Release "rancher-webhook" does not exist. Installing it now.
E0825 06:22:01.759095      26 memcache.go:255] couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request
E0825 06:22:01.779871      26 memcache.go:106] couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request
E0825 06:22:02.590051      26 memcache.go:255] couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request
E0825 06:22:02.591654      26 memcache.go:106] couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request
E0825 06:22:02.608363      26 memcache.go:255] couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request
E0825 06:22:02.623942      26 memcache.go:106] couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request
E0825 06:22:02.633387      26 memcache.go:255] couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request
E0825 06:22:02.654672      26 memcache.go:106] couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request
E0825 06:22:02.675733      26 memcache.go:255] couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request
E0825 06:22:02.691138      26 memcache.go:106] couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request
E0825 06:22:02.700569      26 memcache.go:255] couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request
E0825 06:22:02.717398      26 memcache.go:106] couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request
E0825 06:22:02.727102      26 memcache.go:255] couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request
E0825 06:22:02.734115      26 memcache.go:106] couldn't get resource list for <http://metrics.k8s.io/v1beta1|metrics.k8s.io/v1beta1>: the server is currently unable to handle the request
Error: release: already exists
Not sure if this is relevant
I deployed RKE2 v1.25.12+rke2r1 and experience the same issue.
I tried downgrading the deployment. Now I am using the Rancher supported versions. Rancher Sever - Helm chart 2.7.5 deployed on RKE2 v1.25.11+rke2r1. Downstream RKE2 v1.25.11+rke2r1 clusters, still the same behaviour.
@agreeable-oil-87482 quick update from my side. I upgraded the Rancher server to v2.6.6 and I create RKE2 cluster v2.6.6+rke2r1. The nodes in the cluster are active however, the Rancher-webhook pod will perform three re-tries to become active. As a result, once the Rancher server detects the Rancher-webhook as unhealthy, the cluster gets in the "Provisioning" state. I can get the cluster stats from the UI but not the Rancher "Home" page. Does it make sense at all? At least, those where my observations.
24 Views
Open in Slack
PreviousNext
Powered by