https://rancher.com/ logo
Join SlackCommunities
Powered by
This message was deleted.
# rke2
a
This message was deleted.
s
Copy code
root@vxx7xxxx:~# kubectl get pod -A -o wide
NAMESPACE      NAME                                                    READY   STATUS      RESTARTS        AGE     IP               NODE                   NOMINATED NODE   READINESS GATES
cert-manager   cert-manager-55cf8685cb-fbbh5                           1/1     Running     0               166m    10.42.1.6        vxx6xxxx.xxx.com   <none>           <none>
cert-manager   cert-manager-cainjector-fbd548cb8-vst4x                 1/1     Running     0               166m    10.42.2.6        vxx5xxxx.xxx.com   <none>           <none>
cert-manager   cert-manager-webhook-655b4d58fb-d4r9s                   1/1     Running     0               166m    10.42.1.5        vxx6xxxx.xxx.com   <none>           <none>
kube-system    cloud-controller-manager-vxx5xxxx.xxx.com           1/1     Running     0               3h35m   209.xxx.xxx.xxx    vxx5xxxx.xxx.com   <none>           <none>
kube-system    cloud-controller-manager-vxx6xxxx.xxx.com           1/1     Running     1 (75m ago)     3h34m   194.xxx.xxx.xxx   vxx6xxxx.xxx.com   <none>           <none>
kube-system    cloud-controller-manager-vxx7xxxx.xxx.com           1/1     Running     1 (3h21m ago)   3h40m   66.xxx.xxx.xxx    vxx7xxxx.xxx.com   <none>           <none>
kube-system    etcd-vxx5xxxx.xxx.com                               1/1     Running     0               3h35m   209.xxx.xxx.xxx    vxx5xxxx.xxx.com   <none>           <none>
kube-system    etcd-vxx6xxxx.xxx.com                               1/1     Running     0               3h34m   194.xxx.xxx.xxx   vxx6xxxx.xxx.com   <none>           <none>
kube-system    etcd-vxx7xxxx.xxx.com                               1/1     Running     1 (3h39m ago)   3h39m   66.xxx.xxx.xxx    vxx7xxxx.xxx.com   <none>           <none>
kube-system    helm-install-rke2-canal-4s9zt                           0/1     Completed   0               3h40m   66.xxx.xxx.xxx    vxx7xxxx.xxx.com   <none>           <none>
kube-system    helm-install-rke2-coredns-b7xnk                         0/1     Completed   0               3h40m   66.xxx.xxx.xxx    vxx7xxxx.xxx.com   <none>           <none>
kube-system    helm-install-rke2-ingress-nginx-vfq9s                   0/1     Completed   0               3h40m   10.42.0.3        vxx7xxxx.xxx.com   <none>           <none>
kube-system    helm-install-rke2-metrics-server-vxmvw                  0/1     Completed   0               3h40m   10.42.0.10       vxx7xxxx.xxx.com   <none>           <none>
kube-system    helm-install-rke2-snapshot-controller-crd-fpvmj         0/1     Completed   0               3h40m   10.42.0.5        vxx7xxxx.xxx.com   <none>           <none>
kube-system    helm-install-rke2-snapshot-controller-xp5bw             0/1     Completed   0               3h40m   10.42.0.9        vxx7xxxx.xxx.com   <none>           <none>
kube-system    helm-install-rke2-snapshot-validation-webhook-8lpd9     0/1     Completed   0               3h40m   10.42.0.4        vxx7xxxx.xxx.com   <none>           <none>
kube-system    kube-apiserver-vxx5xxxx.xxx.com                     1/1     Running     0               3h35m   209.xxx.xxx.xxx    vxx5xxxx.xxx.com   <none>           <none>
kube-system    kube-apiserver-vxx6xxxx.xxx.com                     1/1     Running     0               3h34m   194.xxx.xxx.xxx   vxx6xxxx.xxx.com   <none>           <none>
kube-system    kube-apiserver-vxx7xxxx.xxx.com                     1/1     Running     1 (3h39m ago)   3h39m   66.xxx.xxx.xxx    vxx7xxxx.xxx.com   <none>           <none>
kube-system    kube-controller-manager-vxx5xxxx.xxx.com            1/1     Running     0               3h35m   209.xxx.xxx.xxx    vxx5xxxx.xxx.com   <none>           <none>
kube-system    kube-controller-manager-vxx6xxxx.xxx.com            1/1     Running     1 (6m41s ago)   3h34m   194.xxx.xxx.xxx   vxx6xxxx.xxx.com   <none>           <none>
kube-system    kube-controller-manager-vxx7xxxx.xxx.com            1/1     Running     1 (3h21m ago)   3h40m   66.xxx.xxx.xxx    vxx7xxxx.xxx.com   <none>           <none>
kube-system    kube-proxy-vxx5xxxx.xxx.com                         1/1     Running     0               3h35m   209.xxx.xxx.xxx    vxx5xxxx.xxx.com   <none>           <none>
kube-system    kube-proxy-vxx6xxxx.xxx.com                         1/1     Running     0               3h34m   194.xxx.xxx.xxx   vxx6xxxx.xxx.com   <none>           <none>
kube-system    kube-proxy-vxx7xxxx.xxx.com                         1/1     Running     1 (3h38m ago)   3h40m   66.xxx.xxx.xxx    vxx7xxxx.xxx.com   <none>           <none>
kube-system    kube-scheduler-vxx5xxxx.xxx.com                     1/1     Running     0               3h35m   209.xxx.xxx.xxx    vxx5xxxx.xxx.com   <none>           <none>
kube-system    kube-scheduler-vxx6xxxx.xxx.com                     1/1     Running     1 (75m ago)     3h34m   194.xxx.xxx.xxx   vxx6xxxx.xxx.com   <none>           <none>
kube-system    kube-scheduler-vxx7xxxx.xxx.com                     1/1     Running     2 (3h21m ago)   3h40m   66.xxx.xxx.xxx    vxx7xxxx.xxx.com   <none>           <none>
kube-system    rke2-canal-b2gc7                                        2/2     Running     0               3h40m   66.xxx.xxx.xxx    vxx7xxxx.xxx.com   <none>           <none>
kube-system    rke2-canal-tmfzt                                        2/2     Running     0               3h36m   194.xxx.xxx.xxx   vxx6xxxx.xxx.com   <none>           <none>
kube-system    rke2-canal-x9jhc                                        2/2     Running     0               3h36m   209.xxx.xxx.xxx    vxx5xxxx.xxx.com   <none>           <none>
kube-system    rke2-coredns-rke2-coredns-565dfc7d75-2ml5w              1/1     Running     0               3h40m   10.42.0.8        vxx7xxxx.xxx.com   <none>           <none>
kube-system    rke2-coredns-rke2-coredns-565dfc7d75-9mctd              1/1     Running     0               3h36m   10.42.2.2        vxx5xxxx.xxx.com   <none>           <none>
kube-system    rke2-coredns-rke2-coredns-autoscaler-6c48c95bf9-2ld8l   1/1     Running     0               3h40m   10.42.0.2        vxx7xxxx.xxx.com   <none>           <none>
kube-system    rke2-ingress-nginx-controller-89s2w                     1/1     Running     0               72s     10.42.2.12       vxx5xxxx.xxx.com   <none>           <none>
kube-system    rke2-ingress-nginx-controller-q6bmv                     1/1     Running     0               109s    10.42.1.11       vxx6xxxx.xxx.com   <none>           <none>
kube-system    rke2-ingress-nginx-controller-q8wc4                     1/1     Running     0               36s     10.42.0.21       vxx7xxxx.xxx.com   <none>           <none>
kube-system    rke2-metrics-server-c9c78bd66-h4l5c                     1/1     Running     0               3h38m   10.42.0.12       vxx7xxxx.xxx.com   <none>           <none>
kube-system    rke2-snapshot-controller-6f7bbb497d-j9sfg               1/1     Running     1 (3h21m ago)   3h38m   10.42.0.11       vxx7xxxx.xxx.com   <none>           <none>
kube-system    rke2-snapshot-validation-webhook-65b5675d5c-6vlxx       1/1     Running     0               3h38m   10.42.0.6        vxx7xxxx.xxx.com   <none>           <none>
root@vxx7xxxx:~# kubectl get service -A -o wide
NAMESPACE       NAME                                      TYPE           CLUSTER-IP      EXTERNAL-IP            PORT(S)          AGE     SELECTOR
cattle-system   rancher                                   ClusterIP      10.43.177.135   <none>                 80/TCP,443/TCP   162m    app=rancher
cert-manager    cert-manager                              ClusterIP      10.43.73.1      <none>                 9402/TCP         166m    app.kubernetes.io/component=controller,app.kubernetes.io/instance=cert-manager,app.kubernetes.io/name=cert-manager
cert-manager    cert-manager-webhook                      ClusterIP      10.43.197.187   <none>                 443/TCP          166m    app.kubernetes.io/component=webhook,app.kubernetes.io/instance=cert-manager,app.kubernetes.io/name=webhook
default         kubernetes                                ClusterIP      10.43.0.1       <none>                 443/TCP          3h40m   <none>
default         vxx7xxxx-service                         ExternalName   <none>          vxx7xxxx.xxx.com   <none>           38m     <none>
kube-system     rke2-coredns-rke2-coredns                 ClusterIP      10.43.0.10      <none>                 53/UDP,53/TCP    3h40m   app.kubernetes.io/instance=rke2-coredns,app.kubernetes.io/name=rke2-coredns,k8s-app=kube-dns
kube-system     rke2-ingress-nginx-controller-admission   ClusterIP      10.43.30.244    <none>                 443/TCP          3h38m   app.kubernetes.io/component=controller,app.kubernetes.io/instance=rke2-ingress-nginx,app.kubernetes.io/name=rke2-ingress-nginx
kube-system     rke2-metrics-server                       ClusterIP      10.43.9.76      <none>                 443/TCP          3h38m   app=rke2-metrics-server,release=rke2-metrics-server
kube-system     rke2-snapshot-validation-webhook          ClusterIP      10.43.141.198   <none>                 443/TCP          3h38m   app.kubernetes.io/instance=rke2-snapshot-validation-webhook,app.kubernetes.io/name=rke2-snapshot-validation-webhook
I ended up deleting it with 
kubectl delete svc rke2-ingress-nginx-controller-admission -n kube-system
and 
kubectl delete -A ValidatingWebhookConfiguration rke2-ingress-nginx-admission
I can't figure out how to get it back, but everything seems to work properly now, but let me know if I should've done something better
b
Was the service showing as ready if you did 
kubectl get service -n kube-system
the error shows it was talking to the service IP so the hostPort change shouldn't have caused an error. Might have been worth double checking the port it was listening on with the above command to double check though.
s
Yes it was ready, however when I did 
kubectl get endpoints -n kube-system
, it was the only service without an assigned endpoint for whatever reason, I tried stopping all rke2-server on the nodes and starting them back on again, it still didn't have an assigned endpoint
I did try setting the same thing up locally with single node it worked, but not on our multi-node thing, and I don't really know what was wrong 🥲
hm, I don't know what I did but the controller-admission thing came back again today after I deleted it yesterday, and now it has an endpoint, weird, anyway, thanks for your help!
52 Views
Open in Slack
PreviousNext
Powered by