adamant-kite-43734
05/11/2023, 5:46 PMdry-dawn-97788
05/11/2023, 5:48 PMcalico
and canal
as CNIs with their default settings.
Things are running on Proxmox VMs and Ubuntu 22.04. Tried v1.26.4+rke2r1
and v1.25.9+rke2r1
creamy-pencil-82913
05/11/2023, 6:00 PMcreamy-pencil-82913
05/11/2023, 6:00 PMcreamy-pencil-82913
05/11/2023, 6:02 PMcreamy-pencil-82913
05/11/2023, 6:03 PMcreamy-pencil-82913
05/11/2023, 6:04 PMdry-dawn-97788
05/11/2023, 6:04 PMcreamy-pencil-82913
05/11/2023, 6:04 PMdry-dawn-97788
05/11/2023, 6:04 PMdry-dawn-97788
05/11/2023, 6:04 PM$ /usr/local/bin/rke2 --version
rke2 version v1.25.9+rke2r1 (842d05e64bcbf78552f1db0b32700b8faea403a0)
go version go1.19.8 X:boringcrypto
dry-dawn-97788
05/11/2023, 6:10 PMError from server (InternalError): error when creating "cert-manager/k8s/letsencrypt-prod-issuer.yaml": Internal error occurred: failed calling webhook "<http://webhook.cert-manager.io|webhook.cert-manager.io>": failed to call webhook: Post "<https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s>": context deadline exceeded
Error from server (InternalError): error when creating "cert-manager/k8s/letsencrypt-staging-issuer.yaml": Internal error occurred: failed calling webhook "<http://webhook.cert-manager.io|webhook.cert-manager.io>": failed to call webhook: Post "<https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s>": context deadline exceeded
creamy-pencil-82913
05/11/2023, 6:16 PMdry-dawn-97788
05/11/2023, 6:16 PMdry-dawn-97788
05/11/2023, 6:18 PMcreamy-pencil-82913
05/11/2023, 6:36 PMcreamy-pencil-82913
05/11/2023, 6:36 PMcreamy-pencil-82913
05/11/2023, 6:37 PMkubectl get node -o yaml | grep node-args
?creamy-pencil-82913
05/11/2023, 6:42 PMbrandond@dev01:~$ kubectl get validatingwebhookconfigurations cert-manager-webhook -o yaml | grep -A10 webhooks:
webhooks:
- admissionReviewVersions:
- v1
clientConfig:
caBundle: XXX
service:
name: cert-manager-webhook
namespace: cert-manager
path: /validate
port: 443
failurePolicy: Fail
dry-dawn-97788
05/11/2023, 6:49 PMnode-args
command:
kubectl get node -o yaml | grep node-args
<http://rke2.io/node-args|rke2.io/node-args>: '["server","--token","********","--data-dir","/var/lib/rancher/rke2","--cni","canal","--tls-san","cluster.local","--tls-san","<http://v1111-dcs-master-1.dcs.tickup.net|v1111-dcs-master-1.dcs.tickup.net>","--tls-san","<http://k8s-api.dcs.tickup.net|k8s-api.dcs.tickup.net>","--tls-san","<http://v1111-dcs-master-1.dcs.tickup.net|v1111-dcs-master-1.dcs.tickup.net>","--tls-san","10.101.0.11","--tls-san","<http://v1112-dcs-master-2.dcs.tickup.net|v1112-dcs-master-2.dcs.tickup.net>","--tls-san","10.101.0.12","--tls-san","<http://v1113-dcs-master-3.dcs.tickup.net|v1113-dcs-master-3.dcs.tickup.net>","--tls-san","10.101.0.13","--tls-san","<http://v1114-dcs-master-4.dcs.tickup.net|v1114-dcs-master-4.dcs.tickup.net>","--tls-san","10.101.0.14","--tls-san","<http://v1115-dcs-master-5.dcs.tickup.net|v1115-dcs-master-5.dcs.tickup.net>","--tls-san","10.101.0.15","--snapshotter","overlayfs","--node-name","<http://v1111-dcs-master-1.dcs.tickup.net|v1111-dcs-master-1.dcs.tickup.net>"]'
<http://rke2.io/node-args|rke2.io/node-args>: '["server","--server","<https://v1111-dcs-master-1.dcs.tickup.net:9345>","--token","********","--data-dir","/var/lib/rancher/rke2","--cni","canal","--tls-san","cluster.local","--tls-san","<http://v1111-dcs-master-1.dcs.tickup.net|v1111-dcs-master-1.dcs.tickup.net>","--tls-san","<http://k8s-api.dcs.tickup.net|k8s-api.dcs.tickup.net>","--tls-san","<http://v1111-dcs-master-1.dcs.tickup.net|v1111-dcs-master-1.dcs.tickup.net>","--tls-san","10.101.0.11","--tls-san","<http://v1112-dcs-master-2.dcs.tickup.net|v1112-dcs-master-2.dcs.tickup.net>","--tls-san","10.101.0.12","--tls-san","<http://v1113-dcs-master-3.dcs.tickup.net|v1113-dcs-master-3.dcs.tickup.net>","--tls-san","10.101.0.13","--tls-san","<http://v1114-dcs-master-4.dcs.tickup.net|v1114-dcs-master-4.dcs.tickup.net>","--tls-san","10.101.0.14","--tls-san","<http://v1115-dcs-master-5.dcs.tickup.net|v1115-dcs-master-5.dcs.tickup.net>","--tls-san","10.101.0.15","--snapshotter","overlayfs","--node-name","<http://v1112-dcs-master-2.dcs.tickup.net|v1112-dcs-master-2.dcs.tickup.net>"]'
<http://rke2.io/node-args|rke2.io/node-args>: '["server","--server","<https://v1111-dcs-master-1.dcs.tickup.net:9345>","--token","********","--data-dir","/var/lib/rancher/rke2","--cni","canal","--tls-san","cluster.local","--tls-san","<http://v1111-dcs-master-1.dcs.tickup.net|v1111-dcs-master-1.dcs.tickup.net>","--tls-san","<http://k8s-api.dcs.tickup.net|k8s-api.dcs.tickup.net>","--tls-san","<http://v1111-dcs-master-1.dcs.tickup.net|v1111-dcs-master-1.dcs.tickup.net>","--tls-san","10.101.0.11","--tls-san","<http://v1112-dcs-master-2.dcs.tickup.net|v1112-dcs-master-2.dcs.tickup.net>","--tls-san","10.101.0.12","--tls-san","<http://v1113-dcs-master-3.dcs.tickup.net|v1113-dcs-master-3.dcs.tickup.net>","--tls-san","10.101.0.13","--tls-san","<http://v1114-dcs-master-4.dcs.tickup.net|v1114-dcs-master-4.dcs.tickup.net>","--tls-san","10.101.0.14","--tls-san","<http://v1115-dcs-master-5.dcs.tickup.net|v1115-dcs-master-5.dcs.tickup.net>","--tls-san","10.101.0.15","--snapshotter","overlayfs","--node-name","<http://v1113-dcs-master-3.dcs.tickup.net|v1113-dcs-master-3.dcs.tickup.net>"]'
<http://rke2.io/node-args|rke2.io/node-args>: '["server","--server","<https://v1111-dcs-master-1.dcs.tickup.net:9345>","--token","********","--data-dir","/var/lib/rancher/rke2","--cni","canal","--tls-san","cluster.local","--tls-san","<http://v1111-dcs-master-1.dcs.tickup.net|v1111-dcs-master-1.dcs.tickup.net>","--tls-san","<http://k8s-api.dcs.tickup.net|k8s-api.dcs.tickup.net>","--tls-san","<http://v1111-dcs-master-1.dcs.tickup.net|v1111-dcs-master-1.dcs.tickup.net>","--tls-san","10.101.0.11","--tls-san","<http://v1112-dcs-master-2.dcs.tickup.net|v1112-dcs-master-2.dcs.tickup.net>","--tls-san","10.101.0.12","--tls-san","<http://v1113-dcs-master-3.dcs.tickup.net|v1113-dcs-master-3.dcs.tickup.net>","--tls-san","10.101.0.13","--tls-san","<http://v1114-dcs-master-4.dcs.tickup.net|v1114-dcs-master-4.dcs.tickup.net>","--tls-san","10.101.0.14","--tls-san","<http://v1115-dcs-master-5.dcs.tickup.net|v1115-dcs-master-5.dcs.tickup.net>","--tls-san","10.101.0.15","--snapshotter","overlayfs","--node-name","<http://v1114-dcs-master-4.dcs.tickup.net|v1114-dcs-master-4.dcs.tickup.net>"]'
<http://rke2.io/node-args|rke2.io/node-args>: '["server","--server","<https://v1111-dcs-master-1.dcs.tickup.net:9345>","--token","********","--data-dir","/var/lib/rancher/rke2","--cni","canal","--tls-san","cluster.local","--tls-san","<http://v1111-dcs-master-1.dcs.tickup.net|v1111-dcs-master-1.dcs.tickup.net>","--tls-san","<http://k8s-api.dcs.tickup.net|k8s-api.dcs.tickup.net>","--tls-san","<http://v1111-dcs-master-1.dcs.tickup.net|v1111-dcs-master-1.dcs.tickup.net>","--tls-san","10.101.0.11","--tls-san","<http://v1112-dcs-master-2.dcs.tickup.net|v1112-dcs-master-2.dcs.tickup.net>","--tls-san","10.101.0.12","--tls-san","<http://v1113-dcs-master-3.dcs.tickup.net|v1113-dcs-master-3.dcs.tickup.net>","--tls-san","10.101.0.13","--tls-san","<http://v1114-dcs-master-4.dcs.tickup.net|v1114-dcs-master-4.dcs.tickup.net>","--tls-san","10.101.0.14","--tls-san","<http://v1115-dcs-master-5.dcs.tickup.net|v1115-dcs-master-5.dcs.tickup.net>","--tls-san","10.101.0.15","--snapshotter","overlayfs","--node-name","<http://v1115-dcs-master-5.dcs.tickup.net|v1115-dcs-master-5.dcs.tickup.net>"]'
<http://rke2.io/node-args|rke2.io/node-args>: '["agent","--server","<https://v1111-dcs-master-1.dcs.tickup.net:9345>","--token","********","--data-dir","/var/lib/rancher/rke2","--snapshotter","overlayfs","--node-name","<http://v1121-dcs-worker-1.dcs.tickup.net|v1121-dcs-worker-1.dcs.tickup.net>"]'
<http://rke2.io/node-args|rke2.io/node-args>: '["agent","--server","<https://v1111-dcs-master-1.dcs.tickup.net:9345>","--token","********","--data-dir","/var/lib/rancher/rke2","--snapshotter","overlayfs","--node-name","<http://v1122-dcs-worker-2.dcs.tickup.net|v1122-dcs-worker-2.dcs.tickup.net>"]'
<http://rke2.io/node-args|rke2.io/node-args>: '["agent","--server","<https://v1111-dcs-master-1.dcs.tickup.net:9345>","--token","********","--data-dir","/var/lib/rancher/rke2","--snapshotter","overlayfs","--node-name","<http://v1123-dcs-worker-3.dcs.tickup.net|v1123-dcs-worker-3.dcs.tickup.net>"]'
<http://rke2.io/node-args|rke2.io/node-args>: '["agent","--server","<https://v1111-dcs-master-1.dcs.tickup.net:9345>","--token","********","--data-dir","/var/lib/rancher/rke2","--snapshotter","overlayfs","--node-name","<http://v1124-dcs-worker-4.dcs.tickup.net|v1124-dcs-worker-4.dcs.tickup.net>"]'
<http://rke2.io/node-args|rke2.io/node-args>: '["agent","--server","<https://v1111-dcs-master-1.dcs.tickup.net:9345>","--token","********","--data-dir","/var/lib/rancher/rke2","--snapshotter","overlayfs","--node-name","<http://v1125-dcs-worker-5.dcs.tickup.net|v1125-dcs-worker-5.dcs.tickup.net>"]'
creamy-pencil-82913
05/11/2023, 6:50 PMdry-dawn-97788
05/11/2023, 6:51 PMcreamy-pencil-82913
05/11/2023, 6:51 PMcreamy-pencil-82913
05/11/2023, 6:51 PMdry-dawn-97788
05/11/2023, 6:52 PMdry-dawn-97788
05/11/2023, 6:52 PMcreamy-pencil-82913
05/11/2023, 6:53 PM--egress-selector-mode=cluster
on the servers, just to see if that helps?creamy-pencil-82913
05/11/2023, 6:53 PMdry-dawn-97788
05/11/2023, 6:54 PMcreamy-pencil-82913
05/11/2023, 6:54 PMdry-dawn-97788
05/11/2023, 6:55 PMdry-dawn-97788
05/11/2023, 7:14 PM--egress-selector-mode=cluster
did not seem to help.creamy-pencil-82913
05/11/2023, 7:16 PMdry-dawn-97788
05/11/2023, 7:41 PM