able-engineer-22050
10/07/2022, 2:44 PMcareful-piano-35019
10/10/2022, 10:21 AMmost-hairdresser-42454
10/10/2022, 10:23 AMbroad-farmer-70498
10/10/2022, 9:56 PMhallowed-energy-68622
10/11/2022, 9:21 AMable-engineer-22050
10/11/2022, 2:19 PMrough-ocean-41843
10/11/2022, 2:29 PMrough-ocean-41843
10/11/2022, 7:56 PMstale-painting-80203
10/12/2022, 5:30 PMancient-air-32350
10/13/2022, 5:41 PMkube-proxy-replacement
to strict on rancher launched rke2 clusters with cilium ?
if yes, could you please tellme how ?
thankshundreds-hairdresser-46043
10/14/2022, 2:00 PMrich-crowd-36987
10/14/2022, 3:40 PMOct 14 15:09:01 k8worker05 rke2: time="2022-10-14T15:09:01Z" level=info msg="Connecting to proxy" url="<wss://10.149.5.62:9345/v1-rke2/connect>"
Oct 14 15:09:01 k8worker05 rke2: time="2022-10-14T15:09:01Z" level=error msg="Failed to connect to proxy" error="x509: certificate is valid for 10.149.4.146, 10.149.4.32, 10.149.4.77, 10.43.0.1, 127.0.0.1, not 10.149.5.62"
Oct 14 15:09:01 k8worker05 rke2: time="2022-10-14T15:09:01Z" level=error msg="Remotedialer proxy error" error="x509: certificate is valid for 10.149.4.146, 10.149.4.32, 10.149.4.77, 10.43.0.1, 127.0.0.1, not 10.149.5.62"
Obviously 10.149.5.62
is the new IP and doesn't match what the cert is advertising. I'm stumped however about how the cert is being generated. The /etc/rancher/rke2/config.yaml
file doesn't have any IP references... There are IPs in /var/lib/rancher/rke2/server/tls/dynamic-cert.json
though these appear to be the result of some process.
Any idea how to regenerate these certs?flat-notebook-92639
10/14/2022, 6:55 PMmagnificent-vr-88571
10/15/2022, 3:13 AMroot@server:/home/ubuntu# crictl pull --creds "AWS:eyJwYXlsb2FkIjoieXRSVW5JMzkwRlVneitXNnpPNnJGOGRqYU9yZ0tRbEFIdkF0aGprMjlNTU1JWWdQd095QlJsQ01FUmRCWFVjZlZNNkEyRTdYS3ByeVRwRjhPNWlneStEdEtmcXdrR2tkMnlwM3RNUnFNNG8zOW1xdUsrSlVOemVWWDFUbGEwR1RqdjkyMmtXMWNsVUZuVnJxOEUzM3VubG9wdm5HbVp0a3o2YVdVSGNzM20reDEvbTl1K2dLZTk1ZnhaTnIrdU43SmRyNlBod0Z1TXBMUnNxUzZoZC9rYy9xMmwxbDJRNXk0Nm9scDNtNG9uc29pdjRid1JBMVpIaEdvMDhSS1lac" <http://1234.dkr.ecr.us-west-2.amazonaws.com/mlflow-run:latest|1234.dkr.ecr.us-west-2.amazonaws.com/mlflow-run:latest>
This ended up in error
WARN[0000] image connect using default endpoints: [unix:///var/run/dockershim.sock unix:///run/containerd/containerd.sock unix:///run/crio/crio.sock]. As the default settings are now deprecated, you should set the endpoint instead.
ERRO[0002] connect endpoint 'unix:///var/run/dockershim.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded
ERRO[0004] connect endpoint 'unix:///run/containerd/containerd.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded
FATA[0006] connect: connect endpoint 'unix:///run/crio/crio.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded
to overcome above error, I have added /etc/crictl.yaml
file with below content.
runtime-endpoint: unix:///run/k3s/containerd/containerd.sock
image-endpoint: unix:///run/k3s/containerd/containerd.sock
timeout: 10
After creating above file, on pod creation images were pulled successfully from AWS ECR.loud-receptionist-98355
10/17/2022, 7:42 AMloud-receptionist-98355
10/17/2022, 2:18 PMalert-grass-67931
10/17/2022, 3:51 PMrich-crowd-36987
10/17/2022, 4:42 PM/var/lib/rancher/rke2
directory and relaunch rke2 server
, it appears to create an entirely new cluster (as the process starts successfully, but kubectl get nodes
only returns itself.)
On the second master node, after removing the dir and relaunching the service, it is just showing this loop in the logs:
Oct 17 16:37:09 <http://k8mst02.espc-nostromo.nos-amc.io|k8mst02.espc-nostromo.nos-amc.io> rke2[32216]: time="2022-10-17T16:37:09Z" level=info msg="Failed to test data store connection: this server has not yet been promoted from learner to voting member"
Oct 17 16:37:10 <http://k8mst02.espc-nostromo.nos-amc.io|k8mst02.espc-nostromo.nos-amc.io> rke2[32216]: time="2022-10-17T16:37:10Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: <https://127.0.0.1:9345/v1-rke2/readyz>: 500 Internal Server Error"
sparse-fireman-14239
10/18/2022, 11:40 AMnice-answer-21943
10/18/2022, 2:32 PMnice-answer-21943
10/18/2022, 2:49 PMnumerous-country-20400
10/18/2022, 8:58 PMforbidden seccomp may not be set
- for example the new cert-manager 1.10 introduces https://artifacthub.io/packages/helm/cert-manager/cert-manager/1.10.0#default-security-contexts a new default security context - thus i cannot install it on my rke2 cluster. Same goes with bitnami-wordpress start 15.2.0 which also introduce RuntimeDefault as their default runtime. Is there anything missing in my rke2 configuration or do i miss the point entirely?millions-australia-75015
10/19/2022, 11:29 AMgentle-petabyte-40055
10/19/2022, 4:24 PMgentle-petabyte-40055
10/19/2022, 4:41 PMcool-pillow-1781
10/19/2022, 7:48 PMcool-pillow-1781
10/19/2022, 7:50 PMgentle-petabyte-40055
10/20/2022, 12:19 AMsparse-fireman-14239
10/20/2022, 8:26 AMsparse-fireman-14239
10/20/2022, 8:09 PM