I’m trying to do a fresh rke2 install on RHEL8.8 (the same way I did it on RHEL8.7) and it seems like something has changed.

systemctl start rke2-server

eventually times out and fails. It seems like it’s waiting for etcd startup which never happens. I’m guessing something changed in the rhel release, but I’m failing to grok what the issue is. Anyone seen this?

which RKE2 version? looks like RHEL 8.8 isn't on support matrix yet... https://www.suse.com/suse-rke2/support-matrix/all-supported-versions/rke2-v1-26/

RKE2 v1.25.9~rke2r1

Yeah this hasn’t been validated as working yet, but I don’t see any reason why it shouldn’t. Couple of things to check: https://docs.rke2.io/known_issues#networkmanager Also check container-selinux version as there have been multiple issues around this recently with a version bump there that caused rke2-selinux to fail:

rpm -qi container-selinux

Yeah, for some reason I didn’t even consider that a minor update would not work. 🤦 Confirming that

container-selinux

went from

v2.189.0-1

on a working RHEL8.7 system to

v2.205.0-2

on non-working RHEL8.8 system.

yeah container-selinux is the issue! If you install v2.189 then it should all work fine! We’re in the process of making fixes for that on our side, but for now you should be able to just force it before installing rke2 with something like:

yum install container-selinux-2:2.189.0-1.module+el8.7.0+17824+66a0202b

ahh. okay. I will give that a whirl. Thanks so much!

I am running into issue deploying also though I am only trying to deploy to a simple virtualbox vm. I removed the package described above and downgraded it [2.189.0](https://centos.pkgs.org/8-stream/centos-appstream-aarch64/container-selinux-2.189.0-1.module_el8.7.0+1217+ea57d1f1.noarch.rpm.html) and tried again after rebooting, still no luck. I next attempted simply disabling selinux and rebooting and still no dice. I can provide logs if anyone is interested, or is willing to help, it would be much appreciated.

have you looked at

journalctl -u rke2-server

to see it contains any pointers?

Mostly the following:

T15:30:14-06:00" level=info msg="Waiting for API server to become available"
T15:30:14-06:00" level=info msg="Tunnel server egress proxy waiting for runtime core to become available"
T15:30:16-06:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: <https://192.168.56.3:9345/v1-rke2/readyz>: 500 Internal Server Error"
T15:30:19-06:00" level=info msg="Tunnel server egress proxy waiting for runtime core to become available"
T15:30:21-06:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: <https://192.168.56.3:9345/v1-rke2/readyz>: 500 Internal Server Error"
T15:30:24-06:00" level=info msg="Container for etcd not found (no matching container found), retrying"
T15:30:24-06:00" level=info msg="Tunnel server egress proxy waiting for runtime core to become available"
T15:30:26-06:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: <https://192.168.56.3:9345/v1-rke2/readyz>: 500 Internal Server Error"
"ts":"2023-05-25T15:30:29.464-0600","logger":"etcd-client","caller":"v3@v3.5.4-k3s1/retry_interceptor.go:62","msg":"retrying of unary invoker failed","target":"<etcd-endpoints://0xc00079dc00/127.0.0.1:2379>","attempt":0,"error":"rpc error: code = Deadline>
"ts":"2023-05-25T15:30:29.465-0600","logger":"etcd-client","caller":"v3@v3.5.4-k3s1/client.go:210","msg":"Auto sync endpoints failed.","error":"context deadline exceeded"}
T15:30:29-06:00" level=info msg="Tunnel server egress proxy waiting for runtime core to become available"
T15:30:31-06:00" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: <https://192.168.56.3:9345/v1-rke2/readyz>: 500 Internal Server Error"
T15:30:34-06:00" level=info msg="Tunnel server egress proxy waiting for runtime core to become available"

Nothing very specific, as far as I can telll RKE2 isnt even starting any containers checking the /var/log/containers log directory yields an empty folder.

Those log entries look like what I was seeing. I will add that when I just downgraded the container-selinux package, it didn’t seem to solve the issue. Was testing with an ec2 instance, so I just whacked it an made a new one and installed the lower version from the start. Then it worked as I expected….

So perhaps... Try and run: rke2-uninstall Make sure these directories are cleaned up after uninstall (they should but should confirm) /etc/rancher/ /var/lib/rancher /usr/local/lib/systemd/system/ ( make sure rke2-server service scripts are gone) reboot if container-selinux got removed then reinstall then reinstall rke2 and see if this works?