https://rancher.com/ logo
Title
c

clean-airplane-85370

08/04/2022, 2:24 PM
hi, I just recreated my rke2 cluster from v1.23.8 to v1.23.9+rke2r1, and seeing all my ingress stop working. It is getting connection refused. Anyone know what is happening? I saw one of the changes is nginx-ingress updated to 4.1.x that have some hostnetwork changes.... I was not able to find any doc on any config change needed on the new version. How can I fix this?
note that I am running nebula VPN on the nodes, not sure it is related to the issue.
The server and worker nodes are rocky linux 8.5 VM running on a fedora 36 hosts
I checked the nginx-ingress-controller logs but no error I think
All the services behind the ingress works, tested with port forward to it
g

gray-lawyer-73831

08/04/2022, 4:18 PM
Are your nodes hardened? i.e you have set the
profile
flag in your config?
c

clean-airplane-85370

08/04/2022, 4:34 PM
no. I did not set profile in the config
BTW, I am running cilium
my config
write-kubeconfig-mode: "0600"
selinux: false
tls-san:
  - rocky-1.somehost
  - rocky-1.nebula.somehost
cni:
  - cilium
disable:
  - rke2-canal
  - rke2-kube-proxy
  - rke2-metrics-server
g

gray-lawyer-73831

08/04/2022, 4:57 PM
Hmm.. I’m not sure from the top of my head what might be causing this based on the information provided. If you submit an issue with more details/logs then someone is likely to look into it
c

clean-airplane-85370

08/04/2022, 4:58 PM
k, will do
I am wondering how can isolate the issue here in 1.23.9. I am installing rke2 with RPM. How can I pin to 1.23.8 and retry. I want to make sure it is the upgrade to 1.23.9 causing this
g

gray-lawyer-73831

08/04/2022, 5:01 PM
curl -sfL <https://get.rke2.io> | sudo INSTALL_RKE2_VERSION=v1.23.8+rke2r1 sh -
c

clean-airplane-85370

08/04/2022, 5:02 PM
is there a way to do it with RPM?
g

gray-lawyer-73831

08/04/2022, 5:02 PM
Depending on the OS, that’ll likely do RPM by default. You can specify RPM specifically though:
curl -sfL <https://get.rke2.io> | sudo INSTALL_RKE2_VERSION=v1.23.8+rke2r1 INSTALL_RKE2_METHOD=rpm sh -
c

clean-airplane-85370

08/04/2022, 5:02 PM
I have this yum conf
[rancher-rke2-common-latest]
name=Rancher RKE2 Common Latest
baseurl=<https://rpm.rancher.io/rke2/latest/common/centos/8/noarch>
enabled=0
gpgcheck=1
gpgkey=<https://rpm.rancher.io/public.key>

[rancher-rke2-1-23-latest]
name=Rancher RKE2 1.23 Latest
baseurl=<https://rpm.rancher.io/rke2/latest/1.23/centos/8/x86_64>
enabled=0
gpgcheck=1
gpgkey=<https://rpm.rancher.io/public.key>
g

gray-lawyer-73831

08/04/2022, 5:03 PM
ooohhh I see
c

clean-airplane-85370

08/04/2022, 5:03 PM
I have not found much info on that....
so that
INSTALL_RKE2_METHOD=rpm
will install the yum repo conf? I was a bit confusedon that too...
g

gray-lawyer-73831

08/04/2022, 5:05 PM
Yep it’ll do it all for you!
c

clean-airplane-85370

08/04/2022, 5:06 PM
cool. I will try that out 1st to try to get a 1.23.8 going to see the issue goes away (so I know it is something else I messed up)
👍 1
@gray-lawyer-73831 I just rollback to 1.23.8 and ingress is working again. So, there is something funny causing ingress to failed in 1.23.9. I will filed an issue ticket. Thanks for the tips on how to pin version