https://rancher.com/ logo
#rke2
Title
n

narrow-noon-75604

06/27/2022, 2:30 PM
Hi, I am trying to install RKE2 on a centos stream 8 with 3 server nodes and 3 agent nodes...The installation is successful on the first server node but the installation fails on second and third server nodes due to the configuration mismatch error.
Copy code
Jun 27 10:17:43 <http://rke2-master2.xxx.xxx.xxx.43.nip.io|rke2-master2.xxx.xxx.xxx.43.nip.io> rke2[38318]: time="2022-06-27T10:17:43-04:00" level=fatal msg="starting kubernetes: preparing server: failed to validate server configuration: critical configuration value mismatch"
First Server configuration:
Copy code
# BEGIN Adding RKE2 configuration
write-kubeconfig-mode: "0644"
tls-san:
- "<http://rke2-master1.xxx.xxx.xxx.42.nip.io|rke2-master1.xxx.xxx.xxx.42.nip.io>"
node-label:
- "nodetype=master"
node-ip: "xxx.xxx.xxx.42,xxxx:xxx:x:xxx:xxx:xxxx:xxxx:aae"
cluster-cidr: "10.42.0.0/16,2001:cafe:42:0::/56"
service-cidr: "10.43.0.0/16,2001:cafe:42:1::/112"
cluster-dns: "10.43.0.10"
cluster-domain: "<http://rke2-master1.xxx.xxx.xxx.42.nip.io|rke2-master1.xxx.xxx.xxx.42.nip.io>"
cni:
- calico
disable:
- rke2-canal
- rke2-kube-proxy
# END Adding RKE2 configuration
Second Server Configuration:
Copy code
# BEGIN Adding RKE2 configuration
server: "<https://rke2-master1.xxx.xxx.xxx.42.nip.io:9345>"
token: "K10d463a80c8c1323f30fa6d97fcf91992454a43dc5c544f1c9a0de706b733b51ee::server:f6fd26cafff902300ba021b29b11eddc"
tls-san:
- "<http://rke2-master1.xxx.xxx.xxx.42.nip.io|rke2-master1.xxx.xxx.xxx.42.nip.io>"
node-ip: "xxx.xxx.xxx.43,xxxx:xxx:x:xxx:xxx:xxxx:xxxx:5245"
cni:
- calico
disable:
- rke2-canal
- rke2-kube-proxy
# END Adding RKE2 configuration
There is no firewalld or iptables running on any of the nodes...Please help me in finding the issue here
n

nutritious-tomato-14686

06/27/2022, 4:33 PM
Your Cluster DNS and Cluster Domain must be the same across servers
n

narrow-noon-75604

06/27/2022, 4:35 PM
can you please make it some more clear...because I don't see the cluster-dns & cluster-domain fields in the secondary server node configuration
n

nutritious-tomato-14686

06/27/2022, 4:36 PM
Thats the point, you need to add those 2 to the secondary server configs.
n

narrow-noon-75604

06/27/2022, 4:37 PM
Thanks for the details...will try and come back 🙂
Added the cluster-dns & cluster-domain fields in the secondary server node and restarted the rke2-server service....still facing the same issue,
Copy code
# BEGIN Adding RKE2 configuration
server: "<https://rke2-master1.xxx.xxx.xxx.42.nip.io:9345>"
token: "K10d463a80c8c1323f30fa6d97fcf91992454a43dc5c544f1c9a0de706b733b51ee::server:f6fd26cafff902300ba021b29b11eddc"
tls-san:
  - "<http://rke2-master1.xxx.xxx.xxx.42.nip.io|rke2-master1.xxx.xxx.xxx.42.nip.io>"
node-ip: "xxx.xxx.xxx.43,xxxx:xxx:x:xxx:xxx:xxxx:xxxx:5245"
cluster-dns: "10.43.0.10"
cluster-domain: "<http://rke2-master1.xxx.xxx.xxx.42.nip.io|rke2-master1.xxx.xxx.xxx.42.nip.io>"
cni:
  - calico
disable:
  - rke2-canal
  - rke2-kube-proxy
# END Adding RKE2 configuration
n

nutritious-tomato-14686

06/27/2022, 4:47 PM
if you add
debug: true
to the config.yaml, it will tell you in the journalctl logs which arguments are mistmatched
Add that to the secondary server configs
👍 1
n

narrow-noon-75604

06/27/2022, 5:01 PM
I got the debug logs, but could not able to figure out the mismatch,
Copy code
-- Unit rke2-server.service has begun starting up.
Jun 27 12:49:40 <http://rke2-master2.xxx.xxx.xxx.43.nip.io|rke2-master2.xxx.xxx.xxx.43.nip.io> sh[77742]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Jun 27 12:49:40 <http://rke2-master2.xxx.xxx.xxx.43.nip.io|rke2-master2.xxx.xxx.xxx.43.nip.io> sh[77743]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
Jun 27 12:49:40 <http://rke2-master2.xxx.xxx.xxx.43.nip.io|rke2-master2.xxx.xxx.xxx.43.nip.io> rke2[77748]: time="2022-06-27T12:49:40-04:00" level=warning msg="not running in CIS mode"
Jun 27 12:49:40 <http://rke2-master2.xxx.xxx.xxx.43.nip.io|rke2-master2.xxx.xxx.xxx.43.nip.io> rke2[77748]: time="2022-06-27T12:49:40-04:00" level=info msg="Starting rke2 v1.23.6+rke2r2 (40d712e5081ac87e30e8f328f738130acf2c31f8)"
Jun 27 12:49:40 <http://rke2-master2.xxx.xxx.xxx.43.nip.io|rke2-master2.xxx.xxx.xxx.43.nip.io> rke2[77748]: time="2022-06-27T12:49:40-04:00" level=info msg="Managed etcd cluster not yet initialized"
Jun 27 12:49:40 <http://rke2-master2.xxx.xxx.xxx.43.nip.io|rke2-master2.xxx.xxx.xxx.43.nip.io> rke2[77748]: time="2022-06-27T12:49:40-04:00" level=debug msg="This is the server CriticalControlArgs: config.CriticalControlArgs{ClusterDNSs:[]net.IP{net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xa, 0x2b, 0x0, 0xa}}, ClusterIPRanges:[]*net.IPNet{(*net.IPNet)(0xc001ebf170), (*net.IPNet)(0xc001ebf1a0)}, ClusterDNS:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xa, 0x2b, 0x0, 0xa}, ClusterDomain:\"<http://rke2-master1.xxx.xxx.xxx.42.nip.io|rke2-master1.xxx.xxx.xxx.42.nip.io>\", ClusterIPRange:(*net.IPNet)(0xc001ebf1d0), DisableCCM:false, DisableHelmController:false, DisableNPC:true, DisableServiceLB:false, FlannelBackend:\"none\", FlannelIPv6Masq:false, NoCoreDNS:false, ServiceIPRange:(*net.IPNet)(0xc001ebf200), ServiceIPRanges:[]*net.IPNet{(*net.IPNet)(0xc001ebf230), (*net.IPNet)(0xc001ebf260)}}"
Jun 27 12:49:40 <http://rke2-master2.xxx.xxx.xxx.43.nip.io|rke2-master2.xxx.xxx.xxx.43.nip.io> rke2[77748]: time="2022-06-27T12:49:40-04:00" level=debug msg="This is the local CriticalControlArgs: config.CriticalControlArgs{ClusterDNSs:[]net.IP{net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xa, 0x2b, 0x0, 0xa}}, ClusterIPRanges:[]*net.IPNet{(*net.IPNet)(0xc00113ad20)}, ClusterDNS:net.IP{0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0xff, 0xff, 0xa, 0x2b, 0x0, 0xa}, ClusterDomain:\"<http://rke2-master1.xxx.xxx.xxx.42.nip.io|rke2-master1.xxx.xxx.xxx.42.nip.io>\", ClusterIPRange:(*net.IPNet)(0xc00113ad20), DisableCCM:false, DisableHelmController:false, DisableNPC:true, DisableServiceLB:false, FlannelBackend:\"none\", FlannelIPv6Masq:false, NoCoreDNS:false, ServiceIPRange:(*net.IPNet)(0xc00113ad50), ServiceIPRanges:[]*net.IPNet{(*net.IPNet)(0xc00113ad50)}}"
Jun 27 12:49:40 <http://rke2-master2.xxx.xxx.xxx.43.nip.io|rke2-master2.xxx.xxx.xxx.43.nip.io> rke2[77748]: time="2022-06-27T12:49:40-04:00" level=fatal msg="starting kubernetes: preparing server: failed to validate server configuration: critical configuration value mismatch"
n

nutritious-tomato-14686

06/27/2022, 5:06 PM
Copy code
cluster-cidr:
Is missing from the secondary servers as well
Basically anything
cluster*
in the args needs to be match... its a cluster wide configuration value
👍 1
n

narrow-noon-75604

06/27/2022, 5:07 PM
can you please share me any reference links for secondary server node configuration
n

nutritious-tomato-14686

06/27/2022, 5:20 PM
I don't really have any references for you, its all depends on your configuration. At the bare minimum, secondary servers must match the following arguments if they are present in the primary server (this is just converting the go code in the github link above into a yaml arg):
Copy code
cluster-cidr
cluster-dns
cluster-domain
disable-cloud-controller
egress-selector-mode
service-cidr
n

narrow-noon-75604

06/27/2022, 5:21 PM
Thanks much @nutritious-tomato-14686 the other server nodes are now added to the primary server node
🙌 1
465 Views