hey folks I have a quick question for a newbie I have setup rancher-users #general

Title

busy-judge-4614

03/09/2023, 8:58 PM

hey folks, I have a quick question for a newbie. I have setup a 3 node k3s cluster on bare metal, with Oracle 9.1. I have one configured as master and the other two as workers (I beleive). I am only doing this for experience and tinkering with things at the moment. (I’m not too bothered about HA at this stage) I’d like to install Rancher to access the cluster. What is my best or simplest way to do it? I have another server on the network I use for other docker stuff, RHEL 9 and Podman etc. So I could use this? Or am I better installing on one of the nodes on the cluster? Thanks.!

big-hydrogen-97240

03/09/2023, 9:00 PM

It sounds like you want to have Rancher manage this cluster. Is that correct?

busy-judge-4614

03/09/2023, 9:00 PM

yes ideally

big-hydrogen-97240

03/09/2023, 9:00 PM

Then you would create another cluster, install Rancher on that cluster, and import this cluster into that Rancher instance.

busy-judge-4614

03/09/2023, 9:02 PM

ok, so I could do that with docker on my other RHEL box I think? That docker container would then be a single node cluster with just Rancher on? Am I barking up the right tree?

docker run -d --restart=unless-stopped \
  -p 80:80 -p 443:443 \
  --privileged \
  rancher/rancher:latest

so I should be able to do this and then import my 3 node cluster?

ambitious-plastic-3551

03/09/2023, 9:06 PM

when you login to rancher you will need to add the cluster, you need hostname and token to connect to it

Cluster Management > Import Existing

big-hydrogen-97240

03/09/2023, 9:07 PM

You can use the Docker install of Rancher to evaluate, yes. Using the Docker install isn't officially supported, however.

ambitious-plastic-3551

03/09/2023, 9:07 PM

select Generic

busy-judge-4614

03/09/2023, 9:13 PM

thanks guys, I will pop off and give that a go.

one other thing I just read. It seems its possible to add the single node cluster (which rancher is running on) to rancher itself? Is there any reason to do that? I suppose I’m only concerned with using rancher to control the bare metal cluster

ambitious-plastic-3551

03/09/2023, 9:18 PM

also possible

big-hydrogen-97240

03/09/2023, 9:18 PM

That happens automatically.

ambitious-plastic-3551

03/09/2023, 9:18 PM

I have separate cluster for rancher, because of nginx dependency among other

big-hydrogen-97240

03/09/2023, 9:18 PM

It's called the "local" cluster.

creamy-pencil-82913

03/09/2023, 10:04 PM

note that given your

kubectl get nodes

output that you shared outside this thread, you will not be able to import your K3s cluster into Rancher at the moment

Rancher won’t support Kubernetes 1.25 until 2.7.2 which isn’t out yet.

you should use K3s 1.24 until then

busy-judge-4614

03/09/2023, 10:07 PM

oh! thanks for spotting that!

hmm, so that would require me re-installing k3s on the nodes I assume

creamy-pencil-82913

03/09/2023, 10:10 PM

yeah, you can’t downgrade kubernetes, so you should uninstall and reinstall

busy-judge-4614

03/09/2023, 10:24 PM

I0309 22:23:06.018270      64 network_policy_controller.go:162] Starting network policy controller
F0309 22:23:06.019120      64 network_policy_controller.go:380] failed to run iptables command to create KUBE-ROUTER-INPUT chain due to running [/usr/bin/iptables -t filter -S KUBE-ROUTER-INPUT 1 --wait]: exit status 3: iptables v1.8.6 (legacy): can't initialize iptables table `filter': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.
panic: F0309 22:23:06.019120      64 network_policy_controller.go:380] failed to run iptables command to create KUBE-ROUTER-INPUT chain due to running [/usr/bin/iptables -t filter -S KUBE-ROUTER-INPUT 1 --wait]: exit status 3: iptables v1.8.6 (legacy): can't initialize iptables table `filter': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

hmm, I’m using RHEL 9 to try Rancher with.

which it appears may use

nft

for modifying iptables rules. Hmm, more reading needed I think

creamy-pencil-82913

03/09/2023, 10:37 PM

yeah you still need the legacy iptables kernel modules available. k3s should load them for you on startup

busy-judge-4614

03/09/2023, 10:37 PM

Oh geez, that looks like a whole world of pain disabling iptables and such on RHEL 9.1. https://serverfault.com/questions/1119704/nftables-firewall-configuration-on-rocky-9-1 I wasn’t planning on making major changes to this VM as it hosts a load of other stuff. I suppose I could spin up another VM just for Rancher. Seems a little excessive

creamy-pencil-82913

03/09/2023, 10:38 PM

do you not have the iptables-nft package installed on your node?

as per the logs, k3s is using legacy iptables. if you have the nft stuff installed on your node instead it should use that.

busy-judge-4614

03/09/2023, 10:40 PM

hmm. let me check. So the 2 can be run side by side?

yes it appears I do

[root@docker1 Rancher]# ipt
iptables                    iptables-nft-restore        iptables-restore            iptables-save               iptc
iptables-nft                iptables-nft-save           iptables-restore-translate  iptables-translate          iptunnel
[root@docker1 Rancher]#

creamy-pencil-82913

03/09/2023, 10:42 PM

if you run iptables, does that get you nft, or legacy?

busy-judge-4614

03/09/2023, 10:43 PM

[root@docker1 Rancher]# iptables
iptables v1.8.8 (nf_tables): no command specified
Try `iptables -h' or 'iptables --help' for more information.
[root@docker1 Rancher]# which iptables
/sbin/iptables
[root@docker1 Rancher]#

creamy-pencil-82913

03/09/2023, 10:43 PM

hmm interesting. Those logs you posted, were those from k3s on the host, or k3s in docker?

if it’s at the host level, it should prefer the host iptables binaries

but it’s using the bundled iptables 1.8.6 instead

busy-judge-4614

03/09/2023, 10:45 PM

docker run -d --restart=unless-stopped -v /root/docker/Rancher:/var/lib/rancher  -p 192.168.1.3:8686:80 -p 192.168.1.3:8443:443   --privileged  --name Rancher rancher/rancher:latest

thats how I was starting it, the log output was from /root/docker/rancher/k3s.log

creamy-pencil-82913

03/09/2023, 10:47 PM

ah yeah that would be why. when you run it in docker it has no access to the host iptables and no ability to load kernel modules, so you need to be sure that the legacy iptables modules are loaded.

you don’t need to install any packages or switch the whole host over to legacy mode, just load the modules

busy-judge-4614

03/09/2023, 10:57 PM

ok thanks! I’ll check that out. Appreciate this

hmm, actually which module(s) do I need to install?

creamy-pencil-82913

03/09/2023, 11:12 PM

grep for Module in that log file (case insensitive perhaps)

I think it should tell you

I see

INFO[0000] Module nf_conntrack was already loaded
INFO[0000] Module br_netfilter was already loaded
INFO[0000] Module iptable_nat was already loaded
INFO[0000] Module iptable_filter was already loaded

busy-judge-4614

03/09/2023, 11:17 PM

yes, thanks, it was

[root@docker1 Rancher]# modprobe iptable_nat
[root@docker1 Rancher]# modprobe br_netfilter

excellent

maybe nf_conntrack and iptable_filter too then

I imagine its more than a few days wait for 2.7.2 ?

creamy-pencil-82913

03/09/2023, 11:25 PM

Current target is “end of the month”

busy-judge-4614

03/09/2023, 11:35 PM

ok thanks

As I’ve not really done much with the small cluster, its probably easier to start again with it and install the earlier k3s

All good , thanks.

And finally, just for completeness, 00:35 here in the UK, I removed k3s and installed 1.24 on the cluster and have it imported in to Rancher 🙂 Time to hit the hay!

13 Views

#general Join Slack