Hi All Could anyone help me debug MetalLB and RKE2 I believe rancher-users #general

Title

proud-salesmen-12221

09/19/2022, 4:30 PM

Hi All, Could anyone help me debug MetalLB and RKE2? I believe I have metallb installed and configured to hand out IPs in the 192.168.1-240-245 range, but I'm not able to hit the external ip from outside the cluster so I think I'm missing a configuration somewhere. Setup: • I'm using Ubuntu 20.04 as my host and I've deployed 3 RockyLinux8 VMs using Vagrant. One VM is my server, the other two are agents. I'm using Cilium as the CNI. After installing RKE2, kubectl get nodes and pods look good- no obvious failures or errors • I deployed MetalLB using the manifest and configured its IP pool to give out in the range of 192.168.1.240-245 with L2Adertisement • I deployed a vanilla nginx container to my cluster and a loadbalancer service for it and I see that the service was assigned an external ip of 192.168.1.240 Test: • I launched a busybox container on to the cluster. From it I'm able to wget the default homepage of my nginx container, using both the Cluster-IP and External-IP. So I believer MetalLB is working to some level. Problem: • I'm not able to wget the default nginx homepage from my Host Machine using the External-IP. What am I missing? This should be possible right?

✅ 1

david@ubuntu:~/dev/rke2$ kubectl get all -o wide
NAME            READY   STATUS    RESTARTS   AGE     IP           NODE                    NOMINATED NODE   READINESS GATES
pod/busybox     1/1     Running   0          36m     10.42.1.23   rke2-agent1.rke2.test   <none>           <none>
pod/nginx-pod   1/1     Running   0          7m48s   10.42.2.47   rke2-agent2.rke2.test   <none>           <none>

NAME                        TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)        AGE     SELECTOR
service/kubernetes          ClusterIP      10.43.0.1      <none>          443/TCP        2d15h   <none>
service/nginx-pod-service   LoadBalancer   10.43.153.37   192.168.1.240   80:32003/TCP   7m48s   <http://app.kubernetes.io/name=nginx-example|app.kubernetes.io/name=nginx-example>

david@ubuntu:~/dev/rke2$ kubectl get pod -n metallb-system -o wide
NAME                         READY   STATUS    RESTARTS      AGE   IP            NODE                     NOMINATED NODE   READINESS GATES
controller-6d5cb87f6-c5f5g   1/1     Running   1 (51m ago)   23h   10.42.2.144   rke2-agent2.rke2.test    <none>           <none>
speaker-cmh6c                1/1     Running   1 (51m ago)   23h   10.11.0.112   rke2-agent2.rke2.test    <none>           <none>
speaker-gljrr                1/1     Running   1 (51m ago)   23h   10.11.0.111   rke2-agent1.rke2.test    <none>           <none>
speaker-lq42s                1/1     Running   2 (51m ago)   23h   10.11.0.11    rke2-server1.rke2.test   <none>           <none>

I followed these MetalLB troubleshooting steps for checking ARPs and tcpdump and it looks good. https://metallb.universe.tf/configuration/troubleshooting/

I don't know if this reveals anything. From my busybox image, I'm able to wget the External-IP on port 80, but not port 32003.

# server1 VM

[vagrant@rke2-server1 ~]$ kubectl get svc
NAME                TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)        AGE
kubernetes          ClusterIP      10.43.0.1      <none>          443/TCP        2d17h
nginx-pod-service   LoadBalancer   10.43.153.37   192.168.1.240   80:32003/TCP   3h5m

# busybox image

/ # wget -O - 192.168.1.240:80
Connecting to 192.168.1.240:80 (192.168.1.240:80)
writing to stdout
... omitted ...
<title>Welcome to nginx!</title>
... omitted ...
-                    100% |****************************************************************************************************************************|   615  0:00:00 ETA
written to stdout

# busybox image

/ # wget -O - 192.168.1.240:32003
Connecting to 192.168.1.240:32003 (192.168.1.240:32003)
wget: can't connect to remote host (192.168.1.240): Connection timed out

Solved. My coworker was able to point out what I was missing- a route on my host machine to the external ip

50 Views

#general Join Slack