Hi Quick Technical Question I ve deployed a fresh k3s server using klipper I ve got an nginx service up and running and its exposed on an external ip ``` base dsargrad localhost nginx $ k get services NAME TYPE CLUSTER IP EXTERNAL IP PORT S AGE kubernetes ClusterIP 10 43 0 1 lt none gt 443 TCP 6h31m nginx LoadBalancer 10 43 228 3 192 168 56 133 192 168 56 134 9080 32741 TCP 31m``` For some odd reason I can only access it on 192 168 56 134 the worker I cant access it on the master 192 168 56 133 ``` base dsargrad localhost nginx $ curl 192 168 56 133 9080 curl 7 Failed to connect to 192 168 56 133 port 9080 after 0 ms No route to host base dsargrad localhost nginx $ curl 192 168 56 134 9080 < DOCTYPE html> lt html gt lt head gt lt title gt Welcome to nginx lt title gt lt style gt html color scheme light dark body width 35em margin 0 auto font family Tahoma Verdana Arial sans serif lt style gt lt head gt lt body gt lt h1 gt Welcome to nginx lt h1 gt lt p gt If you see this page the nginx web server is successfully installed and working Further configuration is required lt p gt lt p gt For online documentation and support please refer to lt a href= <http nginx org > gt <http nginx org|nginx org> lt a gt lt br gt Commercial support is available at lt a href= <http nginx com > gt <http nginx com|nginx com> lt a gt lt p gt lt p gt lt em gt Thank you for using nginx lt em gt lt p gt lt body gt lt html gt ``` Any suggestions on why this is

``` base dsargrad localhost nginx $ k get nodes NAME STATUS ROLES AGE VERSION w Ready lt none gt 41h v1 23 6+k3s1 master Ready control plane master 44h v1 23 6+k3s1```

Hi Quick Technical Question I ve deployed a fresh k3s server rancher-users #k3s

#k3s

Title

# k3s

handsome-toddler-59547

06/16/2022, 10:58 PM

Hi. Quick Technical Question I've deployed a fresh k3s server.. using klipper. I've got an nginx service up and running and its exposed on an external ip.

Copy code

(base) [dsargrad@localhost nginx]$ k get services

NAME         TYPE           CLUSTER-IP    EXTERNAL-IP                     PORT(S)          AGE

kubernetes   ClusterIP      10.43.0.1     <none>                          443/TCP          6h31m

nginx        LoadBalancer   10.43.228.3   192.168.56.133,192.168.56.134   9080:32741/TCP   31m

For some odd reason I can only access it on 192.168.56.134 (the worker). I cant access it on the master: 192.168.56.133.

Copy code

(base) [dsargrad@localhost nginx]$ curl 192.168.56.133:9080

curl: (7) Failed to connect to 192.168.56.133 port 9080 after 0 ms: No route to host

(base) [dsargrad@localhost nginx]$ curl 192.168.56.134:9080

<!DOCTYPE html>

<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>

</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="<http://nginx.org/>"><http://nginx.org|nginx.org></a>.<br/>
Commercial support is available at
<a href="<http://nginx.com/>"><http://nginx.com|nginx.com></a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Any suggestions on why this is?

freezing-engineer-98215

06/17/2022, 1:45 PM

Where are you making the curl from ? maybe it's the firewall blocking that port on the server.

handsome-toddler-59547

06/18/2022, 11:31 AM

Hi @freezing-engineer-98215 So I've "curled" both from outside the cluster, and inside the cluster. Currently my firewall should be disabled (both on the master and the worker). Also when I tcpdump the traffic coming into the master (on the port that I am trying to curl), I do see something come in. you can see this curl and the tcpdump below:

@freezing-engineer-98215 When I do the same curl to the worker, you see the entire TCP handshake.

I've just discovered something really weird. I can curl to both external IPs (192.168.56.133:9080, and 192.168.56.134:9080) from the worker. I can't curl to either from the master.

From outside the cluster I can only curl to 192.168.56.134:9080

Copy code

(base) [dsargrad@localhost nginx]$ k get nodes
NAME     STATUS   ROLES                  AGE   VERSION
w        Ready    <none>                 41h   v1.23.6+k3s1
master   Ready    control-plane,master   44h   v1.23.6+k3s1

Copy code

(base) [dsargrad@localhost nginx]$ k get services
NAME         TYPE           CLUSTER-IP    EXTERNAL-IP                     PORT(S)          AGE
kubernetes   ClusterIP      10.43.0.1     <none>                          443/TCP          44h
nginx        LoadBalancer   10.43.228.3   192.168.56.133,192.168.56.134   9080:32741/TCP   38h

(base) [dsargrad@localhost nginx]$ k get endpoints
NAME         ENDPOINTS             AGE
kubernetes   192.168.56.133:6443   44h
nginx        10.42.1.20:80         38h

Copy code

(base) [dsargrad@localhost nginx]$ k get pods
NAME                    READY   STATUS    RESTARTS      AGE
svclb-nginx-r4dwt       1/1     Running   0             38h
nginx-65b69c4fd-vzpmg   1/1     Running   0             38h
svclb-nginx-8x272       1/1     Running   1 (14m ago)   38h

the problem may be the way that I specified the flannel interface. Does this flag need to be specified on the master and on the agent?

Copy code

--flannel-iface $I

this is how i configured the master:

Copy code

export H=192.168.56.133
export I=enp0s8
export N=M
export INSTALL_K3S_EXEC="--write-kubeconfig ~/.kube/config --node-name $N --write-kubeconfig-mode 666 --node-ip $H --node-external-ip $H --tls-san $H --flannel-iface $I --node-taint CriticalAddonsOnly=true:NoExecute" 

curl -sfL <https://get.k3s.io> | sh -s - --disable=traefik server

this is how i configured the agent:

Copy code

export H=192.168.56.134
export N=W
export M=192.168.56.133
export INSTALL_K3S_EXEC="--node-name $N --node-ip $H --node-external-ip $H" 
export K3S_URL="https://$M:6443"  
export K3S_TOKEN="changeme"  
curl -sfL <https://get.k3s.io> | sh -s - agent

I did not specify the --flannel-iface for the agent. this may be the problem. thoughts?

I'm convinced this is my problem. Am I able to reconfigure the flannel iface without rebuilding the cluster? if not, i'll rebuild the cluster! 🙂 I can almost rebuild a k3s cluster with my eyes closed.

Reference on stack: https://stackoverflow.com/questions/66449289/is-there-any-way-to-bind-k3s-flannel-to-another-interface

Solved: I simply updated my installer for the agent, and installed k3s a second time. this seemed to work. I didnt realize that you can run install again to update configuration. Correct configuration follows:

Copy code

export H=192.168.56.134
export I=enp0s8
export N=W
export M=192.168.56.133
export INSTALL_K3S_EXEC="--node-name $N --node-ip $H --node-external-ip $H --flannel-iface $I" 
export K3S_URL="https://$M:6443"  
export K3S_TOKEN="changeme"  
curl -sfL <https://get.k3s.io> | sh -s - agent

🙌 1

23 Views