https://rancher.com/ logo
#k3s
Title
# k3s
h

handsome-toddler-59547

06/16/2022, 10:58 PM
Hi. Quick Technical Question I've deployed a fresh k3s server.. using klipper. I've got an nginx service up and running and its exposed on an external ip.
Copy code
(base) [dsargrad@localhost nginx]$ k get services

NAME         TYPE           CLUSTER-IP    EXTERNAL-IP                     PORT(S)          AGE

kubernetes   ClusterIP      10.43.0.1     <none>                          443/TCP          6h31m

nginx        LoadBalancer   10.43.228.3   192.168.56.133,192.168.56.134   9080:32741/TCP   31m
For some odd reason I can only access it on 192.168.56.134 (the worker). I cant access it on the master: 192.168.56.133.
Copy code
(base) [dsargrad@localhost nginx]$ curl 192.168.56.133:9080

curl: (7) Failed to connect to 192.168.56.133 port 9080 after 0 ms: No route to host

(base) [dsargrad@localhost nginx]$ curl 192.168.56.134:9080

<!DOCTYPE html>

<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>

</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="<http://nginx.org/>"><http://nginx.org|nginx.org></a>.<br/>
Commercial support is available at
<a href="<http://nginx.com/>"><http://nginx.com|nginx.com></a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
Any suggestions on why this is?
f

freezing-engineer-98215

06/17/2022, 1:45 PM
Where are you making the curl from ? maybe it's the firewall blocking that port on the server.
h

handsome-toddler-59547

06/18/2022, 11:31 AM
Hi @freezing-engineer-98215 So I've "curled" both from outside the cluster, and inside the cluster. Currently my firewall should be disabled (both on the master and the worker). Also when I tcpdump the traffic coming into the master (on the port that I am trying to curl), I do see something come in. you can see this curl and the tcpdump below:
@freezing-engineer-98215 When I do the same curl to the worker, you see the entire TCP handshake.
I've just discovered something really weird. I can curl to both external IPs (192.168.56.133:9080, and 192.168.56.134:9080) from the worker. I can't curl to either from the master.
From outside the cluster I can only curl to 192.168.56.134:9080
Copy code
(base) [dsargrad@localhost nginx]$ k get nodes
NAME     STATUS   ROLES                  AGE   VERSION
w        Ready    <none>                 41h   v1.23.6+k3s1
master   Ready    control-plane,master   44h   v1.23.6+k3s1
Copy code
(base) [dsargrad@localhost nginx]$ k get services
NAME         TYPE           CLUSTER-IP    EXTERNAL-IP                     PORT(S)          AGE
kubernetes   ClusterIP      10.43.0.1     <none>                          443/TCP          44h
nginx        LoadBalancer   10.43.228.3   192.168.56.133,192.168.56.134   9080:32741/TCP   38h

(base) [dsargrad@localhost nginx]$ k get endpoints
NAME         ENDPOINTS             AGE
kubernetes   192.168.56.133:6443   44h
nginx        10.42.1.20:80         38h
Copy code
(base) [dsargrad@localhost nginx]$ k get pods
NAME                    READY   STATUS    RESTARTS      AGE
svclb-nginx-r4dwt       1/1     Running   0             38h
nginx-65b69c4fd-vzpmg   1/1     Running   0             38h
svclb-nginx-8x272       1/1     Running   1 (14m ago)   38h
the problem may be the way that I specified the flannel interface. Does this flag need to be specified on the master and on the agent?
Copy code
--flannel-iface $I
this is how i configured the master:
Copy code
export H=192.168.56.133
export I=enp0s8
export N=M
export INSTALL_K3S_EXEC="--write-kubeconfig ~/.kube/config --node-name $N --write-kubeconfig-mode 666 --node-ip $H --node-external-ip $H --tls-san $H --flannel-iface $I --node-taint CriticalAddonsOnly=true:NoExecute" 

curl -sfL <https://get.k3s.io> | sh -s - --disable=traefik server
this is how i configured the agent:
Copy code
export H=192.168.56.134
export N=W
export M=192.168.56.133
export INSTALL_K3S_EXEC="--node-name $N --node-ip $H --node-external-ip $H" 
export K3S_URL="https://$M:6443"  
export K3S_TOKEN="changeme"  
curl -sfL <https://get.k3s.io> | sh -s - agent
I did not specify the --flannel-iface for the agent. this may be the problem. thoughts?
I'm convinced this is my problem. Am I able to reconfigure the flannel iface without rebuilding the cluster? if not, i'll rebuild the cluster! 🙂 I can almost rebuild a k3s cluster with my eyes closed.
Solved: I simply updated my installer for the agent, and installed k3s a second time. this seemed to work. I didnt realize that you can run install again to update configuration. Correct configuration follows:
Copy code
export H=192.168.56.134
export I=enp0s8
export N=W
export M=192.168.56.133
export INSTALL_K3S_EXEC="--node-name $N --node-ip $H --node-external-ip $H --flannel-iface $I" 
export K3S_URL="https://$M:6443"  
export K3S_TOKEN="changeme"  
curl -sfL <https://get.k3s.io> | sh -s - agent
🙌 1
23 Views