https://rancher.com/ logo
Title
w

witty-honey-18052

11/01/2022, 11:26 AM
(WSL, RD 1.6.1, k8s 1.25) Is there anything I can do to debug the ports that traefik should be listening on? I disabled traefik to use another stack. I've since re-enabled traefik, even done a factory reset, but I can't seem to get traefik to respond on anything *.localhost on 80 or 443. Traefik is properly enabled as well as other settings like the WSL distro. I can forward ports from my workloads and access those directly, but the page never responds when trying to hit my-site.localhost, suggesting those ports aren't listening. I don't even get a 404 as expected, if the ingress wasn't properly configured. I don't see a port forward for anything traefik related, but I don't remember seeing that before anyways
> kubectl get all --namespace=kube-system
NAME                                          READY   STATUS      RESTARTS      AGE
pod/helm-install-traefik-crd-sxvd6            0/1     Completed   0             14h
pod/helm-install-traefik-c88gt                0/1     Completed   1             14h
pod/svclb-traefik-901669a0-fjfzz              2/2     Running     2 (14h ago)   14h
pod/coredns-75fc8f8fff-4tz6c                  1/1     Running     1 (14h ago)   14h
pod/local-path-provisioner-5b5579c644-q27m7   1/1     Running     1 (14h ago)   14h
pod/traefik-9c6dc6686-z6xks                   1/1     Running     1 (14h ago)   14h
pod/metrics-server-5c8978b444-9k8s5           1/1     Running     1 (14h ago)   14h

NAME                     TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                      AGE
service/kube-dns         ClusterIP      10.43.0.10      <none>           53/UDP,53/TCP,9153/TCP       14h
service/metrics-server   ClusterIP      10.43.223.10    <none>           443/TCP                      14h
service/traefik          LoadBalancer   10.43.106.199   172.24.222.159   80:32537/TCP,443:30380/TCP   14h

NAME                                    DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/svclb-traefik-901669a0   1         1         1       1            1           <none>          14h

NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/coredns                  1/1     1            1           14h
deployment.apps/local-path-provisioner   1/1     1            1           14h
deployment.apps/traefik                  1/1     1            1           14h
deployment.apps/metrics-server           1/1     1            1           14h

NAME                                                DESIRED   CURRENT   READY   AGE
replicaset.apps/coredns-75fc8f8fff                  1         1         1       14h
replicaset.apps/local-path-provisioner-5b5579c644   1         1         1       14h
replicaset.apps/traefik-9c6dc6686                   1         1         1       14h
replicaset.apps/metrics-server-5c8978b444           1         1         1       14h

NAME                                 COMPLETIONS   DURATION   AGE
job.batch/helm-install-traefik-crd   1/1           11s        14h
job.batch/helm-install-traefik       1/1           16s        14h
OS Name:                   Microsoft Windows 11 Pro
OS Version:                10.0.22621 N/A Build 22621
j

jolly-forest-99711

11/01/2022, 6:35 PM
Have you forwarded ports 80 and/or 443 from the RD WSL distro to the host machine? I need to do this on my Linux install. Make sure you have "Include Kubernetes services" checked, you need to do that to see the traefik services
Another possible problem is that you're trying to access k8s service ports that have been forwarded through the RD GUI from inside a WSL distro. Recently there was another user who couldn't do that. I also experienced this. I have been meaning to look for an issue pertaining to that, and to create it if it doesn't exist, but I haven't gotten to it yet
Do either of those things solve/explain your problem?
w

witty-honey-18052

11/01/2022, 8:53 PM
Unfortunately not. When I try to forward the ports and click the check, the input box for the port number just disappears
The RD GUI is on the windows side, not embedded in a distro in wsl
but the integration is definitely working with WSL otherwise
maybe there's a chance that since the other stack included a traefik:latest image, that theres some crossover there? not certain, just grasping for straws
the debug logs don't really show anything obvious
i should have just shown this to y'all in kubecon, ugh, but i didn't think it would persist. thought it was related to the windows update (still might be)
j

jolly-forest-99711

11/01/2022, 9:25 PM
When I try to forward the ports and click the check, the input box for the port number just disappears
Sounds like a bug to me.
The RD GUI is on the windows side, not embedded in a distro in wsl
Sorry, I wasn't specific enough. It's expected that the RD GUI is on the windows side. I was referring to how RD integrates with WSL distros when you configure it to do so (it sounds like you did this)
the debug logs don't really show anything obvious
Are you sure? The relevant logs should be in k8s.log
I'd be surprised if the logs didn't show anything at all in relation to port forwarding
If you log into the RD VM using
rdctl shell
, are you able to talk to traefik using
curl
?
Another thing that came to mind but probably isn't the problem: how do DNS records like
my-site.localhost
work? I've never seen
localhost
used in any way other than by itself. Is this a Windows thing? Provided by k8s? Either way it would be best to test with
localhost
I think
w

witty-honey-18052

11/01/2022, 9:42 PM
here's the output when I attempted the port forwarding:
2022-11-01T20:46:17.969Z: Setting up new port forwarding to kube-system/traefik:web...
2022-11-01T20:46:17.971Z: Forwarding server for kube-system/traefik:web created.
2022-11-01T20:46:17.971Z: Forwarding server for kube-system/traefik:web added to server list.
2022-11-01T20:46:21.386Z: Found existing server for kube-system/traefik:web.
2022-11-01T20:46:21.386Z: Server listening on 58951, but we want 80. Closing it.
2022-11-01T20:46:21.387Z: Setting up new port forwarding to kube-system/traefik:web...
2022-11-01T20:46:21.388Z: Error: listen EACCES: permission denied 127.0.0.1:80
2022-11-01T20:46:24.002Z: Setting up new port forwarding to kube-system/traefik:web...
2022-11-01T20:46:24.004Z: Forwarding server for kube-system/traefik:web created.
2022-11-01T20:46:24.004Z: Forwarding server for kube-system/traefik:web added to server list.
2022-11-01T20:46:25.979Z: Found existing server for kube-system/traefik:web.
2022-11-01T20:46:25.979Z: Server listening on 58955, but we want 80. Closing it.
2022-11-01T20:46:25.980Z: Setting up new port forwarding to kube-system/traefik:web...
2022-11-01T20:46:25.981Z: Error: listen EACCES: permission denied 127.0.0.1:80
2022-11-01T20:46:27.271Z: Setting up new port forwarding to kube-system/traefik:websecure...
2022-11-01T20:46:27.272Z: Forwarding server for kube-system/traefik:websecure created.
2022-11-01T20:46:27.272Z: Forwarding server for kube-system/traefik:websecure added to server list.
2022-11-01T20:46:29.379Z: Found existing server for kube-system/traefik:websecure.
2022-11-01T20:46:29.379Z: Server listening on 58957, but we want 443. Closing it.
2022-11-01T20:46:29.379Z: Setting up new port forwarding to kube-system/traefik:websecure...
2022-11-01T20:46:29.380Z: Error: listen EACCES: permission denied 127.0.0.1:443
the permission denied is interesting
(i tried with a couple of random ports in there as well)
on the localhost thing, anything subdomain of localhost simply goes to localhost. It's just helpful for the hostname matching done by traefik
re: rdctl shell:
$ rdctl shell
Error: can't find the lima-home directory at "/home/{user}/.local/share/rancher-desktop/lima"
j

jolly-forest-99711

11/01/2022, 10:27 PM
lima-home... that's definitely wrong. You said you're on Windows right? Or do you have some hack to install RD inside a WSL distro?
It's weird that it says anything about lima, because lima is only used in the macOS and Linux versions of Rancher Desktop
w

witty-honey-18052

11/01/2022, 10:28 PM
definitely windows install, unless the updater grabbed something wrong
👍 1
j

jolly-forest-99711

11/01/2022, 10:29 PM
Regarding the logs, are you trying to forward the traefik ports to 80 and 443 on your host machine? I suspect that could cause issues with the WSL distro - usually you need root privileges to listen on ports below 1024. Have you tried forwarding to ports above 1024?
w

witty-honey-18052

11/01/2022, 10:30 PM
I thought RD had an update where 80 and 443 were supported, but i could be taking crazy pills
j

jolly-forest-99711

11/01/2022, 10:31 PM
Either way, wouldn't hurt to try?
If
rdctl shell
is producing a message about lima something is pretty screwed up anyways
These could be separate issues though
w

witty-honey-18052

11/01/2022, 10:36 PM
fails on 8080 and 8443 as well
might need to reinstall completely. this might be fubar
j

jolly-forest-99711

11/01/2022, 10:37 PM
Yeah it's worth a shot. If that doesn't fix it, it's probably best if you create an issue. Good to include the logs if you do
w

witty-honey-18052

11/01/2022, 10:43 PM
this is interesting too:
that started happening after my windows update
that UI bug still happens on fresh install haha
j

jolly-forest-99711

11/01/2022, 10:59 PM
That is interesting indeed. What kind of machine is it? Company laptop?
w

witty-honey-18052

11/01/2022, 11:00 PM
its a lenovo T14, windows pro
personal laptop
j

jolly-forest-99711

11/01/2022, 11:01 PM
Welp... create an issue I guess? Unless we can reproduce it it's going to be tough to solve though
That's crazy about that UI bug though. There might be something in the logs. I can't help but wonder if it's an electron bug or something like that though
w

witty-honey-18052

11/01/2022, 11:02 PM
yea that's prob electron, so i didn't raise a stink about it
finishing install now, trying the ports again
stumped
going to need to come back to it later
might try to revert back to 1.5
j

jolly-forest-99711

11/02/2022, 4:20 PM
Yeah that's a good thing to try. Let me know how it goes