https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# k3s
a
This message was deleted.
c
Figure out what other pod is using those node ports.
f
The only suspicious thing I'm seeing is Pods with 
.spec.containers[].ports[].hostPort
set, which seems odd to me
Is hostPort on a pod like a nodePort on a service (except a pod can only get scheduled to one node)
c
hostport means that the pod binds to that specific port on the node. Only one pod can use that port on each node, for obvious reasons.
f
So I see another 
svclb-*
pod in the kube-system namespace using that same hostPort, but the thing I can't trace back is wha's creating that pod?
Both sets of 
svclb-*
pods are running klipper-lb:v0.4.9, and what's funny is in the k3s cluster I upgraded to 1.30, I don't see those 
svclb-*
pods at all.
c
those are for LoadBalancer services. The name is based on the service name. If you describe the pod, there are also some labels that tell you what service they’re for.
f
So the service is showing:
Copy code
spec:
  allocateLoadBalancerNodePorts: true
  clusterIP: 10.43.129.45
  clusterIPs:
  - 10.43.129.45
  - fd00:2b::d61
  externalTrafficPolicy: Local
  healthCheckNodePort: 31269
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  - IPv6
  ipFamilyPolicy: RequireDualStack
  ports:
  - appProtocol: http
    name: http
    nodePort: 30670
    port: 80
    protocol: TCP
    targetPort: http
  - appProtocol: https
    name: https
    nodePort: 32379
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    <http://app.kubernetes.io/component|app.kubernetes.io/component>: controller
    <http://app.kubernetes.io/instance|app.kubernetes.io/instance>: nginx-ingress-internal
    <http://app.kubernetes.io/name|app.kubernetes.io/name>: ingress-nginx
  sessionAffinity: None
  type: LoadBalancer
When I look at the corresponding 
svc-*
Pods:
Copy code
image: rancher/klipper-lb:v0.4.9
imagePullPolicy: IfNotPresent
name: lb-tcp-80
ports:
- containerPort: 80
  hostPort: 80        # ??? not 30670
  name: lb-tcp-80
  protocol: TCP
and
Copy code
ports:
- containerPort: 443
  hostPort: 443       # ??? not 32379
  name: lb-tcp-443
  protocol: TCP
I would think that the hostPort would wind up being the same as the nodePort
ohhhhhh
You know what? We're running MetalLB in both clusters
But one cluster (the production cluster) is lacking 
--disable servicelb
So question for you -- I'm running a cluster with 3 control nodes, I know that's one of the settings needs to be synchronized across all control nodes... Is that something I can change after the fact?
Is that one of those things where I need to change the setting and restart k3s on all the control nodes all at once
Alright -- narrating my own thoughts
I went through each control node, added that "disable" flag, and restarted it
Those pods went away
Some scary status messages popped up in k9s (SysctlForbidden or something), then they disappeared 😆
aaaand everything is still online
c
yeah if you’re deploying a different loadbalancer controller you definitely want to disable servicelb
f
Yeah, it think that was an oversight on our part when deploying that cluster
I'm just glad I can CHANGE it after the fact
c
yeah, disables are fine to do after the fact. It’s CNI stuff that’s hard to change later.
f
Yeah - I've seen a blog post on how to do a Flannel -> Cilium change that was pretty complicated. Honestly if we were going to do that I'd consider shutting down the cluster in a maintenance window to make the change
10 Views
Open in Slack
PreviousNext
Powered by