https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# k3s
a
This message was deleted.
c
Without a local kubelet and CNI, the control-plane node has no way to reach in-cluster endpoints. Try setting --egress-selector-mode=pod or cluster of pod causes problems.
b
I thought this would be the case but certainly something is working since I can curl against the webhook url from the control plane pod. There's the 
Proxy via http_connect protocol over tcp
message which makes me believe the control plane is proxying the connection back to the node hosting the pod.
Where do I put the 
egress-selector-mode
flag?
I put 
egress-selector-mode: pod
in the cp configmap and I think it might have done the trick - will test more and report back.
c
what configmap? K3s doesn’t use a configmap to configure itself. You would want to put it in the server CLI flags or config.yaml.
b
Oh I'm sorry, that's what I meant - I put it in config.yaml which is in a configmap since K3s is running in a pod... Either way, could you point me towards some documentation of what this setting does and what the differences between them?
c
b
Thanks, I moved my cluster's control plane from 3 masters at home to an AKS Pod - seems to work so far. Though, it would be nice to have this argument documented in the official docs. Also, not sure if the 
pod
setting is default, because until I set it - nothing was working...
c
Agent is the default. Was changed after that pr.
👍 1
b
Thanks for your help @creamy-pencil-82913 - always on point 🙂
Hey @creamy-pencil-82913, I noticed a small issue with K3s starting in a K8s pod. Etcd doesn't like the fact the pod IP changes.
Copy code
time="2022-10-05T08:59:35Z" level=info msg="Failed to test data store connection: this server is a not a member of the etcd cluster. Found [k3st-control-plane-0-9ed411d0=<https://10.244.1.66:2380>], expect: k3st-control-plane-0-9ed411d0=<https://10.244.1.68:2380>"
I worked around it using a small script to update the endpoint IP in etcd allowing it recognize itself as a member and all works well. But I thought, perhaps it would be a good idea to raise an issue on it? Or is there already some kind of flag that would handle it. 
--node-ip=$(NODE_IP)
with the variable having the IP of the pod doesn't do the trick.
c
When using etcd, node IP addresses are expected to be static for the life of a server node in the cluster.
🤣 1
If you’re running a single-node k3s cluster in a pod, why not just use kine+sqlite?
b
That is a good idea, just haven't explored kine yet. I wanted to keep the possibility of scaling up the statefulset to 2/3 nodes and have it work - but there might be no point in doing that so sqlite might be a good idea.
451 Views
Open in Slack
PreviousNext
Powered by