this was on a single node k3s. running on RKE2 ins...
# general
f
this was on a single node k3s. running on RKE2 instead fixed the initial issue, but now I have an issue with creating a new downstream cluster. I guess 2.14.x isn't "stable" yet, even though it's labelled as such?
c
It is. there shouldn’t be any significant difference between k3s and rke2, when it comes to running rancher on them.
👍 1
Do you have something else on your node that is mucking with firewall rules for pods? It shouldn’t be possible for pods to get errors connecting to the apiserver on the server node itself. In k3s, if the apiserver is down then so is all of K3s, and there wouldn’t be anything to run the pod.
f
Copy code
I0610 17:35:10.725677   36816 reflector.go:578] "Warning: watch ended with error" reflector="pkg/mod/k8s.io/client-go@v0.35.1/tools/cache/reflector.go:289" type="*v1.Secret" err="an error on the server (\"unable to decode an event from the watch stream: stream error: stream ID 3; INTERNAL_ERROR; received from peer\") has prevented the request from succeeding"
I'm getting this on the rancher-system-agent on the first control-plane I'm trying to add during a new cluster creation.
c
ok?
You’re kinda just throwing random stuff out there, there’s not really enough info to work with
What versions of K3s or RKE2 are you deploying rancher to? How much cores/cpu/iops does the node hosting Rancher have available to it? What ingress and/or load-balancers are between Rancher and the downstream cluster members that you are attempting to create?
you really need to take this one step at a time. Just throwing one line of error message out doesn’t help.
f
default version of k3s, 4 cores, 16GB RAM, SSD. I just noticed I have other iptable rules, but the behavior is odd since I can connect to the API just after waiting 1 second.
^ regarding the first issue of fleet unable to run a job properly when on k3s
c
Yeah that sounds like firewalld, ufw, or some other security agent messing with pod networking. That is not a problem I ever see people running into
👍 1
There is no reason there should be any delay in a pods’s ability to hit the in-cluster apiserver endpoint
👍 1
f
I'll look at that and see if that fixes everything
damn. it was definitely that. sorry for the trouble. haha. I also forgot to add the cis prereqs, but nothing screamed at me to do that. curious if some people could benefit from that.
similar to iptables. perhaps something can be like "you already have some iptable rules. maybe you want to delete those??"