https://rancher.com/ logo
Title
b

breezy-autumn-81048

04/28/2023, 7:25 AM
Hi folks, Perhaps someone will have an idea of what should be checked and what could cause that issue, let me describe. The K3S cluster version is
v1.26.2+k3s1 (ea094d1d)
. When the pod is running on the server nodes everything works fine, but when it gets created on the agent nodes after some time it stops working, logs from the controller that is running in that pod on the agent node:
NAMESPACE                       NAME                                         READY   STATUS             RESTARTS         AGE     IP           NODE                   NOMINATED NODE
  READINESS GATES
actions-runner-system           actions-runner-controller-798b5fdf46-7p9bn   1/2     CrashLoopBackOff   16 (3m12s ago)   84m     10.42.7.9    <http://yc3142.host.com|yc3142.host.com>   <none>
Defaulted container "manager" out of: manager, kube-rbac-proxy
2023-04-28T06:31:29Z	ERROR	Failed to get API Group-Resources	{"error": "Get \"<https://10.43.0.1:443/api?timeout=32s>\": dial tcp 10.43.0.1:443: i/o timeout"}
<http://sigs.k8s.io/controller-runtime/pkg/cluster.New|sigs.k8s.io/controller-runtime/pkg/cluster.New>
	<http://sigs.k8s.io/controller-runtime@v0.14.4/pkg/cluster/cluster.go:161|sigs.k8s.io/controller-runtime@v0.14.4/pkg/cluster/cluster.go:161>
<http://sigs.k8s.io/controller-runtime/pkg/manager.New|sigs.k8s.io/controller-runtime/pkg/manager.New>
	<http://sigs.k8s.io/controller-runtime@v0.14.4/pkg/manager/manager.go:359|sigs.k8s.io/controller-runtime@v0.14.4/pkg/manager/manager.go:359>
main.main
	./main.go:173
runtime.main
	runtime/proc.go:250
2023-04-28T06:31:29Z	ERROR	unable to start manager	{"error": "Get \"<https://10.43.0.1:443/api?timeout=32s>\": dial tcp 10.43.0.1:443: i/o timeout"}
main.main
	./main.go:188
runtime.main
	runtime/proc.go:250
Was trying to check connectivity from the node
<http://yc3142.host.com|yc3142.host.com>
directly:
nc -v 10.43.0.1 443
Ncat: Version 7.70 ( <https://nmap.org/ncat> )
Ncat: Connected to 10.43.0.1:443.
So, the node can access 10.43.0.1 on port 443, but can't get why the app stops working after some time on the agent nodes (again, on the master nodes works fine) Firewalld is disabled. In the internal firewall, I have allowed all traffic. Thanks in advance,
Master node where cluster was initiated:
# k3s kubectl cluster-info

Kubernetes control plane is running at <https://127.0.0.1:6443>
CoreDNS is running at <https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy>
Metrics-server is running at <https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy>