worried-activity-29498
08/16/2022, 9:29 PMnarrow-action-54485
08/16/2022, 10:02 PMAug 16 20:56:25 ip-xx-xx-xx-xx.us-east-2.compute.internal k3s[4056]: W0816 20:56:25.659366 4056 reflector.go:324] <http://k8s.io/client-go@v1.24.3-k3s1/tools/cache/reflector.go:167|k8s.io/client-go@v1.24.3-k3s1/tools/cache/reflector.go:167>: failed to list *v1.Endpoints: Get "<https://127.0.0.1:6444/api/v1/namespaces/default/endpoints?fieldSelector=metadata.name%3Dkubernetes&limit=500&resourceVersion=0>": net/http: TLS handshake timeout
Any ideas on what to try would be greatly appreciated.brainy-postman-1566
08/16/2022, 11:17 PMAug 16 18:12:49 Kube-0 k3s[3070]: I0816 18:12:49.519375 3070 server.go:576] external host was not specified, using 192.168.0.83
In journal
conf:
kubes@Kube-0:/etc/rancher/k3s$ cat config.yaml
write-kubeconfig-mode: 644
token: "<removed>"
advertise-address: <https://conallogribin.dyns.cx:6443>
node-external-ip: <https://conallogribin.dyns.cx:6443>
busy-cartoon-13211
08/16/2022, 11:53 PMbulky-air-13975
08/17/2022, 1:47 AMflaky-shampoo-86024
08/17/2022, 1:53 AMdamp-shoe-96378
08/17/2022, 7:58 AM"2022/08/17 07:59:25 [ERROR] Failed to handling tunnel request from remote address response 400: cluster not found
2022/08/17 07:59:26 [ERROR] Failed to handling tunnel request from remote address response 400: cluster not found
2022/08/17 07:59:26 [ERROR] Failed to handling tunnel request from remote address response 400: cluster not found
2022/08/17 07:59:27 [ERROR] Failed to handling tunnel request from remote address response 400: cluster not found
2022/08/17 07:59:28 [ERROR] Failed to handling tunnel request from remote address response 400: cluster not found
2022/08/17 07:59:29 [ERROR] Failed to handling tunnel request from remote address response 400: cluster not found
2022/08/17 07:59:30 [ERROR] Failed to handling tunnel request from remote address response 400: cluster not found
2022/08/17 07:59:32 [ERROR] Failed to handling tunnel request from remote address : response 400: cluster not found
2022/08/17 07:59:32 [ERROR] Failed to handling tunnel request from remote address response 400: cluster not found"
helpful-beard-54962
08/17/2022, 1:17 PMsome-oil-32535
08/17/2022, 1:23 PMsome-oil-32535
08/17/2022, 1:25 PMflat-shoe-2195
08/17/2022, 4:35 PM~/.rd/docker.sock
created? I’m trying to understand how Rancher works internally, but can’t figure out how is the socket for the docker communication created.narrow-action-54485
08/17/2022, 5:47 PMWarning FailedAttachVolume 30s (x3 over 34s) attachdetach-controller AttachVolume.Attach failed for volume "pvc-59d0a314-528d-4477-9556-118d2e7bf756" :
rpc error: code = DeadlineExceeded desc = volume pvc-59d0a314-528d-4477-9556-118d2e7bf756 failed to attach to node ip-10-0-126-227.us-east-2.compute.internal
I checked that both iscsid
and nfs-utils
services are active.sparse-exabyte-74414
08/17/2022, 6:42 PMlittle-actor-95014
08/17/2022, 8:33 PMswift-market-16829
08/17/2022, 9:46 PMlate-flower-16846
08/18/2022, 1:26 AMcareful-optician-75900
08/18/2022, 3:05 AMstraight-vr-13697
08/18/2022, 6:59 AMstraight-vr-13697
08/18/2022, 6:59 AMstraight-vr-13697
08/18/2022, 6:59 AMalert-doctor-43173
08/18/2022, 7:30 AMdry-gpu-27616
08/18/2022, 11:35 AMbrash-monitor-41966
08/18/2022, 1:02 PMmost-sunset-36476
08/18/2022, 1:49 PMINFO: Arguments: --no-register --only-write-certs --node-name system1 --server <https://rancher.sauron.mordor.net> --token REDACTED --ca-checksum 928a476fa0b0610ef46217292d51ac438f4ffa56ea3b155f240ee89ff4c1f31b
INFO: Environment: CATTLE_ADDRESS=10.100.0.4 CATTLE_AGENT_CONNECT=mordor CATTLE_INTERNAL_ADDRESS= CATTLE_NODE_NAME=system1 CATTLE_SERVER=<https://rancher.sauron.kangaroot.net> CATTLE_TOKEN=REDACTED CATTLE_WRITE_CERT_ONLY=true
INFO: Using resolv.conf: nameserver 127.0.0.53 options edns0 search <http://u52yollp52yetgbxgli0ra2ssb.ax.internal.cloudapp.net|u52yollp52yetgbxgli0ra2ssb.ax.internal.cloudapp.net>
WARN: Loopback address found in /etc/resolv.conf, please refer to the documentation how to configure your cluster to resolve DNS properly
ERROR: <https://rancher.sauron.mordor.net/ping> is not accessible (Failed to connect to <http://rancher.sauron.mordor.net|rancher.sauron.mordor.net> port 443: Connection timed out)
Any idea ? I would greatly appreciate help on this!
I opened an issue with the code used and provisioning log for the cluster:
https://github.com/rancher/terraform-provider-rancher2/issues/972```most-sunset-36476
08/18/2022, 2:02 PMcurved-lifeguard-39360
08/18/2022, 4:22 PMrancher kubectl rollout status -w deployment acmepa -n acmepa
It immediately replys with deployment "acmepa" successfully rolled out
but then hangs there for 5 minutes or so.numerous-angle-77908
08/18/2022, 9:25 PMfull-fireman-68072
08/19/2022, 12:38 AMfull-fireman-68072
08/19/2022, 12:42 AMripe-actor-83292
08/19/2022, 2:50 AM