high-lawyer-16566
10/19/2022, 10:42 PMbrainy-tomato-18651
10/20/2022, 1:07 AMhandsome-kilobyte-1423
10/20/2022, 5:54 AMancient-engine-93983
10/20/2022, 6:47 AMhundreds-sugar-37524
10/20/2022, 7:05 AMdamp-hydrogen-20107
10/20/2022, 2:23 PMmammoth-vegetable-85565
10/20/2022, 2:24 PMsilly-salesclerk-22619
10/20/2022, 3:16 PMmagnificent-dentist-34072
10/20/2022, 4:14 PMable-insurance-45977
10/20/2022, 4:43 PM2022/10/20 13:26:17 [INFO] Handling backend connection request [c-qfkfg:m-7dde54e01d9f]
2022/10/20 13:26:17 [DEBUG] cluster [c-qfkfg]: created node plan for node [m-7dde54e01d9f]
2022/10/20 13:26:17 [DEBUG] getDockerInfo for node [m-7dde54e01d9f] dockerInfo [/var/lib/docker]
2022/10/20 13:52:41 [DEBUG] Failed to get node for machine [m-7dde54e01d9f], preparing to delete
on a new node agent logs
time="2022-10-20T13:50:15Z" level=info msg="Starting plan monitor, checking every 120 seconds"
We see no other errors.
Any ideas how could we get nodes join cluster again?brash-architect-12851
10/20/2022, 5:00 PM[3:55 PM] Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 4m9s default-scheduler Successfully assigned default/wav-pod to sec-hv017.stt.internal
Warning FailedMount 2m6s kubelet Unable to attach or mount volumes: unmounted volumes=[wav-storage], unattached volumes=[wav-storage kube-api-access-6fdkg]: timed out waiting for the condition
Warning FailedAttachVolume 116s (x9 over 4m8s) attachdetach-controller AttachVolume.Attach failed for volume "wav-pod-pv" : rpc error: code = InvalidArgument desc = volume wav2letter invalid frontend type iscsi
cat pvc.yaml
apiVersion: v1
kind: Pod
metadata:
name: wav-pod
spec:
volumes:
- name: wav-storage
persistentVolumeClaim:
claimName: wav-pod-pvc
containers:
- name: wav-pod-container
image: nginx
ports:
- containerPort: 80
name: "http-server"
volumeMounts:
- mountPath: "/usr/share/nginx/html"
name: wav-storage
jgajra@sec-hv004 ~/pvc $
I created the volume, pv and pvc from Web UI
Container long-share-manager is also not running and has following error message
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Pulling 34s kubelet Pulling image "longhornio/longhorn-share-manager:v1_20221003"
Normal Pulled 32s kubelet Successfully pulled image "longhornio/longhorn-share-manager:v1_20221003" in 2.114492891s
Normal Created 32s kubelet Created container share-manager
Normal Started 31s kubelet Started container share-manager
Warning Unhealthy 4s (x5 over 24s) kubelet Readiness probe failed: cat: /var/run/ganesha.pid: No such file or directory
jgajra@sec-hv004 ~/pvc $ kubectl describe pods share-manager-wav2letter -n longhorn-system
brash-architect-12851
10/20/2022, 5:00 PMbrash-architect-12851
10/20/2022, 5:02 PMcool-vegetable-65608
10/20/2022, 5:59 PMroot@rancher-stg1:/etc/kubernetes/ssl# ls | grep -i kube-etcd-172-16-154
kube-etcd-172-16-154-36-key.pem
kube-etcd-172-16-154-36.pem
root@rancher-stg1:/etc/kubernetes/ssl#
billions-garage-37276
10/20/2022, 7:18 PMhallowed-morning-78693
10/20/2022, 7:44 PMcurl <http://helloworld-go.default.192.168.5.15.sslip.io/>
curl: (6) Could not resolve host: <http://helloworld-go.default.192.168.5.15.sslip.io|helloworld-go.default.192.168.5.15.sslip.io>
hallowed-morning-78693
10/20/2022, 7:46 PMcalm-twilight-27465
10/20/2022, 8:07 PMclean-sundown-64594
10/20/2022, 8:39 PMcattle-unauthenticated
was created on the local cluster. This triggered some security alarm bells and I need to confirm whether or not it was us. I cannot find this in the helm chart so I’m wondering if an operator did this.miniature-garden-10578
10/20/2022, 9:30 PMfast-airline-86487
10/20/2022, 10:46 PMflaky-shampoo-86024
10/21/2022, 1:16 AMadorable-shoe-16839
10/21/2022, 6:51 AMgentle-scooter-5213
10/21/2022, 7:05 AM1.19
-> `1.20`; triggered via Rancher) nor Rancher Update (Rancher 2.5.9
-> `Rancher 2.6.7`; running on k3s) did solve this issue. Rancher is using rancher/eks-operator:v1.1.5
to handle the AWS communication atm.
Any ideas besides recreating the AWS clusters and migrate the workload?
Do you think it’s worth opening an issue in the eks-operator project?aloof-gpu-35180
10/21/2022, 8:04 AMvictorious-river-3598
10/21/2022, 12:08 PMclever-air-65544
10/21/2022, 1:58 PMnarrow-vr-31549
10/21/2022, 2:19 PMnp-default
networkpolicies in a rke2 cluster that is imported as a generic kubernetes cluster in rancher 2.6.9narrow-vr-31549
10/21/2022, 2:20 PMnarrow-vr-31549
10/21/2022, 2:23 PM