https://rancher.com/ logo
Title
w

wonderful-appointment-6480

12/24/2022, 8:07 AM
Hi all! I started using longhorn a few days ago on k3s single node setup. I really like it so I decided to expand the cluster to a k3s etcd 3 master nodes setup. I installed ubuntu 22.04 and installed open-iscsi as described in the docs, but when the cluster starts to run the new longhorn pods on the new nodes, the
longhorn-manager
fails to get ready, unable to list/watch resources, showing various errors such as:
W1224 07:44:23.287997       1 client_config.go:617] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
time="2022-12-24T07:44:23Z" level=info msg="cannot list the content of the src directory /var/lib/rancher/longhorn/engine-binaries for the copy, will do nothing: failed to execute: nsenter [--mount=/host/proc/1/ns/mnt --net=/host/proc/1/ns/net bash -c ls /var/lib/rancher/longhorn/engine-binaries/*], output , stderr, ls: cannot access '/var/lib/rancher/longhorn/engine-binaries/*': No such file or directory\n, error exit status 2"
time="2022-12-24T07:44:24Z" level=info msg="Skip the leader election for the upgrade since the current Longhorn system is already up to date"
W1224 07:44:24.943618       1 client_config.go:617] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I1224 07:44:24.961708       1 shared_informer.go:240] Waiting for caches to sync for longhorn datastore
W1224 07:44:25.003320       1 reflector.go:324] <http://k8s.io/client-go/informers/factory.go:134|k8s.io/client-go/informers/factory.go:134>: failed to list *v1beta1.CronJob: the server could not find the requested resource
E1224 07:44:25.003407       1 reflector.go:138] <http://k8s.io/client-go/informers/factory.go:134|k8s.io/client-go/informers/factory.go:134>: Failed to watch *v1beta1.CronJob: failed to list *v1beta1.CronJob: the server could not find the requested resource
W1224 07:44:25.020929       1 reflector.go:324] <http://k8s.io/client-go/informers/factory.go:134|k8s.io/client-go/informers/factory.go:134>: failed to list *v1beta1.PodDisruptionBudget: the server could not find the requested resource
E1224 07:44:25.020957       1 reflector.go:138] <http://k8s.io/client-go/informers/factory.go:134|k8s.io/client-go/informers/factory.go:134>: Failed to watch *v1beta1.PodDisruptionBudget: failed to list *v1beta1.PodDisruptionBudget: the server could not find the requested resource
E1224 07:44:25.027597       1 kubernetes_node_controller.go:227] couldn't get kubernetes node master1: node "master1" not found 
time="2022-12-24T07:44:25Z" level=warning msg="Can't find instance manager for pod instance-manager-e-e867fc5d, may be deleted"
time="2022-12-24T07:44:25Z" level=warning msg="Can't find instance manager for pod instance-manager-r-10c2a1ba, may be deleted"
Is it something I did wrong while installing? Any hints what may have gotten wrong?
k

kind-alarm-73406

12/30/2022, 12:35 AM
Did you upgrade k3s to 1.25?
Longhorn <1.4 is incompatible with 1.25 We are about to release 1.4, here is the RC https://github.com/longhorn/longhorn/releases/tag/v1.4.0-rc3