https://rancher.com/ logo
Title
b

bitter-tailor-6977

04/24/2023, 8:03 PM
Hi All, i have installed longhorn in k3s node and uninstalled and tried to reinstall again in same node… i can see below error.. i have properly deleted / uninstalled as well…still issue persists logs from longhorn-manager-988w4 pod
required setting default-share-manager-image shouldn't be empty" node=prodquartic
E0424 19:59:02.718884       1 volume_controller.go:226] failed to sync longhorn-system/pvc-52f24223-1029-4812-b10a-0c388a453c69: unable to get engine image longhornio/longhorn-engine:v1.4.0: <http://engineimage.longhorn.io|engineimage.longhorn.io> "ei-fc06c6fb" not found
E0424 19:59:02.718909       1 volume_controller.go:226] failed to sync longhorn-system/pvc-0d44dadf-a3df-4842-949c-84a3f688645d: unable to get engine image longhornio/longhorn-engine:v1.4.0: <http://engineimage.longhorn.io|engineimage.longhorn.io> "ei-fc06c6fb" not found
time="2023-04-24T19:59:02Z" level=warning msg="Dropping Longhorn volume longhorn-system/pvc-52f24223-1029-4812-b10a-0c388a453c69 out of the queue" controller=longhorn-volume error="failed to sync longhorn-system/pvc-52f24223-1029-4812-b10a-0c388a453c69: unable to get engine image longhornio/longhorn-engine:v1.4.0: <http://engineimage.longhorn.io|engineimage.longhorn.io> \"ei-fc06c6fb\" not found" node=prodquartic
E0424 19:59:02.718887       1 volume_controller.go:226] failed to sync longhorn-system/pvc-0276039c-c505-411c-842e-941ad5213db1: unable to get engine image longhornio/longhorn-engine:v1.4.0: <http://engineimage.longhorn.io|engineimage.longhorn.io> "ei-fc06c6fb" not found
time="2023-04-24T19:59:02Z" level=warning msg="Dropping Longhorn volume longhorn-system/pvc-0d44dadf-a3df-4842-949c-84a3f688645d out of the queue" controller=longhorn-volume error="failed to sync longhorn-system/pvc-0d44dadf-a3df-4842-949c-84a3f688645d: unable to get engine image longhornio/longhorn-engine:v1.4.0: <http://engineimage.longhorn.io|engineimage.longhorn.io> \"ei-fc06c6fb\" not found" node=prodquartic
E0424 19:59:02.720284       1 volume_controller.go:226] failed to sync longhorn-system/pvc-a405ca9d-4778-4b49-a1b1-75ccaa00fd33: unable to get engine image longhornio/longhorn-engine:v1.4.0: <http://engineimage.longhorn.io|engineimage.longhorn.io> "ei-fc06c6fb" not found
E0424 19:59:02.720296       1 volume_controller.go:226] failed to sync longhorn-system/pvc-de46d2c9-2529-428d-bb33-4ecbd3172589: unable to get engine image longhornio/longhorn-engine:v1.4.0: <http://engineimage.longhorn.io|engineimage.longhorn.io> "ei-fc06c6fb" not found

E0424 19:59:02.772007       1 volume_controller.go:226] failed to sync longhorn-system/pvc-b6610285-2d44-4203-87d8-3fe524bf8047: unable to get engine image longhornio/longhorn-engine:v1.4.0: <http://engineimage.longhorn.io|engineimage.longhorn.io> "ei-fc06c6fb" not found
time="2023-04-24T19:59:02Z" level=fatal msg="Error starting manager: admission webhook \"<http://validator.longhorn.io|validator.longhorn.io>\" denied the request: failed to set the setting taint-toleration with invalid value : cannot modify toleration setting before all volumes are detached"
n

narrow-egg-98197

04/25/2023, 2:02 AM
I might think this is related to the un-installation incomplete.
Did you refer to this doc: uninstalling-longhorn-using-kubectl to uninstall Longhorn?
Would you mind running this command to see if there are any remaining resources that weren't deleted?
kubectl api-resources --verbs=list --namespaced -o name   | xargs -n 1 kubectl get --show-kind --ignore-not-found -n longhorn-system
ref. https://github.com/longhorn/longhorn/issues/5311#issuecomment-1409795180
b

bright-wall-24452

04/25/2023, 2:45 AM
Did you execute environment shell script? Is the iscsid process running smoothly on this node?
b

bitter-tailor-6977

04/25/2023, 2:53 AM
no i have executed env shell script …can you share that ? @bright-wall-24452
b

bright-wall-24452

04/25/2023, 2:55 AM
Synchronization
curl -sSfL <https://raw.githubusercontent.com/longhorn/longhorn/v1.3.2/scripts/environment_check.sh> | bash
b

bitter-tailor-6977

04/25/2023, 2:55 AM
@narrow-egg-98197 u are right, it was not uninstalled properly..that manually i deleted few crds and the namespace was uninstalled fine…but still i m not sure why this issue is comming
[WARN]  Unable to check kernel config CONFIG_NFS_V4_2 on node prodquartic
[WARN]  NFS client kernel support, CONFIG_NFS_V4_1 CONFIG_NFS_V4_2, is not enabled on Longhorn nodes. Please refer to <https://longhorn.io/docs/1.4.0/deploy/install/#installing-nfsv4-client> for more information
@bright-wall-24452
even i followed the blog, still issue persists, something i miss, my server is Ubuntu
cat /boot/config-`uname -r`| grep CONFIG_NFS_V4_1
CONFIG_NFS_V4_1=y
CONFIG_NFS_V4_1_IMPLEMENTATION_ID_DOMAIN="<http://kernel.org|kernel.org>"
CONFIG_NFS_V4_1_MIGRATION=y
root@prodquartic:/home/quarticadmin# cat /boot/config-`uname -r`| grep CONFIG_NFS_V4_2
CONFIG_NFS_V4_2=y
when i run inside node itself
curl -sSfL <https://raw.githubusercontent.com/longhorn/longhorn/v1.4.0/scripts/environment_check.sh> | bash
[INFO]  Required dependencies 'kubectl jq mktemp' are installed.
[INFO]  Hostname uniqueness check is passed.
[INFO]  Waiting for longhorn-environment-check pods to become ready (0/1)...
[INFO]  All longhorn-environment-check pods are ready (1/1).
[INFO]  Required packages are installed.
[INFO]  Cleaning up longhorn-environment-check pods...
[INFO]  Cleanup completed.
i got this, previous one done from my localhost ( connected from kubeconfig)
but when i am running this is in a new node…it is success…what prob with current node…how to entirely cleanup
b

bright-wall-24452

04/25/2023, 3:14 AM
longhorn settings follows:
b

bitter-tailor-6977

04/25/2023, 3:18 AM
no my point…how i can make successful installation on this node…i was trying over last 10hours…multiple restarts…deleting and recreting …nothing works…do i need to uninstall entire k3s cluster and give a try ?
now fine
i have entirely uninstalled
and reinstalled
1