How can I force delete a <http nodes longhorn io|nodes longh rancher-users #longhorn-storage

Title

late-needle-80860

02/02/2023, 12:13 PM

How can I force delete a nodes.longhorn.io node object/node … even if I scale back the node into the K3s cluster ( v1.25.6+k3s1 ) - I still can’t remove the node. From neither the Longhorn UI or by using

kubectl delete …

directly. Any ideas and thank you

Event uninstalling longhorn do not seem to help … the longhorn-uninstaller-… Pod just loops over

Removed finalizer … controller …. node=NODE_NAME

… again and again

Hmm editing

etcd

directly using some

etcdctl get

and

etcd del

on the key of the specific node did the trick. Just feels wrong…. there must be a better way or?

narrow-egg-98197

02/07/2023, 9:53 AM

If an attempt is made to delete a node, the disk should have scheduling disabled and all associated Longhorn volumes detached.

Step-by-step information can be referenced here .

late-needle-80860

02/07/2023, 6:41 PM

I’ll implement the steps we’re not doing - referring to the guide you link to - and then see if it happens again. Thank you @narrow-egg-98197

🙂 1

@narrow-egg-98197 or others … is this possible to do via the Longhorn API? Disable e.g. disk scheduling.

I’m reverse engineering the longhorn project Python based tests to get the steps needed for automation in code … getting there … but a bit cumbersome.

narrow-egg-98197

03/20/2023, 8:35 AM

I think you might can reference the document to using python client and

longhorn/longhorn-test

repo to disable disk scheduling.

node = client.by_id_node(lht_hostId)
    disks = node.disks
    disk = disks[extra_disk_fsid]
    disk.allowScheduling = False
    update_disks = get_update_disks(disks)
    node = update_node_disks(client, node.name, disks=update_disks,
                             retry=True)
    node = wait_for_disk_status(client, lht_hostId,
                                extra_disk_fsid,
                                "allowScheduling", False)
    wait_for_disk_status(client, lht_hostId, extra_disk_fsid,
                         "storageScheduled", 0)

46 Views

#longhorn-storage Join Slack