Hi, I am having a problem with upgrading my harvester cluster. Everything went smoothly until the first node was in the

post-draining

status. At some point during that process a vm was spawned which got stuck in Containercreating because of a problem with the longhorn volume it was trying to mount. I deleted the vm launcher, the pvc and the post-drain job (because it was making it impossible to delete the vm + pvc). Now I want to restart the upgrade but when trying

kubectl delete <http://upgrade.harvesterhci.io/hvst-upgrade-gxc4t|upgrade.harvesterhci.io/hvst-upgrade-gxc4t> -n harvester-system

it fails with Error from server (BadRequest): admission webhook

"<http://validator.harvesterhci.io|validator.harvesterhci.io>" denied the request: cluster fleet-local/local status is provisioning, please wait for it to be provisioned

is there anything I can do to fix this?

hi @lively-zebra-61132 try rebooting the stuck node. i've read several issues resolved by rebooting the stuck node, and it worked for me on a different issue as well. when i rebooted it though, there didnt seem to be much indication that the node was progressing until it finished

That did not help unfortunately. I think I finally broke it completely and will have to do a fresh install

bummer 😞 sorry, i'm only meh so far, still figuring out a lot and learning things myself.

Hi @lively-zebra-61132 have you already started doing a fresh install? If not I'd like to help, I work on the Harvester team and am currently acting as community coordinator

Hi @limited-breakfast-50094 thank you for reaching out. Yes we have already done a fresh install. In the end I should not have deleted the

post-drain

job and tried to fix the issue with the VM instead. In order to prevent this issue in the future harvester could alert users if longhorn is unable to schedule sufficiently large volumes before an upgrade or a warning about having enough space in longhorn disks could be included in the documentation.

Sounds good, can you file a feature request in github.com/harvester/harvester/issues/ ?