Hi team, I have a three-node Harvester cluster run...
# harvester
g
Hi team, I have a three-node Harvester cluster running version 1.4.3. I started an upgrade to 1.5.1, but the process got stuck on the post-drain step for one of the nodes. I’m seeing the following error:
Copy code
Error starting VirtualMachine <http://virtualmachine.kubevirt.io|virtualmachine.kubevirt.io> "upgrade-repo-hvst-upgrade-85csn" not found
+ true
+ sleep 10
++ get_repo_vm_status
++ kubectl get <http://virtualmachines.kubevirt.io|virtualmachines.kubevirt.io> upgrade-repo-hvst-upgrade-85csn -n harvester-system '-o=jsonpath={.status.printableStatus}'
Error from server (NotFound): <http://virtualmachines.kubevirt.io|virtualmachines.kubevirt.io> "upgrade-repo-hvst-upgrade-85csn" not found
+ [[ '' == \R\u\n\n\i\n\g ]]
+ echo 'Try to bring up the upgrade repo VM...'
+ virtctl start upgrade-repo-hvst-upgrade-85csn -n harvester-system
Try to bring up the upgrade repo VM...
Error starting VirtualMachine <http://virtualmachine.kubevirt.io|virtualmachine.kubevirt.io> "upgrade-repo-hvst-upgrade-85csn" not found
...
This happens when running:
Copy code
kubectl -n harvester-system logs hvst-upgrade-85csn-post-drain-mgmt-harv-03-n9pj4 --tail=200 -f
I also have another issue:
Copy code
kubectl get jobs -n harvester-system -l <http://harvesterhci.io/upgradeComponent=node|harvesterhci.io/upgradeComponent=node>
NAME                                         STATUS     COMPLETIONS   DURATION   AGE
hvst-upgrade-85csn-post-drain-mgmt-harv-03   Running    0/1           11h        11h
hvst-upgrade-85csn-pre-drain-mgmt-harv-03    Complete   1/1           6h23m      18h

kubectl logs -n harvester-system hvst-upgrade-85csn-post-drain-mgmt-harv-03
error: error from server (NotFound): pods "hvst-upgrade-85csn-post-drain-mgmt-harv-03" not found
Could you please advise how I can proceed with the cluster upgrade and unblock the process? Thanks in advance for your time and help. Best regards, Damyan