https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# harvester
a
This message was deleted.
a
Screenshot from the Rancher perspective:
Only other oddity that I see is that the output of "kubectl get nodes -o wide" seems to imply that the cluster is on v1.1.1: lab-harvester-03:~ # kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME lab-harvester-01 Ready control-plane,etcd,master 274d v1.24.7+rke2r1 10.100.60.164 <none> Harvester v1.1.1 5.3.18-150300.59.101-default containerd://1.6.8-k3s1 lab-harvester-02 Ready control-plane,etcd,master 273d v1.24.7+rke2r1 10.100.60.93 <none> Harvester v1.1.1 5.3.18-150300.59.101-default containerd://1.6.8-k3s1 lab-harvester-03 Ready control-plane,etcd,master 274d v1.24.7+rke2r1 10.100.60.90 <none> Harvester v1.1.1 5.3.18-150300.59.101-default containerd://1.6.8-k3s1
And helm history (as well as the harvester UI version in the lower left of the web page) implies that the cluster has been updated to 1.1.2:
lab-harvester-03:~ # helm history harvester -n harvester-system REVISION UPDATED STATUS CHART APP VERSION DESCRIPTION 1900 Tue Sep 12 152143 2023 superseded harvester-1.1.2 v1.1.2 Upgrade complete 1901 Tue Sep 12 153312 2023 superseded harvester-1.1.2 v1.1.2 Upgrade complete 1902 Tue Sep 12 153526 2023 superseded harvester-1.1.2 v1.1.2 Upgrade complete 1903 Tue Sep 12 153648 2023 superseded harvester-1.1.2 v1.1.2 Upgrade complete 1904 Tue Sep 12 154242 2023 superseded harvester-1.1.2 v1.1.2 Upgrade complete 1905 Tue Sep 12 160305 2023 superseded harvester-1.1.2 v1.1.2 Upgrade complete 1906 Tue Sep 12 160523 2023 superseded harvester-1.1.2 v1.1.2 Upgrade complete 1907 Tue Sep 12 173304 2023 superseded harvester-1.1.2 v1.1.2 Rollback to 1905 1908 Tue Sep 12 190059 2023 superseded harvester-1.1.2 v1.1.2 Upgrade complete 1909 Tue Sep 12 191039 2023 deployed harvester-1.1.2 v1.1.2 Upgrade complete
w
This is the first known issue and links to bug 3839 which you've found though you don't appear to have seen the listed workaround at https://github.com/harvester/harvester/issues/3839#issuecomment-1534438192
a
Thanks @witty-jelly-95845. I was impacted by this when I went from 1.1.1 to 1.1.2. I did follow the workaround steps outlined in 3839, hence the Rancher screenshot above showing that rrancher-monitoring has only been up for 3.2 hours instead of 274 days. I'll review the steps outlined in bug 3839 to see if I missed something. Worst case, I can perform a clean re-install of Harvester.
w
sorry replied from phone so missed the time in your screenshot. unless you need to re-install to get going again I'd suggest creating an issue referencing the bug and your previous experience.
👍 1
g
a support bundle would be appreciated
also the validation is for a managed chart, output of 
kubectl get managedchart -n fleet-local
would really help
a
Thanks @great-bear-19718. I'll get an issue created with the support bundle attached. Also, here is the output of the "kubectl get managedchart -n fleet-local" command: lab-harvester-01:~ # kubectl get managedchart -n fleet-local NAME AGE harvester 275d harvester-crd 275d local-managed-system-upgrade-controller 275d rancher-logging 275d rancher-logging-crd 275d rancher-monitoring 275d rancher-monitoring-crd 275d
It may be tomorrow before I get the issue created.
g
sure.. looking forward to the support bundle
if you could please provide me the issue here.. this way i will not loose it among everything else going on
👍 1
a
https://github.com/harvester/harvester/issues/4537 Support bundle is generating now.
g
like my colleague Jian mentioned in the GH issue.. there is nothing in the webhook pod logs to indicate the monitoring error
might be best to try and refresh UI.. and then restart webhook pods if that doesnt work
a
Thanks @great-bear-19718. I'll give the suggestions a try and report my findings here and in GitHub.
(1) Opened a private browsing tab and visited the Harvester UI. Clicked the Upgrade button, but received the same error. Force-refreshed the page and tried again, but received the error. (2) Consoled into one of the Harvester control plane nodes and deleted the three harvester webhook pods. Verified that they were recreated. Waited > 2 minutes. Opened a private browsing tab and visited the Harvester UI. Clicked the Upgrade button, but received the same error message. (3) From the control plane node console, I repeated the 
kubectl get bundledeployment -A
command as instructed. I did this for a few minutes, waited for awhile, performed it for a few more minutes, and repeated this process over an hour. At no point did anything display in the status column. This one is a real head scratcher.
g
any chance we could please have a new support bundle uploaded
a
I'll generate it and add it to the ticket.
g
thanks
2 Views
Open in Slack
PreviousNext
Powered by