what is the best practice in terms of where to install rancher vm to manage harvester cluster? is it ok to install the rancher vm on top of the harvester it will be managing? any chicken/egg concern in doing that?

In general, it's recommended to install Rancher on a separate machine or VM from the Harvester cluster that it will be managing. This ensures that Rancher can effectively monitor and manage the cluster without being impacted by any issues that may arise on the Harvester nodes.

Thanks. This is different from what we normally do for vmware vm mgmt cluster, we normally just install vcenter appliance on top of its own mgmt cluster. This would require us dedicating another different physical host running docker, just to run rancher application on that host later.