This kubevirt doc might help you:
https://kubevirt.io/user-guide/compute/virtual_hardware/#enabling-default-cluster-cpu-model
Harvester (and Kubevirt) would set up the cpuModel on the VM based on "host model", which is problematic in a heterogeneous cluster.
You need to set up a default cpu model for the cluster, which is compatible with both "cascadelake" and "skylake". There is however around these Intel generations, some limitations in CPU features, that you need to check out (if I remember well, there is a feature that was introduced in Skylake and later dropped). You best bet might be "Broadwell".
After that, you might not have all the latest CPU features available to your VM, but live migration should be possible and Broadwell is a "recent-enough" CPU generation for virtualization.