anyone else using harvester w/ containerdisk VMs? has some cool features we want but wondering if I've missed any footguns. everything appears to work fine but can't find much mention of anyone else doing it

Mostly we haven't had a strong use case for them. There hasn't been many containers that have X thing that is more useful in a VM vs a standard k8s cluster. I thought it was a cool idea, but a lot of our VMs are more pets than cattle. (HigherEd/research etc)

yeah, benefits for us are we can define/build VM from Dockerfile + push/deploy via OCI registry same as everything else, plus they're effectively immutable so you're forced to define what VM state you want to persist

But isn't the disk just stored in a different format then (.img / .raw vs layered container filesystem)? Not really seeing the benefits?

at the end of the day it's all just 1s and 0s. ymmv

I think the container images are cached and tmp vs qcow images that are persistent and block devices.

yeah the actual container image is just scratch + qcow2, but then it can be distributed via oci registry, upgrade VM OS is just bump the image ref, don't need to solve getting qcow2 into harvester and swapping out the OS on a VM, etc

vs longhorn where you have to flatten a disk and basically migrate the data to a new storage class to actually delete an image.

yeah our original OS was rancher k3os but that's long dead so we want similar functionality. elemental theoretically solves this but I found complicated to get the full lifecycle working

We're running elemental on bare metal, but in VMs would add some weird abstraction. In general I like the immutable OSes, (Fedora core/atomic, slemicro, etc) but I can see the usecase for container images. I just don't have any appropriate workloads.