This message was deleted.

many containerized environments on a single VM

Why not running them directly as native workloads in K8s? If you want a UI, you can use the embedded Rancher for that.

That’s a fair point — for many cases, running them as native workloads in Kubernetes works well. What we found is that for high-density dev environments, the overhead (pods, networking, resource isolation) can add up quickly, especially when scaling to many concurrent users. We’re experimenting with a lighter-weight approach (LXC-style environments on a single VM) to optimize for density and cost, while keeping centralized monitoring and security checks. So less about replacing K8s, more about exploring a different point in the tradeoff space.

In my experience have a little bit of separation goes a long way. Decoupling infra from dev and prod means you can upgrade and manage differently. I would even suggest separate hardware for infra.

That makes a lot of sense — especially for production. Separation between infra/dev/prod definitely helps with stability and upgrade cycles. What we were exploring is slightly different: within a dev-only environment, how far we can push density and resource utilization when isolation requirements are a bit more relaxed. In that context, we found ourselves trading some of that separation for efficiency — especially when running large numbers of short-lived dev sandboxes on shared hardware.

Incus is interesting as a sorta all-in-one opinionated answer to this. ymmv