Just to wrap up this thread, we worked with some Rancher engineers to track this down, the high memory usage and the DeltaFIFO errors posted above, and we believe it's from this bug - https://github.com/longhorn/longhorn/issues/6866 We were running Longhorn 1.5.3, and this is fixed in 1.5.4. I went ahead and upgraded our production cluster to 1.62 and so far things seem better, but it's something I'll have to keep an eye on. It took about 6 months of running 1.5.3 for this to manifest, but I don't plan on going that long between upgrades in the future.