Has anyone seen the following error when spinning up Longhor rancher-users #longhorn-storage

Title

millions-gold-16761

01/24/2023, 9:46 PM

Has anyone seen the following error when spinning up Longhorn? I'm starting it from Rancher (itself spun up from latest) via the Helm chart. I'll put a few extra details in thread, as I've fiddled with it a bit.

spec: failed to generate spec: path "/var/lib/longhorn/" is mounted on "/" but it is not a shared mount

From what I've seen on the internet, it seems like it might be related to the host, so I've added the following to set all mounts to

shared

(as well as

rshared

)

shared-mounted.start.sh

I also modified

/lib/systemd/system/docker.service

to use

MountFlags

so that it'd be able to use it.

This is an experimental sandbox/toy server setup, running Ubuntu 22, so there's a lot of non-optimal things. Like how docker and containerd are running on ZFS zvols. I'm also early enough in the setup that I can start over to try other things out. (Aside from switching from ZFS.)

Here's where the Rancher dashboard says Longhorn's at. Image 2:

CreateContainerError (failed to generate container "6f8e6dfe2d8bbe8ce3f849cc50c4be90dc727ee7855d189de51dce2294a59e9a" spec: failed to generate spec: path "/var/lib/longhorn/" is mounted on "/" but it is not a shared mount)

alert-policeman-61846

01/25/2023, 6:28 PM

Yes, solved this issue on ubuntu with

mount --make-rshared /

millions-gold-16761

01/25/2023, 6:32 PM

findmnt -o TARGET,PROPAGATION /

says it's currently just

shared

, but the start script says for

rshared

, so I'm restarting the server now to see if it's just somehow not been rebooted since it took effect

Hmm, even after restart, it's still only set to

shared

. Even after manually setting it, it still only shows

shared

. So I'll have to hunt down if some other nonsense is going on.

famous-journalist-11332

01/26/2023, 6:17 AM

It seems that Longhorn doesn't play well with ZFS filesystem on the node. Could you use XFS or EXT4 on the host's disk? @millions-gold-16761

Ref: https://github.com/longhorn/longhorn/issues/533#issuecomment-492415902

And https://longhorn.io/docs/1.4.0/deploy/install/#:~:text=The%20host%20filesystem,XFS

millions-gold-16761

01/26/2023, 1:45 PM

It's a bit frustrating that mounting a

zvol

formatted to

ext4

doesn't do it. That fixed the weird k3s snapshot issues and finally got the Rancher manager docker container to do anything. I guess I'll have to do a full server rebuild and reformat the drive arrays.

👍 1

famous-journalist-11332

01/26/2023, 11:01 PM

I guess I'll have to do a full server rebuild and reformat the drive arrays.

Please update us the result

87 Views

#longhorn-storage Join Slack