https://rancher.com/ logo
Title
b

busy-crowd-80458

07/13/2022, 5:31 PM
On 1.22.10+rke2r2, longhorn 1.2.4 I've got a bunch of issues like this:
MountVolume.WaitForAttach failed for volume "pvc-8c978b7c-28db-43e9-88d3-f4b43c532891" : volume pvc-8c978b7c-28db-43e9-88d3-f4b43c532891 has GET error for volume attachment csi-c1dd4b0084f8679ff1701abcdc314f2c594e85c0cfb9c287d954b7a6cf3ac4ba: volumeattachments.storage.k8s.io "csi-c1dd4b0084f8679ff1701abcdc314f2c594e85c0cfb9c287d954b7a6cf3ac4ba" is forbidden: User "system:node:emerald02" cannot get resource "volumeattachments" in API group "storage.k8s.io" at the cluster scope: no relationship found between node 'emerald02' and this object
also getting
Unable to attach or mount volumes: unmounted volumes=[media], unattached volumes=[kube-api-access-7qpqm config secrets netbox-tmp media]: timed out waiting for the condition
it looks like one node in particular is not working with Longhorn
this is on v1.22.10+rke2r2, longhorn 1.2.4
basically some of my Longhorn volumes are impossible to mount
after rebooting emerald02, things seemed to come good
a

aloof-hair-13897

07/14/2022, 12:21 AM
Hi, could please you provide the support bundle to let us find out what happened? Thanks.
b

busy-crowd-80458

07/14/2022, 3:08 AM
@aloof-hair-13897 sure, who do I send it to?
a

aloof-hair-13897

07/14/2022, 3:12 AM
You could send to longhorn-support-bundle@Suse.com or just attach to here. Thanks.
it's not quite out of the woods yet 😞 still got volumes randomly disconnecting.
...and going read only.
could it be because I'm using XFS disks?
a

aloof-hair-13897

07/14/2022, 3:31 AM
Longhorn only supports ext4 now
b

busy-crowd-80458

07/14/2022, 3:35 AM
facepalm that would do it
I read on a github issue that it had XFS support
a

aloof-hair-13897

07/14/2022, 3:40 AM
Sorry, it supports XFS, my bad.
b

busy-crowd-80458

07/14/2022, 3:47 AM
well
I've replaced two XFS bricks with ext4
and the cluster is getting healthier as I go
guessing XFS support is buggy, OR... Longhorn was somehow detecting my XFS volume as ext4 and putting ext4 volumes on it