full-toddler-53694
10/07/2022, 7:02 AMbig-judge-33880
10/07/2022, 11:18 AMgreat-photographer-94826
10/10/2022, 1:48 PM/dev/longhorn
Is there a way to change this path? I didn't find any arg in Helm chart vaules file (https://github.com/longhorn/charts/blob/master/charts/longhorn/values.yaml).late-needle-80860
10/10/2022, 2:02 PMVolumeSnapshotClass
• Longhorn backs up and is configured to integrate with MinIO to store the data …
We’re seeing a lot of storage usage and after having read > https://longhorn.io/docs/1.3.1/volumes-and-nodes/volume-size/ I’m thinking about if it’ll be beneficial to configure a snapshot recurring job
with a retention of 1
in order to get these snapshots taking up storage space to reclaim disk space.
As my thinking/understanding is that the data is stored on MinIO anyways and Velero
tracks the data on MinIO and on restore will trigger Longhorn
- again over CSI
VolumeSnapshot
for restore … and the data will be fetched from MinIO
.
So the data in all these snapshots
that one can see on the sub-volume page for a specific Volume
can be reclaimed and we still have backup data in the case we need it for restore.
Is this correctly understood? And thank you very much.quiet-umbrella-37745
10/11/2022, 10:16 AMUnable to attach or mount volumes: unmounted volumes=[<app>-pvc]
AttachVolume.Attach failed for volume "<app>" : rpc error: code = InvalidArgument desc = volume <app> invalid frontend type iscsi
I found this page that may have some more information: https://docs.ranchermanager.rancher.io/v2.5/how-to-guides/advanced-user-guides/manage-cluste[…]orage/manage-persistent-storage/install-iscsi-volumes
and the next instruction 'After installing the initiator tool on your nodes, edit the YAML for your cluster, editing the kubelet configuration to mount the iSCSI binary and configuration, as shown in the sample below'
I don't know which YAML it is referring to, any help here would be great, thanks in advancelate-needle-80860
10/14/2022, 12:29 PMflaky-coat-75909
10/17/2022, 1:35 PM8.0G pvc-48586aa6-cc79-40e0-94b1-bf20e48ffa72-01eb5a76
28G pvc-48586aa6-cc79-40e0-94b1-bf20e48ffa72-6f9f75b9
8.5G pvc-48586aa6-cc79-40e0-94b1-bf20e48ffa72-ca55e8c5
on single node to one PVC?careful-australia-9472
10/18/2022, 12:17 AMastonishing-electrician-27820
10/18/2022, 11:27 AME1018 11:12:14.544813 1 reflector.go:126] <http://github.com/longhorn/longhorn-manager/k8s/pkg/client/informers/externalversions/factory.go:117|github.com/longhorn/longhorn-manager/k8s/pkg/client/informers/externalversions/factory.go:117>: Failed to list *v1beta1.Setting: the server could not find the requested resource (get <http://settings.longhorn.io|settings.longhorn.io>)
E1018 11:12:14.745037 1 reflector.go:126] <http://github.com/longhorn/longhorn-manager/k8s/pkg/client/informers/externalversions/factory.go:117|github.com/longhorn/longhorn-manager/k8s/pkg/client/informers/externalversions/factory.go:117>: Failed to list *v1beta1.Engine: the server could not find the requested resource (get <http://engines.longhorn.io|engines.longhorn.io>)
E1018 11:12:14.944640 1 reflector.go:126] <http://github.com/longhorn/longhorn-manager/k8s/pkg/client/informers/externalversions/factory.go:117|github.com/longhorn/longhorn-manager/k8s/pkg/client/informers/externalversions/factory.go:117>: Failed to list *v1beta1.Replica: the server could not find the requested resource (get <http://replicas.longhorn.io|replicas.longhorn.io>)
E1018 11:12:15.144968 1 reflector.go:126] <http://github.com/longhorn/longhorn-manager/k8s/pkg/client/informers/externalversions/factory.go:117|github.com/longhorn/longhorn-manager/k8s/pkg/client/informers/externalversions/factory.go:117>: Failed to list *v1beta1.InstanceManager: the server could not find the requested resource (get <http://instancemanagers.longhorn.io|instancemanagers.longhorn.io>)
E1018 11:12:15.367282 1 reflector.go:126] <http://github.com/longhorn/longhorn-manager/k8s/pkg/client/informers/externalversions/factory.go:117|github.com/longhorn/longhorn-manager/k8s/pkg/client/informers/externalversions/factory.go:117>: Failed to list *v1beta1.EngineImage: conversion webhook for <http://longhorn.io/v1beta2|longhorn.io/v1beta2>, Kind=EngineImage failed: Post "<https://longhorn-conversion-webhook.longhorn-system.svc:9443/v1/webhook/conversion?timeout=30s>": service "longhorn-conversion-webhook" not found
So some webhook was deleted before it should have been?
Further evident when I try to remove the finalizers from some objects manually,
~ kubectl edit <http://nodes.longhorn.io|nodes.longhorn.io> -n longhorn-system ✔ red ⎈
error: <http://nodes.longhorn.io|nodes.longhorn.io> "mcsh1" could not be patched: Internal error occurred: failed calling webhook "<http://validator.longhorn.io|validator.longhorn.io>": Post "<https://longhorn-admission-webhook.longhorn-system-2.svc:9443/v1/webhook/validaton?timeout=10s>": service "longhorn-admission-webhook" not found
error: <http://nodes.longhorn.io|nodes.longhorn.io> "mcsh2" could not be patched: Internal error occurred: failed calling webhook "<http://validator.longhorn.io|validator.longhorn.io>": Post "<https://longhorn-admission-webhook.longhorn-system-2.svc:9443/v1/webhook/validaton?timeout=10s>": service "longhorn-admission-webhook" not found
error: <http://nodes.longhorn.io|nodes.longhorn.io> "pfdc-store-2" could not be patched: Internal error occurred: failed calling webhook "<http://validator.longhorn.io|validator.longhorn.io>": Post "<https://longhorn-admission-webhook.longhorn-system-2.svc:9443/v1/webhook/validaton?timeout=10s>": service "longhorn-admission-webhook" not found
error: <http://nodes.longhorn.io|nodes.longhorn.io> "ryzen1" could not be patched: Internal error occurred: failed calling webhook "<http://validator.longhorn.io|validator.longhorn.io>": Post "<https://longhorn-admission-webhook.longhorn-system-2.svc:9443/v1/webhook/validaton?timeout=10s>": service "longhorn-admission-webhook" not found
All and all, I have these resources left based on the status of this NS
- lastTransitionTime: "2021-11-04T05:49:28Z"
message: 'Discovery failed for some groups, 3 failing: unable to retrieve the
complete list of server APIs: <http://subresources.kubevirt.io/v1|subresources.kubevirt.io/v1>: the server is currently
unable to handle the request, <http://upload.cdi.kubevirt.io/v1alpha1|upload.cdi.kubevirt.io/v1alpha1>: the server is
currently unable to handle the request, <http://upload.cdi.kubevirt.io/v1beta1|upload.cdi.kubevirt.io/v1beta1>: the
server is currently unable to handle the request'
How do I completely remove longhorn from here so I can reinstall it on my cluster?ancient-raincoat-46356
10/19/2022, 5:11 PMlocal-path
was patched to unset it as a "default", but I have noticed that over time the local-path
comes back as a default storage class. I've unset it several times now but I'd like it to stop reverting back to that state. Does anyone know how I can accomplish this or have you seen this yourself?great-photographer-94826
10/20/2022, 7:26 AMflaky-coat-75909
10/21/2022, 8:05 AMcareful-truck-98666
10/21/2022, 11:40 AMsticky-summer-13450
10/24/2022, 11:37 AMcareful-truck-98666
10/24/2022, 2:55 PME1024 14:49:44.621450 1 volume_controller.go:216] fail to sync longhorn-system/pvc-336ed0a5-1889-4e0a-be69-e9cd6602b4af: fail to reconcile volume state for pvc-336ed0a5-1889-4e0a-be69-e9cd6602b4af: volume pvc-336ed0a5-1889-4e0a-be69-e9cd6602b4af has already attached to node kube-w-04, but asked to attach to node kube-w-03
time="2022-10-24T14:49:44Z" level=warning msg="Dropping Longhorn volume longhorn-system/pvc-336ed0a5-1889-4e0a-be69-e9cd6602b4af out of the queue" controller=longhorn-volume error="fail to sync longhorn-system/pvc-336ed0a5-1889-4e0a-be69-e9cd6602b4af: fail to reconcile volume state for pvc-336ed0a5-1889-4e0a-be69-e9cd6602b4af: volume pvc-336ed0a5-1889-4e0a-be69-e9cd6602b4af has already attached to node kube-w-04, but asked to attach to node kube-w-03" node=kube-w-03
How can I tell it that it's not attached to w-04?late-needle-80860
10/28/2022, 9:13 PMfull-fireman-68072
10/29/2022, 1:34 AMfull-fireman-68072
10/29/2022, 1:37 AMsticky-summer-13450
10/30/2022, 2:24 PMdamp-magician-5939
11/04/2022, 1:48 PMpurple-mechanic-99423
11/07/2022, 8:03 PMbright-fireman-42144
11/10/2022, 10:15 PMNode
tab, select one of the nodes, and select Edit Disks
in the dropdown menu."
any ideas... command line or gui is fine.steep-furniture-72588
11/13/2022, 1:52 AMbright-fireman-42144
11/13/2022, 5:43 PMbright-fireman-42144
11/13/2022, 8:39 PMbulky-forest-9145
11/16/2022, 8:11 PM$ kubectl get all -n longhorn-system
No resources found in longhorn-system namespace.
$ kubectl get namespace longhorn-system
NAME STATUS AGE
longhorn-system Terminating 104m
should I just rebuild the cluster? 😢red-printer-88208
11/17/2022, 2:47 PM-> df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 6.0G 1.5G 4.5G 24% /
devtmpfs 7.9G 0 7.9G 0% /dev
/dev/sda2 93M 32M 55M 37% /media/hdCnf
/dev/sda8 43G 6.7G 34G 17% /media/hdVar
/dev/sda5 93M 28K 86M 1% /media/hdUser
/dev/loop0 2.0G 13M 1.9G 1% /media/loopfs
cgroup 7.9G 0 7.9G 0% /sys/fs/cgroup
/dev/sda7 100M 69M 31M 69% /boot
/dev/sda1 200M 69M 131M 35% /run/media/sda1
/dev/mapper/vg01-longhorn 177G 29M 168G 1% /var/lib/longhorn
But now the PV are not getting attached to the pods and keeps giving error message failed to attach the volume.
Any idea why this might happening.orange-state-45356
11/21/2022, 8:10 PMbrief-rainbow-54144
11/25/2022, 9:12 PMfailed calling webhook \"<http://validator.longhorn.io|validator.longhorn.io>\": failed to call webhook: Post \"<https://longhorn-admission-webhook.longhorn-system.svc:9443/v1/webhook/validaton?timeout=10s>
Any pointers welcome - I'm about to fire up a dnsutils pod to look at the DNS now.sticky-summer-13450
11/28/2022, 12:43 PM