https://rancher.com/ logo
Powered by
Title
s

stale-painting-80203

01/05/2023, 5:04 PM
Need help with using the CSI driver: In an RKE2 cluster running on Harvester I would like to allow the cluster to create volumes on the longhorn storage created by harvester. I was told that I can use the CSI driver as per https://docs.harvesterhci.io/v1.1/rancher/csi-driver/#deploying-with-harvester-k3s-node-driver. Unfortunately I am unable to get this to work because I created the cluster using Rancher's "Use existing nodes and create a cluster using RKE2/K3s" . Apparently if you already have an existing cluster you can use the method described for "Harvester K3s Node Driver". I am able to deploy the driver, but when the cluster tries to create the volume it fails even though I can see the correct storage driver. 
kubectl describe sc
Name:                  harvester
IsDefaultClass:        Yes
Annotations:           <http://meta.helm.sh/release-name=harvester-csi-driver,meta.helm.sh/release-namespace=kube-system,storageclass.kubernetes.io/is-default-class=true|meta.helm.sh/release-name=harvester-csi-driver,meta.helm.sh/release-namespace=kube-system,storageclass.kubernetes.io/is-default-class=true>
Provisioner:           <http://driver.harvesterhci.io|driver.harvesterhci.io>
Parameters:            <none>
AllowVolumeExpansion:  True
MountOptions:          <none>
ReclaimPolicy:         Delete
VolumeBindingMode:     Immediate
Events:                <none>
b

billowy-painting-56466

01/06/2023, 1:26 AM
• What are the Longhorn components' statuses? 
kubectl -n longhorn-system get pod
• Are there any errors in the Longhorn manager logs? • What is the Longhorn version? You can also provide the support bundle so we can have a better context.
I would like to allow the cluster to create volumes on the longhorn storage created by harvester.
And have you checked with Harvester folks about 
<http://driver.harvesterhci.io|driver.harvesterhci.io>
?
s

stale-painting-80203

01/06/2023, 1:39 AM
kubectl -n longhorn-system get pod
NAME                                           READY   STATUS    RESTARTS       AGE
backing-image-manager-d7ad-e02f                1/1     Running   0              52d
csi-attacher-5d4cc8cfc8-khv5x                  1/1     Running   2 (52d ago)    52d
csi-attacher-5d4cc8cfc8-np2dw                  1/1     Running   2 (52d ago)    52d
csi-attacher-5d4cc8cfc8-qcrrk                  1/1     Running   2 (52d ago)    52d
csi-provisioner-77b757f445-7b82n               1/1     Running   2 (52d ago)    52d
csi-provisioner-77b757f445-ksq67               1/1     Running   2 (52d ago)    52d
csi-provisioner-77b757f445-pngtj               1/1     Running   2 (52d ago)    52d
csi-resizer-76fdffd797-5r49l                   1/1     Running   2 (52d ago)    52d
csi-resizer-76fdffd797-dj48h                   1/1     Running   2 (52d ago)    52d
csi-resizer-76fdffd797-fzjxv                   1/1     Running   2 (52d ago)    52d
csi-snapshotter-6c4d5c4fc9-4hzhh               1/1     Running   2 (52d ago)    52d
csi-snapshotter-6c4d5c4fc9-vnhwk               1/1     Running   2 (52d ago)    52d
csi-snapshotter-6c4d5c4fc9-z2c78               1/1     Running   2 (52d ago)    52d
engine-image-ei-a5371358-82b4m                 1/1     Running   2 (52d ago)    52d
instance-manager-e-ba86e369                    1/1     Running   0              52d
instance-manager-r-cb3f0ddb                    1/1     Running   0              52d
longhorn-admission-webhook-7c7d9b9dd9-9vjgq    1/1     Running   2 (52d ago)    52d
longhorn-admission-webhook-7c7d9b9dd9-jn6lz    1/1     Running   2 (52d ago)    52d
longhorn-conversion-webhook-5f86fd8fcc-5qz5s   1/1     Running   2 (52d ago)    52d
longhorn-conversion-webhook-5f86fd8fcc-njpkd   1/1     Running   2 (52d ago)    52d
longhorn-csi-plugin-hmrlg                      2/2     Running   10 (52d ago)   52d
longhorn-driver-deployer-78fcc6fcfb-d2sls      1/1     Running   2 (52d ago)    52d
longhorn-loop-device-cleaner-5qdm6             1/1     Running   2 (52d ago)    52d
longhorn-manager-8bpct                         1/1     Running   2 (52d ago)    52d
longhorn-ui-bf9cbc848-6gbwf                    1/1     Running   8 (52d ago)    52d
not sure how I look for any errors
longhorn dashboard shows v1.3.2.
Harvester folks seem to indicate that the driver should work. https://github.com/harvester/harvester/issues/3308
b

billowy-painting-56466

01/06/2023, 1:45 AM
• Can you test Longhorn with simple workload? • Can you provide the support-bundle from the Longhorn dashboard?
Harvester folks seem to indicate that the driver should work.
Asking because I have no idea about 
<http://driver.harvesterhci.io|driver.harvesterhci.io>
. Longhorn driver looks like 
<http://driver.longhorn.io|driver.longhorn.io>
.
s

stale-painting-80203

01/06/2023, 2:34 AM
longhorn seems to be working from the harvester cluster as the VM's I create have their volumes on longhorn
never created a support bundle in longhorn before. What should be the Issue URL?
b

billowy-painting-56466

01/06/2023, 2:44 AM
longhorn seems to be working from the harvester cluster as the VM's I create have their volumes on longhorn
The issue URL is an optional field.
longhorn seems to be working from the harvester cluster as the VM's I create have their volumes on longhorn
it sound like Longhorn is functioning correctly.
when the cluster tries to create the volume it fails
Any errors? Did the volume failed to at Harvester or reached Longhorn then failed? Can you see the failed volume in Longhorn: 
kubectl -n longhorn-system get volumes
?
You can also generate Harvester support bundle, it should also collect Longhorn support bundle.
s

stale-painting-80203

01/06/2023, 6:37 PM
I am trying to deploy an application called Nextcloud, which creates two volumes. This works fine if I install longhorn in the cluster, but if I try to use the CSI driver the volumes do not get created on harvester longhorn. 
kubectl get pv -n nextcloud
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                STORAGECLASS   REASON   AGE
pvc-45afd47c-e98f-4406-a8f4-ee262de07b07   8Gi        RWO            Delete           Bound    nextcloud/data-nextcloud-mariadb-0   harvester               94s
pvc-df44b4f6-6df5-4354-b4ea-a05c19cae83a   8Gi        RWO            Delete           Bound    nextcloud/nextcloud-nextcloud        harvester               94s

kubectl get pvc -n nextcloud
NAME                       STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
data-nextcloud-mariadb-0   Bound    pvc-45afd47c-e98f-4406-a8f4-ee262de07b07   8Gi        RWO            harvester      4m3s
nextcloud-nextcloud        Bound    pvc-df44b4f6-6df5-4354-b4ea-a05c19cae83a   8Gi        RWO            harvester      4m4s
kubectl describe pv -n nextcloud
Name:            pvc-45afd47c-e98f-4406-a8f4-ee262de07b07
Labels:          <none>
Annotations:     <http://pv.kubernetes.io/provisioned-by|pv.kubernetes.io/provisioned-by>: <http://driver.harvesterhci.io|driver.harvesterhci.io>
Finalizers:      [<http://kubernetes.io/pv-protection|kubernetes.io/pv-protection> external-attacher/driver-harvesterhci-io]
StorageClass:    harvester
Status:          Bound
Claim:           nextcloud/data-nextcloud-mariadb-0
Reclaim Policy:  Delete
Access Modes:    RWO
VolumeMode:      Filesystem
Capacity:        8Gi
Node Affinity:   <none>
Message:         
Source:
    Type:              CSI (a Container Storage Interface (CSI) volume source)
    Driver:            <http://driver.harvesterhci.io|driver.harvesterhci.io>
    FSType:            ext4
    VolumeHandle:      pvc-45afd47c-e98f-4406-a8f4-ee262de07b07
    ReadOnly:          false
    VolumeAttributes:      <http://storage.kubernetes.io/csiProvisionerIdentity=1672469997027-8081-driver.harvesterhci.io|storage.kubernetes.io/csiProvisionerIdentity=1672469997027-8081-driver.harvesterhci.io>
Events:                <none>


Name:            pvc-df44b4f6-6df5-4354-b4ea-a05c19cae83a
Labels:          <none>
Annotations:     <http://pv.kubernetes.io/provisioned-by|pv.kubernetes.io/provisioned-by>: <http://driver.harvesterhci.io|driver.harvesterhci.io>
Finalizers:      [<http://kubernetes.io/pv-protection|kubernetes.io/pv-protection> external-attacher/driver-harvesterhci-io]
StorageClass:    harvester
Status:          Bound
Claim:           nextcloud/nextcloud-nextcloud
Reclaim Policy:  Delete
Access Modes:    RWO
VolumeMode:      Filesystem
Capacity:        8Gi
Node Affinity:   <none>
Message:         
Source:
    Type:              CSI (a Container Storage Interface (CSI) volume source)
    Driver:            <http://driver.harvesterhci.io|driver.harvesterhci.io>
    FSType:            ext4
    VolumeHandle:      pvc-df44b4f6-6df5-4354-b4ea-a05c19cae83a
    ReadOnly:          false
    VolumeAttributes:      <http://storage.kubernetes.io/csiProvisionerIdentity=1672469997027-8081-driver.harvesterhci.io|storage.kubernetes.io/csiProvisionerIdentity=1672469997027-8081-driver.harvesterhci.io>
Events:                <none>
I can't see the above volumes in longhorn
b

billowy-painting-56466

01/09/2023, 3:14 AM
Doesn't look like volume creation reached Longhorn. You need to check with the harvester folks.
107 Views
#longhorn-storageJoin Slack
Powered by