I have a cluster of 6 nodes three master and three worker I rancher-users #longhorn-storage

Channels

Title

ripe-queen-73614

06/28/2022, 3:17 PM

I have a cluster of 6 nodes, three master and three worker. I want to use longhorm for persistent volume storage. On which nodes do I have to install longhorm?

✅ 1

icy-agency-38675

06/28/2022, 4:07 PM

You can check the official doc. By default, you don’t have to worry about it. Longhorn will help install the components on the all worker nodes.

🎯 1

ripe-queen-73614

06/29/2022, 9:09 AM

Thanks @icy-agency-38675, I asked because if it has to be installed on all the nodes, the nodes have to meet certain requirements, and a series of utilities have to be installed

https://longhorn.io/docs/1.3.0/deploy/install/#installation-requirements

icy-agency-38675

06/29/2022, 9:11 AM

Yes. You need to install the prerequisite packages before install longhorn system. The environment_check.sh script in the doc can help you check if the packages are installed or not.

ripe-queen-73614

06/29/2022, 9:56 AM

when launching the script in one of the masters it tells you that the prerequisites are not in the workers, then the prerequisites must be in all the nodes of the cluster, regardless of master or worker

icy-agency-38675

06/29/2022, 10:01 AM

The script should only check your workers node. But if your master nodes are also workers, it will also check them.

ripe-queen-73614

06/29/2022, 10:02 AM

then longhorm should not have installed it on the master but on the worker?

icy-agency-38675

06/29/2022, 10:06 AM

Longhorn only installs the components on the k8s’

worker

nodes. If your k8s master nodes are also workers nodes, longhorn will also install the components on those nodes because they are also worker nodes.

ripe-queen-73614

06/29/2022, 10:08 AM

ok, thank you very much @icy-agency-38675

icy-agency-38675

06/29/2022, 10:09 AM

Anytime. 🙂

ripe-queen-73614

06/29/2022, 10:16 AM

sorry so many questions

it's still too much

but I have a doubt, in the documentation it indicates "Create an Ingress with Basic Authentication (nginx)". I'm not sure if I have to do it

icy-agency-38675

06/29/2022, 10:20 AM

The purpose of the ingress setup is to allow external traffic to reach the Longhorn UI. I think you might need to setup it for using the longhorn system via UI.

ripe-queen-73614

06/29/2022, 10:21 AM

It can be accessed through ranchero right?

icy-agency-38675

06/29/2022, 10:21 AM

OK, if you install by rancher, you don’t need to setup the ingress.

You can access Longhorn UI on rancher.

ripe-queen-73614

06/29/2022, 10:22 AM

I have installed it with helm but if I give it there it takes me to the rancher interface

icy-agency-38675

06/29/2022, 10:24 AM

🤔I would suggest you uninstall Longhorn and install it via Rancher. Let the Rancher manage Longhorn app.

ripe-queen-73614

06/29/2022, 10:49 AM

I have uninstalled it with helm and it has deleted all the pods except one that is still working

icy-agency-38675

06/29/2022, 11:02 AM

longhorn-ui? That will be deleted automatically later.

ripe-queen-73614

06/29/2022, 11:02 AM

longhorn-ui- 1/1 Running 0 21h

if I select any tab of the ui it gives me request failed with status code 502

icy-agency-38675

06/29/2022, 12:37 PM

Can you check if all longhorn pods work correctly?

ripe-queen-73614

06/29/2022, 1:00 PM

no, only the pod longhorm-ui

uninstall and install again?

icy-agency-38675

06/29/2022, 1:01 PM

Did you install longhorn on rancher marketplace?

ripe-queen-73614

06/29/2022, 1:02 PM

yes

icy-agency-38675

06/29/2022, 1:03 PM

This is weird. But you can uninstall it. And check all longhorn pods are cleaned up first.

ripe-queen-73614

06/29/2022, 1:13 PM

I have uninstalled it and deleted the namespace because it did not delete it. When I install it again it gives me this error

helm upgrade --install=true --namespace=longhorn-system --timeout=10m0s --values=/home/shell/helm/values-longhorn-crd-100.1.2-up1.2.4.yaml --version=100.1.2+up1.2.4 --wait=true longhorn-crd /home/shell/helm/longhorn-crd-100.1.2-up1.2.4.tgz
Release "longhorn-crd" does not exist. Installing it now.
Error: rendered manifests contain a resource that already exists. Unable to continue with install: CustomResourceDefinition "<http://engines.longhorn.io|engines.longhorn.io>" in namespace "" exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: key "<http://meta.helm.sh/release-name|meta.helm.sh/release-name>" must equal "longhorn-crd": current value is "longhorn"

icy-agency-38675

06/29/2022, 1:27 PM

Do you mean you install and uninstall via rancher? And then encounter it when install again? Can you check with kubectl -n longhorn-system get crds?

ripe-queen-73614

06/29/2022, 1:31 PM

there are a lot of crds

icy-agency-38675

06/29/2022, 1:31 PM

Can you show that?

ripe-queen-73614

06/29/2022, 1:31 PM

<http://addons.k3s.cattle.io|addons.k3s.cattle.io>                             2022-05-31T11:50:57Z
<http://apiservices.management.cattle.io|apiservices.management.cattle.io>                 2022-06-28T09:22:52Z
<http://apps.catalog.cattle.io|apps.catalog.cattle.io>                           2022-06-28T09:22:52Z
<http://authconfigs.management.cattle.io|authconfigs.management.cattle.io>                 2022-06-28T09:22:53Z
<http://backingimagedatasources.longhorn.io|backingimagedatasources.longhorn.io>              2022-06-28T13:51:26Z
<http://backingimagemanagers.longhorn.io|backingimagemanagers.longhorn.io>                 2022-06-28T13:51:26Z
<http://backingimages.longhorn.io|backingimages.longhorn.io>                        2022-06-28T13:51:26Z
<http://backups.longhorn.io|backups.longhorn.io>                              2022-06-28T13:51:26Z
<http://backuptargets.longhorn.io|backuptargets.longhorn.io>                        2022-06-28T13:51:26Z
<http://backupvolumes.longhorn.io|backupvolumes.longhorn.io>                        2022-06-28T13:51:26Z
<http://bgpconfigurations.crd.projectcalico.org|bgpconfigurations.crd.projectcalico.org>          2022-05-31T11:51:32Z
<http://bgppeers.crd.projectcalico.org|bgppeers.crd.projectcalico.org>                   2022-05-31T11:51:32Z
<http://blockaffinities.crd.projectcalico.org|blockaffinities.crd.projectcalico.org>            2022-05-31T11:51:32Z
<http://clusterinformations.crd.projectcalico.org|clusterinformations.crd.projectcalico.org>        2022-05-31T11:51:32Z
<http://clusterregistrationtokens.management.cattle.io|clusterregistrationtokens.management.cattle.io>   2022-06-28T09:22:52Z
<http://clusterrepos.catalog.cattle.io|clusterrepos.catalog.cattle.io>                   2022-06-28T09:22:52Z
<http://clusters.management.cattle.io|clusters.management.cattle.io>                    2022-06-28T09:22:52Z
<http://engineimages.longhorn.io|engineimages.longhorn.io>                         2022-06-28T13:51:26Z
<http://engines.longhorn.io|engines.longhorn.io>                              2022-06-28T13:51:26Z
<http://features.management.cattle.io|features.management.cattle.io>                    2022-06-28T09:22:49Z
<http://felixconfigurations.crd.projectcalico.org|felixconfigurations.crd.projectcalico.org>        2022-05-31T11:51:32Z
<http://globalnetworkpolicies.crd.projectcalico.org|globalnetworkpolicies.crd.projectcalico.org>      2022-05-31T11:51:32Z
<http://globalnetworksets.crd.projectcalico.org|globalnetworksets.crd.projectcalico.org>          2022-05-31T11:51:32Z
<http://groupmembers.management.cattle.io|groupmembers.management.cattle.io>                2022-06-28T09:22:53Z
<http://groups.management.cattle.io|groups.management.cattle.io>                      2022-06-28T09:22:53Z
<http://helmchartconfigs.helm.cattle.io|helmchartconfigs.helm.cattle.io>                  2022-05-31T11:50:57Z
<http://helmcharts.helm.cattle.io|helmcharts.helm.cattle.io>                        2022-05-31T11:50:57Z
<http://hostendpoints.crd.projectcalico.org|hostendpoints.crd.projectcalico.org>              2022-05-31T11:51:32Z
<http://instancemanagers.longhorn.io|instancemanagers.longhorn.io>                     2022-06-28T13:51:26Z
<http://ipamblocks.crd.projectcalico.org|ipamblocks.crd.projectcalico.org>                 2022-05-31T11:51:32Z
<http://ipamconfigs.crd.projectcalico.org|ipamconfigs.crd.projectcalico.org>                2022-05-31T11:51:32Z
<http://ipamhandles.crd.projectcalico.org|ipamhandles.crd.projectcalico.org>                2022-05-31T11:51:32Z
<http://ippools.crd.projectcalico.org|ippools.crd.projectcalico.org>                    2022-05-31T11:51:32Z
<http://navlinks.ui.cattle.io|navlinks.ui.cattle.io>                            2022-06-28T09:22:52Z
<http://networkpolicies.crd.projectcalico.org|networkpolicies.crd.projectcalico.org>            2022-05-31T11:51:32Z
<http://networksets.crd.projectcalico.org|networksets.crd.projectcalico.org>                2022-05-31T11:51:32Z
<http://nodes.longhorn.io|nodes.longhorn.io>                                2022-06-28T13:51:26Z
<http://operations.catalog.cattle.io|operations.catalog.cattle.io>                     2022-06-28T09:22:52Z
<http://orphans.longhorn.io|orphans.longhorn.io>                              2022-06-28T13:51:26Z
<http://preferences.management.cattle.io|preferences.management.cattle.io>                 2022-06-28T09:22:52Z
<http://recurringjobs.longhorn.io|recurringjobs.longhorn.io>                        2022-06-28T13:51:26Z
<http://replicas.longhorn.io|replicas.longhorn.io>                             2022-06-28T13:51:26Z
<http://settings.longhorn.io|settings.longhorn.io>                             2022-06-28T13:51:26Z
<http://settings.management.cattle.io|settings.management.cattle.io>                    2022-06-28T09:22:52Z
<http://sharemanagers.longhorn.io|sharemanagers.longhorn.io>                        2022-06-28T13:51:26Z
<http://snapshots.longhorn.io|snapshots.longhorn.io>                            2022-06-28T13:51:26Z
<http://tokens.management.cattle.io|tokens.management.cattle.io>                      2022-06-28T09:22:53Z
<http://userattributes.management.cattle.io|userattributes.management.cattle.io>              2022-06-28T09:22:53Z
<http://users.management.cattle.io|users.management.cattle.io>                       2022-06-28T09:22:53Z
<http://volumes.longhorn.io|volumes.longhorn.io>                              2022-06-28T13:51:26Z

icy-agency-38675

06/29/2022, 1:38 PM

Oh. That's weird. There are v1.3.0 crds on your system. Looks the your previous uninstallation using helm was not successfully. Did you run ‘helm delete longhorn -n longhorn-system’ before?

And how did you install it at the first time?

ripe-queen-73614

06/29/2022, 1:40 PM

no, kubectl delete

what do you recommmend me to do

icy-agency-38675

06/29/2022, 1:50 PM

I see. At the first time. Did you install and uninstall both using kubectl? If yes, you need to uninstall by the instructions in https://longhorn.io/docs/latest/deploy/uninstall/

ripe-queen-73614

06/29/2022, 3:10 PM

the first time i installed with helm and uninstalled with kubectl

icy-agency-38675

06/29/2022, 3:12 PM

Got it… usually need to use the same tool. Maybe you can try to use helm delete clean them up

ripe-queen-73614

06/29/2022, 3:13 PM

clean cdr with helm delete?

icy-agency-38675

06/29/2022, 3:14 PM

Yes. helm delete longhorn -n longhorn-system

ripe-queen-73614

06/29/2022, 3:16 PM

Error: uninstall: Release not loaded: longhorn: release: not found

in the documentation it indicates to execute this to clean the cdr

Delete CRD finalizers, instances and definitions
for crd in $(kubectl get crd -o jsonpath={.items[*].metadata.name} | tr ' ' '\n' | grep <http://longhorn.rancher.io|longhorn.rancher.io>); do
  kubectl -n ${NAMESPACE} get $crd -o yaml | sed "s/\- <http://longhorn.rancher.io//g|longhorn.rancher.io//g>" | kubectl apply -f -
  kubectl -n ${NAMESPACE} delete $crd --all
  kubectl delete crd/$crd
done

icy-agency-38675

06/29/2022, 3:23 PM

Yes, You can try it for removing the CRDs

ripe-queen-73614

06/29/2022, 3:28 PM

do you have the command?

icy-agency-38675

06/29/2022, 3:36 PM

https://longhorn.io/docs/1.3.0/deploy/uninstall/#problems-with-crds You can replace ${NAMESPACE} with longhorn-system

ripe-queen-73614

06/29/2022, 4:00 PM

@icy-agency-38675 I have installed it again and it has been deployed correctly, but only one of the workers is scheduleable because the other two do not have enough space in the /var

the disk default-disk-fd0500000000(/var/lib/longhorn/) on the node ceslikwk01dy has 3565158400 available, but requires reserved 4506569932, minimal 25% to schedule more replicas

I have a dedicated drive for /storage how can I change it there instead of it being referenced in /var/lib/longhorm

icy-agency-38675

06/29/2022, 4:04 PM

Edit the node UI > Node page > edit node 1. Set the /var/lib/longhorn scheduling=false, eviction=true, and delete it. 2. Then, add a new disk on that node. And also set scheduling=true, eviction=false. More info can be found here https://longhorn.io/docs/1.3.0/volumes-and-nodes/multidisk/#add-a-disk

ripe-queen-73614

06/30/2022, 7:09 AM

@icy-agency-38675 when adding the new disk and deleting the /var/lib/longhorn how much space do I have to allocate to that disk? I understand that not all the disk that I have available right? Is there any recommendation? Or does it allocate the same thing that you were trying to reserve in /var/lib/longhorn?

icy-agency-38675

07/01/2022, 1:55 AM

@ripe-queen-73614 If the mount (/storage) is ready on your worker, you can add it to your node via Longhorn UI. For the settings of the disks, you can set them according to your application. You can refer to the official doc.

ripe-queen-73614

07/01/2022, 8:37 AM

@icy-agency-38675 I have the icsi and nfs pods crashing in the workers even though I have installed the components and the check script considers it valid

icy-agency-38675

07/01/2022, 8:39 AM

Do you want to install nfs and open-iscsi packages using these pod? Can you check the logs of the pod?

ripe-queen-73614

07/01/2022, 8:49 AM

I think it was because I installed it with yum and kubectl, I did the delete with kubectl

View count: 120

Back to #longhorn-storage Join thread in Slack