https://rancher.com/ logo
Powered by
Title
r

ripe-queen-73614

07/13/2022, 7:26 PM
Hi all, I've been stuck for a week that elasticsearch (ECK) doesn't work for me when using storage longhorm. The volumes are attached and unattached and the logs of the pods show this: [17:52] Warning FailedAttachVolume 10s attachdetach-controller AttachVolume.Attach failed for volume "pvc-70a16c86-2e98-4a7e-a306-ef7238969f85" : rpc error: code = DeadlineExceeded desc = vol
a

aloof-hair-13897

07/14/2022, 12:34 AM
Hi, 1. Do you have all requirements #installation-requirements 2. Could you check if 
open-iscsi
is installed and 
iscsid
is running on all work nodes? Try 
systemctl status iscsid.
3. Could you please provide us a support bundle for further investigation? You can generate a support bundle on Longhorn UI.
r

ripe-queen-73614

07/14/2022, 11:51 AM
@cuddly-vase-67379 iscid is running on all work nodes. doing a describe of the pod shows me the following traces:
Warning  FailedAttachVolume  25m (x33 over 3h45m)    attachdetach-controller  AttachVolume.Attach failed for volume "pvc-xxx" : rpc error: code = Aborted desc = volume pvc-xxxxx is not ready for workloads
  Warning  FailedMount         19m (x20 over 3h29m)    kubelet                  Unable to attach or mount volumes: unmounted volumes=[volv], unattached volumes=[kube-api-access-js429 volv]: timed out waiting for the conditio
a

aloof-hair-13897

07/14/2022, 12:54 PM
Could you please provide us a support bundle for further investigation? You can generate a support bundle on Longhorn UI.
r

ripe-queen-73614

07/14/2022, 1:11 PM
I have tried but point 5 does not work for me
https://www.suse.com/support/kb/doc/?id=000020145
a

aloof-hair-13897

07/14/2022, 1:19 PM
You could find 
Generate Support Bundle
link in the Bottom of the UI.
r

ripe-queen-73614

07/14/2022, 1:51 PM
Done!
a

aloof-hair-13897

07/14/2022, 1:55 PM
Could you attach the support bundle file here?
r

ripe-queen-73614

07/14/2022, 1:58 PM
Can I send it to an email address, for security?
a

aloof-hair-13897

07/14/2022, 2:00 PM
Yes, please mail to longhorn-support-bundle@suse.com, Thanks.
r

ripe-queen-73614

07/14/2022, 2:17 PM
sent the mail @cuddly-vase-67379 please any update I would appreciate you tell me
a

aloof-hair-13897

07/14/2022, 3:33 PM
Hi @ripe-queen-73614, I saw 
"Failed to init frontend: device pvc-db03aac2-7027-45fe-b86c-e38e85b74ad9: fail to delete target iqn.2019-10.io.longhorn:pvc-db03aac2-7027-45fe-b86c-e38e85b74ad9"
this log in the instance-manager. Could you check that 
tgtd
service status and is tgtd busy? BTW I noticed that you built the system on RHEL8.6 , that may have some potential risk because it is not verified yet for Longhorn system.
r

ripe-queen-73614

07/14/2022, 3:42 PM
how do i check that tgtd?
@cuddly-vase-67379 how do i check that tgtd?
a

aloof-hair-13897

07/15/2022, 12:36 AM
You could use 
top
or 
atop
to see if there any process is busy, usually 
systemctl status tgtd
to check but RHEL8.6 seems use the 
target
service instead.
Ref: https://github.com/longhorn/longhorn/issues/923 Maybe you could log into the corresponding engine manager pod, and manually delete the target first.
r

ripe-queen-73614

07/15/2022, 10:32 AM
@cuddly-vase-67379 I don't see this process
systemctl status tgtd Unit tgtd.service could not be found.
f

famous-journalist-11332

07/26/2022, 9:50 PM
Can you try to scale down the workload that are using Longhorn volumes and delete the instance-manager-e-xxx pods inside 
longhorn-system
namespace?
36 Views
#longhorn-storageJoin Slack
Powered by