https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# harvester
a
This message was deleted.
g
hard to say.. #general may be a better place for this
i suspect its likely because k3s/rke2 uses v2 provisioning which runs via k8s jobs, and those cant likely be created in a docker based install
q
i swear i got it to work once, but cant figure out how.
g
i am not questioning that.. but i am not across all of the rancher packaging
q
on a side note, i have this odd issue now where a pv is stuck in an attach/detach loop. i dont know why i keep having stability issues, but it's driving me bonkers... feel like taking a peep?
g
may be best to check in general
sure please share the support bundle
q
it's the pvc: pvc-4fe789bf-4d15-46f0-9318-acee7a122592 for the vm: vdi/josephpennella
for some reason, that pvc only had one valid replica too. found it on this node: harvester-03:/var/lib/harvester/extra-disks/533acf928b914b7fc49e5f104c76e131/replicas/pvc-4fe789bf-4d15-46f0-9318-acee7a122592-d3245087
g
those objects are in harvester-longhorn namespace
i dont think that is a default ns for support bundle info collection
can you please trigger another support bundle after updating the setting for support-bundle-kit namespaces
q
i have a longhorn bundle. do you want that, or should i pull another harvester one w/ harvester-longhorn added?
g
another harvester one would be nice since i have some tools to easily browse it
q
longhorn-system sound right?
g
the VM's are in namespace 
harvester-longhorn
q
i dont see one for harvester-longhorn
g
longhorn-system is one of the default ns
sorry its in vdi namespace
and that is included..
q
that vm is in vdi. and vid is in the list to inclue
g
my bad
that was the storage-class
q
no worries 🙂
g
you only have 1 replica for this pvc.. which is strange since storage class is the default one
q
yeah, i know. and i dont recall removing any. so i dont know what happened there.
i've also already rebooted the node too. fyi.
g
did you resize this volume?
Copy code
longhorn-manager-j85tj longhorn-manager 2023-02-20T04:57:47.472220687Z time="2023-02-20T04:57:47Z" level=warning msg="pvc-4fe789bf-4d15-46f0-9318-acee7a122592-e-11eb6755: time=\"2023-02-20T04:57:34Z\" level=warning msg=\"backend <tcp://10.52.0.223:10060> size does not match 107374182400 != 536870912000 in the engine initiation phase\""
q
found this in an engine-manager log too. not sure what to do about it.
2023-02-20T040206.983644495Z [pvc-4fe789bf-4d15-46f0-9318-acee7a122592-e-11eb6755] time="2023-02-20T040206Z" level=warning msg="backend tcp://10.52.0.139:10105 size does not match 107374182400 != 536870912000 in the engine initiation phase"
lol jinks?
it was 50gb, expanded to 100gb
if i remember correctly atleast
g
let me ask our longhorn team too
how did you resize the pvc?
q
wait... this volume is 100gb, becuase i created it last week w/ dd when i cloned it form one vol to another. so it should be 100gb
i basically did a bunch of dd cloning last week because i didnt like the way the backing image were working.
Copy code
NAME                          STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS         AGE
josephpennella-disk-2-8r3nb   Bound    pvc-4fe789bf-4d15-46f0-9318-acee7a122592   100Gi      RWX            harvester-longhorn   7d10h
g
q
Copy code
-rw------- 1 root root         4096 Feb 19 22:32 revision.counter
-rw-r--r-- 1 root root 536870912000 Feb 19 22:33 volume-head-002.img
-rw-r--r-- 1 root root          161 Feb 19 22:33 volume-head-002.img.meta
-rw-r--r-- 1 root root 107374182400 Feb 19 22:32 volume-snap-expand-536870912000.img
-rw-r--r-- 1 root root          210 Feb 19 22:33 volume-snap-expand-536870912000.img.meta
-rw-r--r-- 1 root root 107374182400 Feb 19 22:32 volume-snap-f794fa03-6bea-4365-8e3c-863a013f00d8.img
-rw-r--r-- 1 root root          158 Feb 19 17:53 volume-snap-f794fa03-6bea-4365-8e3c-863a013f00d8.img.meta
-rw-r--r-- 1 root root          179 Feb 20 05:37 volume.meta
g
let me ask the longhorn team
q
good find on the bug. i didnt see anything when i searched. bad google-fu tonight i guess 😞
messing w/ it. i'll let you know.
so it seems in the case of the bug, it was able to recover because of the other two replicas, but with me having only one, i'm not sure what to do.
g
can you please provide me the longhorn bundle
or upload it on the same issue you commented on?
q
sure, here you go. the PVC in question is: pvc-4fe789bf-4d15-46f0-9318-acee7a122592 for the vm: vdi/jp-old.
longhorn-support-bundle_b0b85b06-bd06-434b-95e8-c47b11be903f_2023-02-21T14-08-26Z.zip
i'll post on that comment as well.
8 Views
Open in Slack
PreviousNext
Powered by