https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# harvester
a
This message was deleted.
s
It seems that I need to understand Longhorn and nurse it along in order to keep Harvester v1.0.2 working. But I really don’t understand Longhorn, and I’m very worried that I may screw things up through my ignorance.
deleted
I upgraded Harvester from v1.0.2 to v1.0.3, but I still have the issue I mentioned with one VM. It cannot be started or be backed-up. Longhorn is constantly looping - attaching and detaching the volume. Two of the instance managers are doing the same thing constantly. Any idea what's going on, or should I take this query to the #longhorn-storage channel? This is from an instance-manager's logs...
Copy code
2022-09-06T15:14:04+01:00 time="2022-09-06T14:14:04Z" level=info msg="Listening on sync 0.0.0.0:10092"
2022-09-06T15:14:04+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:04Z" level=info msg="Process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6 has started at localhost:10090"
2022-09-06T15:14:05+01:00 [pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6] time="2022-09-06T14:14:05Z" level=info msg="New connection from: 10.52.2.235:58878"
2022-09-06T15:14:05+01:00 [pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6] time="2022-09-06T14:14:05Z" level=info msg="Opening volume /host/var/lib/longhorn/replicas/pvc-af69ea5b-9798-443a-8527-583c5fd35b70-a0da610f, size 21474836480/512"
2022-09-06T15:14:05+01:00 [pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6] time="2022-09-06T14:14:05Z" level=info msg="Lost connection from: 10.52.2.235:58878"
2022-09-06T15:14:07+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:07Z" level=debug msg="Process Manager: start getting logs for process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6"
2022-09-06T15:14:08+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:08Z" level=debug msg="Process Manager: got logs for process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6"
2022-09-06T15:14:10+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:10Z" level=debug msg="Process Manager: prepare to delete process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6"
2022-09-06T15:14:10+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:10Z" level=debug msg="Process Manager: deleted process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6"
2022-09-06T15:14:10+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:10Z" level=debug msg="Process Manager: wait for process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6 to shutdown before unregistering process"
2022-09-06T15:14:10+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:10Z" level=debug msg="Process Manager: trying to stop process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6"
2022-09-06T15:14:10+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:10Z" level=info msg="wait for process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6 to shutdown"
2022-09-06T15:14:10+01:00 [pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6] time="2022-09-06T14:14:10Z" level=warning msg="Received signal interrupt to shutdown"
2022-09-06T15:14:10+01:00 [pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6] time="2022-09-06T14:14:10Z" level=warning msg="Starting to execute registered shutdown func <http://github.com/longhorn/longhorn-engine/app/cmd.startReplica.func4|github.com/longhorn/longhorn-engine/app/cmd.startReplica.func4>"
2022-09-06T15:14:10+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:10Z" level=info msg="Process Manager: process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6 stopped"
2022-09-06T15:14:10+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:10Z" level=debug msg="Process Manager: prepare to delete process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6"
2022-09-06T15:14:10+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:10Z" level=debug msg="Process Manager: deleted process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6"
2022-09-06T15:14:10+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:10Z" level=info msg="Process Manager: successfully unregistered process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6"
2022-09-06T15:14:10+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:10Z" level=info msg="Process Manager: successfully unregistered process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6"
2022-09-06T15:14:11+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:11Z" level=info msg="Process Manager: prepare to create process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6"
2022-09-06T15:14:11+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:11Z" level=debug msg="Process Manager: validate process path: /host/var/lib/longhorn/engine-binaries/longhornio-longhorn-engine-v1.2.4/longhorn dir: /host/var/lib/longhorn/engine-binaries/ image: longhornio-longhorn-engine-v1.2.4 binary: longhorn"
2022-09-06T15:14:11+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:11Z" level=info msg="Process Manager: created process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6"
2022-09-06T15:14:11+01:00 [pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6] time="2022-09-06T14:14:11Z" level=info msg="Listening on gRPC Replica server 0.0.0.0:10090"
2022-09-06T15:14:11+01:00 [pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6] time="2022-09-06T14:14:11Z" level=info msg="Listening on data server 0.0.0.0:10091"
2022-09-06T15:14:11+01:00 time="2022-09-06T14:14:11Z" level=info msg="Listening on sync agent server 0.0.0.0:10092"
2022-09-06T15:14:11+01:00 time="2022-09-06T14:14:11Z" level=info msg="Listening on sync 0.0.0.0:10092"
2022-09-06T15:14:11+01:00 [longhorn-instance-manager] time="2022-09-06T14:14:11Z" level=info msg="Process pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6 has started at localhost:10090"
2022-09-06T15:14:12+01:00 [pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6] time="2022-09-06T14:14:12Z" level=info msg="New connection from: 10.52.2.235:47670"
2022-09-06T15:14:12+01:00 [pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6] time="2022-09-06T14:14:12Z" level=info msg="Opening volume /host/var/lib/longhorn/replicas/pvc-af69ea5b-9798-443a-8527-583c5fd35b70-a0da610f, size 21474836480/512"
2022-09-06T15:14:12+01:00 [pvc-af69ea5b-9798-443a-8527-583c5fd35b70-r-2879f2f6] time="2022-09-06T14:14:12Z" level=info msg="Lost connection from: 10.52.2.235:47670"
p
Hi Mark, is the volume healthy? could you check it from Longhorn GUI to see if there are any warning messages.
s
Sorry - I went on holiday. When I got back there were no unhealthy volumes.
But just today a similar problem is happening: https://rancher-users.slack.com/archives/C01GKHKAG0K/p1664278785862039
27 Views
Open in Slack
PreviousNext
Powered by