quaint-alarm-7893
12/05/2022, 6:16 AMsalmon-city-57654
12/05/2022, 4:23 PMquaint-alarm-7893
12/05/2022, 5:06 PMsalmon-city-57654
12/06/2022, 5:50 PMharvester-03
and harvester-02
down?
for harvester-01
, this extra-disk is already full? or that could not schedule due to the overcommit setup?quaint-alarm-7893
12/06/2022, 6:13 PMsalmon-city-57654
12/07/2022, 2:47 PMthat disk, does show as not schedulable. shoudl i just delete the replica and let l.h. move it to another disk?If the related volume is healthy, you could manually delete replica to trigger reschedule
also what’s odd, HCI shows the disk has 184gib avail, 850 scheduled, and has a max of 916gb, but it it’s not schedulable... odd.what is your overcommit setting? Or could you check the more information from longhorn UI? (Maybe there are some errors on it)
quaint-alarm-7893
12/07/2022, 9:27 PMWarning FailedAttachVolume (3429) 3.3 mins ago AttachVolume.Attach failed for volume "pvc-7616922b-a530-4bcc-b281-0a0438955d4d" : rpc error: code = DeadlineExceeded desc = volume pvc-7616922b-a530-4bcc-b281-0a0438955d4d failed to attach to node harvester-04
Warning FailedMount (3075) 2.1 mins ago (combined from similar events): Unable to attach or mount volumes: unmounted volumes=[prometheus-rancher-monitoring-prometheus-db], unattached volumes=[web-config prometheus-nginx nginx-home config config-out prometheus-rancher-monitoring-prometheus-rulefiles-0 kube-api-access-g4vsb tls-assets prometheus-rancher-monitoring-prometheus-db]: timed out waiting for the condition
State: Detached
Health:
Unknown
Ready for workload:Ready
Conditions:
restore
scheduled
Frontend:Block Device
Attached Node & Endpoint:
Size:
50 Gi
Actual Size:64.7 Gi
Data Locality:disabled
Access Mode:ReadWriteOnce
Engine Image:longhornio/longhorn-engine:v1.2.4
Created:2 months ago
Encrypted:False
Node Tags:
Disk Tags:
Last Backup:
Last Backup At:
Replicas Auto Balance:ignored
Instance Manager:
Namespace:cattle-monitoring-system
PVC Name:prometheus-rancher-monitoring-prometheus-db-prometheus-rancher-monitoring-prometheus-0
PV Name:pvc-7616922b-a530-4bcc-b281-0a0438955d4d
PV Status:Bound
Revision Counter Disabled:False
Pod Name:prometheus-rancher-monitoring-prometheus-0
Pod Status:Pending
Workload Name:prometheus-rancher-monitoring-prometheus
Workload Type:StatefulSet
salmon-city-57654
12/08/2022, 3:17 PMquaint-alarm-7893
12/08/2022, 4:27 PMsalmon-city-57654
12/12/2022, 4:57 PMquaint-alarm-7893
12/12/2022, 4:58 PMsalmon-city-57654
12/12/2022, 5:01 PMquaint-alarm-7893
12/24/2022, 5:29 AM