https://rancher.com/ logo
Title
f

flaky-coat-75909

08/19/2022, 11:22 AM
Hi, I'm receinvg a events with message
Message: EXT4-fs error (device sda): ext4_find_entry:1446: inode #12: comm grafana-server: reading directory lblock 0
And my Node Problem detector is showing
{*reason*="Ext4Error"}
more than 1 thousand appears on one node but in UI everything works fine (all is on green) Grafana also working as well I have 5 nodes let's say node1, node2, node3, node4, node5 grafana is working on node3 and the data replicas is on node4 and node5 Mayby something is missing?
I have found https://longhorn.io/kb/troubleshooting-volume-readonly-or-io-error/
Root causes
2. The network bandwidth is not sufficient. Normally 1Gbps network will only able to serve 3 volumes if all of those volumes are running a high intensive workload
My Bandwith is
930 Mbits/sec
between nodes And node1 have 0 replicas (master) node2 have 5 replicas of diffrent pvc node3 have 6 replicas node4 have 3 replicas node5 have 3 replicas and number of volume is 6 (one of the biggest pvc volume is 15Gi) It can make a issue in my case?
i

icy-agency-38675

08/22/2022, 2:59 AM
The io error might be probably caused by the insufficient network bandwidth or short network outage.
f

flaky-coat-75909

08/22/2022, 11:05 AM
@icy-agency-38675 thanks for response hmm I suspect dns resoultion in my cluster (short network outage, or time resolution is sometimes 3sec instead of instant ) how longhorn-managers and other components will communicate to each other? they are using service name or something? (for example
instance-manager.svc...
) if not the second is short network outage in random periods
i

icy-agency-38675

08/23/2022, 12:47 AM
Use the IP address mostly.