https://rancher.com/ logo
Title
a

alert-policeman-61846

01/25/2023, 6:22 PM
Hello friends. I recently added longhorn 1.4 on a multi-az and multi-region k3s cluster. Works like a charm. I tested the "best-effort" data locality. Im wondering if best effort will take care of regions (from the doc, seems node colocation and az are taken care off, not sure about regions https://kubernetes.io/docs/reference/labels-annotations-taints/#topologykubernetesioregion). https://longhorn.io/docs/1.4.0/high-availability/data-locality/ My ideal setup would place active replica on same node as pod, and replicas in other nodes in same az. To avoid inter-region replication traffic. Is this possible ?
f

famous-journalist-11332

01/26/2023, 6:23 AM
My ideal setup would place active replica on same node as pod,
The Longhorn data locality feature can achieve this
and replicas in other nodes in same az. To avoid inter-region replication traffic. Is this possible ?
Unfortunately, no Longhorn actually try to separate replicas onto different AZ to increase the availability cc @famous-shampoo-18483 @billowy-painting-56466
a

alert-policeman-61846

01/26/2023, 1:15 PM
Thks @famous-journalist-11332 I saw a reference to region https://longhorn.io/docs/1.4.0/volumes-and-nodes/scheduling/#notice, and
<http://topology.kubernetes.io/region=|topology.kubernetes.io/region=><Region name
Id like to have some clarification. How does the region play with this availability placement ?
f

famous-journalist-11332

01/26/2023, 10:47 PM
Region a bigger object than AZ. Region contains AZ. AZ contains nodes. Longhorn attempts to spread the replicas of volume between different regions, then different AZs, then different nodes
f

famous-shampoo-18483

01/28/2023, 2:28 AM
IIUC, cross-AZ replication should be fine. But no one would distribute replicas in different regions, since the network latency among different region (> 1ms) would greatly hurt the volume performance and make it impractical.
👍 1
a

alert-policeman-61846

01/30/2023, 3:16 PM
Agreed cross region replicas should be in control of end user, if needed. I understand longhorn 1.4.0 will try to schedule on different regions by default. Is there a way to avoid this ? disk-level tags maybe ?
f

famous-shampoo-18483

03/09/2023, 4:28 PM
YES. You group nodes by tags then use the corresponding selector for replica scheduling. This is an pretty old but powerful feature. IIRC there is no special feature for the region-level scheduling in v1.4.0...