https://rancher.com/ logo
Powered by
Title
l

lively-zoo-40381

04/24/2023, 6:19 AM
Hello, I’ve installed Harvester 1.1.1 and configured a DHCP server which is used for the VM network. I have an issue in which VMs that are created on some Harvester nodes, are not able to access internet (for example ping to 8.8.8.8 timeout) but their internal IP which they received from the DHCP is working and I’m able to SSH the these Vms. • Other VMs (same template with same network and all other settings) on different Harvester nodes don’t have this issue • All Harvester nodes have the Same HW (DELL R640) and are installed with the same PXE and harvester auto-configuration Did someone came across this issue ?
b

bright-wall-24452

04/24/2023, 8:20 AM
It's UntaggedNetwork or L2VlanNetwork? Did you ping network gateway? Did you use telnet or traceroute？
l

lively-zoo-40381

04/24/2023, 10:32 AM
It’s L2VlanNetwork:
there is no ping to gateway (when the vm can’t ping 8.8.8.8)
b

bright-wall-24452

04/24/2023, 11:27 AM
ping network gateway?
Was your network configured with mac address in your VM? Were you checking host network configured?
You can use 
bridge -c vlan show
to check host bridge network.
l

lively-zoo-40381

04/24/2023, 11:47 AM
This is the DHCP network (no specific macs for Vms) 
subnet 192.168.226.0 netmask 255.255.254.0 {
range 192.168.226.16 192.168.226.99;
option domain-name-servers 8.8.8.8;
option routers 192.168.227.253;
option broadcast-address 192.168.227.255;
default-lease-time 7200;
max-lease-time 14400;
}
There is no reply when pinging network gateway. However with my SSL VPN I’m able to connect to the VM ip address (which it got from the DHCP)
VMs on this node works fine:
b

bright-wall-24452

04/24/2023, 11:52 AM
Can you check exchange mode?Is it access mode or trunk mode?
l

lively-zoo-40381

04/24/2023, 11:52 AM
Node that VMs not working correctly
How do I check exchange mode ? from switch side ?
basically there is no trunk. we only pass one Vlan
b

bright-wall-24452

04/24/2023, 11:57 AM
Did your vlanID configured on the switch?
l

lively-zoo-40381

04/24/2023, 11:59 AM
yes
b

bright-wall-24452

04/24/2023, 11:59 AM
Can you use untagged network?
l

lively-zoo-40381

04/24/2023, 11:59 AM
The Harvester nodes use it in the network config and it works
also as mentioned some VMs are working OK
we can’t use untagged network at this point
b

bright-wall-24452

04/24/2023, 12:02 PM
Do you fill them manual？
l

lively-zoo-40381

04/24/2023, 12:03 PM
no, it is set on Auto(DHCP)
image.png
image.png
b

bright-wall-24452

04/24/2023, 12:03 PM
Can you use mamual?
Have a try?
l

lively-zoo-40381

04/24/2023, 12:04 PM
I already have am RKE cluster running on the existing VMs…
b

bright-wall-24452

04/24/2023, 12:05 PM
Ohhhh
l

lively-zoo-40381

04/24/2023, 12:05 PM
They could lose connectivity no? and I think I started with the config you mentioned, and when it didn’t work I changed it to how it is now
Also, this does not explain why on some nodes there is no issue and on some there is, while they are identical HW and same Auto Config
I’m not sure what is going on here,,, if it’s Harvester network issue or plain network issue
b

bright-wall-24452

04/24/2023, 12:08 PM
Is the graph below normal on some nodes that have issues?
l

lively-zoo-40381

04/24/2023, 12:16 PM
I don’t see “Network” as you showed in any node (working Ok or not)
b

bright-wall-24452

04/24/2023, 12:19 PM
Oh！ I know it is my fault. My management network and data network is separated.
r

red-king-19196

04/28/2023, 3:33 AM
Hi @lively-zoo-40381, would you please check the config on the switch side? The settings on the Harvester side seem to be fine. As you mentioned, some of the nodes have no such problem, it might be worth taking a look at the config of corresponding ports on the switch. Thank you.
l

lively-zoo-40381

04/28/2023, 8:24 AM
Hello @red-king-19196 Yes, we have been checking that and comparing between switch ports and configuration for long time now… so far we could not find any difference between them. Thanks
This is what I found so far. If I save and then remove all iptables rules from a node which has vms not working, they then able to access the internet
This is iptables issue and not clear why this happens
Then I restored the iptables and the vm does not have network again. If I reboot the node, then the iptables rules are re-created again, and this time vms network works
I removed the Harvester Cluster, wiped all hosts, cleaned it up from rancher. Then I reinstalled the hosts with Harvester 1.1.2. Facing the exact same issue, iptables on the harvester hosts is causing vms not to have access to internet. Again, cleaning all iptables rules solves the issue, bringing them back blocks the vm
r

red-king-19196

05/04/2023, 2:51 AM
Does the same issue happen on the same set of nodes with Harvester 1.1.2?
It would be helpful if you could provide us a support bundle to look into. Thanks!
l

lively-zoo-40381

05/04/2023, 6:35 AM
This happens with same nodes + different nodes
@red-king-19196 https://github.com/harvester/harvester/issues/3852
🙌 1
r

red-king-19196

05/04/2023, 7:16 AM
But the VMs did get the IP address from the DHCP server running outside the Harvester cluster. Right after the VMs got the IP addresses, they lose network connectivity to the gateway (and beyond). Is that correct?
l

lively-zoo-40381

05/04/2023, 10:27 AM
Not exactly, here is what I saw so far. VM is created, it gets an IP from out dhcp service, which is configured in the VM network on Harvester. I can SSH to that internal address and access the vm, however the vm can’t go to the internet. it also can’t ping the gw. Then I remove all iptables rules from the Harvester host. and then the VM network starts to work, ping to gw, google etc… If I restore the iptables on the Harvester node, the issue shows up again. So there is no “Network/IP” issue, It’s just that the iptables somehow causes this. and this also causes the VM not to show it’s ip in the harvester UI, but infact as mentioned it has IP, and I can SSH to it. Hope that is more clear. Thank you for you help with this
r

red-king-19196

05/04/2023, 3:03 PM
I’ve checked the info you provided on the GitHub issue, and I’m curious about your network configuration for the management interface on the node. Though it might not directly relate to the issue, it seems you’re trying to accommodate VLAN 1 and 962 on the same switch port in access mode. May I ask what value was set to the “VLAN ID” field during the installation? If you installed the cluster via iPXE, the setting is under 
install.management_interface.vlan_id
.
Also, I assume 
harvester-4mhd6
is the problematic node, could you also provide us info about the normal one for references (
ip link
and 
ip addr
results)?
l

lively-zoo-40381

05/04/2023, 3:58 PM
@red-king-19196 Where did you see VLAN 1 ? I didn’t set that anywhere so I wonder how it was set ? The installation was done with PXE and Auto config files. and those do not hold any Vlan config: 
# cat /usr/share/nginx/html/harvester/config-create.yaml
scheme_version: 1
token: ourtoken # Replace with a desired token
os:
hostname: harvester-node1 # Set a hostname. This can be omitted if DHCP server offers hostnames
ssh_authorized_keys:
- ssh-rsa AAAA...
password: NtwBwv4oJNSN     # Replace with your password
ntp_servers:
- <http://0.suse.pool.ntp.org|0.suse.pool.ntp.org>
- <http://1.suse.pool.ntp.org|1.suse.pool.ntp.org>
install:
mode: create
#  force_mbr: true
management_interface: # available as of v1.1.0
interfaces:
- name: eno1
default_route: true
method: dhcp
bond_options:
mode: balance-tlb
miimon: 100
device: /dev/sda # The target disk to install
data_disk: /dev/sdb # It is recommended to use a separate disk to store VM data
iso_url: <http://192.168.226.129:8000/harvester/harvester-v1.1.2-amd64.iso>
#  tty: ttyS1,115200n8   # For machines without a VGA console
vip: 192.168.226.222        # The VIP to access the Harvester GUI. Make sure the IP is free to use
vip_mode: static        # Or dhcp, check configuration file for more information
#  vip_hw_addr: 52:54:00:ec:0e:0b   # Leave empty when vip_mode is static
Here are outputs from a node on which VMs are OK right now: 
rancher@harvester-bvr4m:~> ip link
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
link/ether e4:3d:1a:43:6d:42 brd ff:ff:ff:ff:ff:ff
altname enp1s0f0
3: eno1np0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master mgmt-bo state UP mode DEFAULT group default qlen 1000
link/ether e4:3d:1a:43:6d:44 brd ff:ff:ff:ff:ff:ff
altname enp25s0f0np0
4: eno2np1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether e4:3d:1a:43:6d:45 brd ff:ff:ff:ff:ff:ff
altname enp25s0f1np1
5: eno4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN mode DEFAULT group default qlen 1000
link/ether e4:3d:1a:43:6d:43 brd ff:ff:ff:ff:ff:ff
altname enp1s0f1
6: mgmt-br: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether e4:3d:1a:43:6d:44 brd ff:ff:ff:ff:ff:ff
7: mgmt-bo: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master mgmt-br state UP mode DEFAULT group default qlen 1000
link/ether e4:3d:1a:43:6d:44 brd ff:ff:ff:ff:ff:ff
10: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN mode DEFAULT group default
link/ether 8e:d2:5c:0e:98:dc brd ff:ff:ff:ff:ff:ff
14: cali5324dbda990@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-d69977c6-9fad-089d-9494-6d9ee988ec71
16: calief77afc0611@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-1a00b884-9db1-195c-32e0-9528dc035d4d
17: cali6c749091891@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-b7a8945a-57f5-ac17-3275-2656f2ab572d
18: calia1a4fa4fa43@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-21ca0f74-10cf-2240-91ba-ceefee22f1c2
19: cali228f289367c@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-88c19ae3-a0df-0737-ecd0-b2a00ec01eab
20: caliaacb25b0c43@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-5b1e66fe-ba1e-31f7-9042-cf21adf8b5e8
21: cali4722fb31728@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-b0e3baa4-715d-104d-e9c5-884e3334118c
22: calida1aac2c3a3@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-042fb428-7ff1-7c37-75c9-3cca2072cc02
23: calidd90ed9348b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-5b336815-9107-88df-203c-8914c43b4528
77: cali685f2a67d9b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-726a70c1-9ca1-8433-70ae-33d8038830c0
80: cali89b23d281ba@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-b55e3f51-f88c-02fa-8d0e-61411b9dd26b
81: cali2d923128859@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-c78db810-2a5d-0daf-53ed-d37899b36c74
86: calidb774742a75@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-505967be-d98c-aad9-7bad-c457439ed9c8
87: calic8d3cdba063@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-91c15436-8d7a-d830-ee04-4486c46d9bf8
88: calic55dc719ade@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-77496162-42d4-b673-68fb-b773f84418ad
89: cali64601ff138e@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-182c4d95-2a50-ae30-44fc-fb49a0c93c4f
90: cali68e89200b36@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-c410a049-d00f-2026-2cf8-a5516fb489b5
91: calic40b25563f5@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-fbe6c84b-351b-7c95-31e9-34e30c02534c
92: cali519725c218b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-35bbb1fe-835a-a638-922b-adaf0b9c546c
93: cali0c37d337f56@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-c7a0b592-dc3c-9713-cf77-8466ccc1e8bb
94: cali6e86d5d5395@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-eb33c411-909e-b03f-339d-0f9ddff19656
95: calia2b6edc3a06@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-4b63f626-04a3-790f-e8b8-93ddb7c968b9
96: cali1b548065acb@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-fedff3a9-5b01-d126-9d40-acafd87ac527
97: cali6ed7cfe8715@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-760b735a-ceef-8b58-0806-c8d084f9c06a
98: cali394b3d657dc@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-a1577e90-e4c2-315f-ee4a-b8d7107319d9
99: cali0d384ed8889@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-ec52c349-0d1a-2323-6264-6ef6789c3d19
100: veth8ed292fe@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master mgmt-br state UP mode DEFAULT group default
link/ether fa:5f:bb:70:f3:cf brd ff:ff:ff:ff:ff:ff link-netns cni-ec52c349-0d1a-2323-6264-6ef6789c3d19
101: calibf7a0cfd81f@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-a7b30c67-d796-8a9f-eb98-66e9a7699736
102: veth60a9edf7@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master mgmt-br state UP mode DEFAULT group default
link/ether 8a:2c:9f:80:ab:2f brd ff:ff:ff:ff:ff:ff link-netns cni-a7b30c67-d796-8a9f-eb98-66e9a7699736
103: cali58198ee4cf5@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-b4da8342-ce5e-5324-42dd-22395148312c
104: caliec923649e5e@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-b25fa076-deb2-f0f7-edab-2bec7cdf4aab
105: cali0c7da1f03ad@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-eb6802a2-6d66-fa27-090a-9ddb13a08e78
106: cali4371ed33ace@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-7ec59d37-ef37-dbb2-babf-0686b2348170
107: cali94233d61402@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-298ebe16-1f43-17af-775f-11731355243f
108: califb984cf6b44@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-212b8933-d77b-9693-a0c2-091070ba4af9
109: calif838a0a056b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-8465168f-eaf3-d491-7a23-31116e82c5db
111: cali61aa616e9ba@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-6b11a42e-5472-09ae-18a7-96af0655ff73
113: cali17948ce1d0b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-83d33408-6107-7152-c675-a9eb88c6403a
114: calib7cd418feb7@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP mode DEFAULT group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-b5eef315-d025-e096-ef7a-a0c336771f74
rancher@harvester-bvr4m:~> ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
2: eno3: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether e4:3d:1a:43:6d:42 brd ff:ff:ff:ff:ff:ff
altname enp1s0f0
3: eno1np0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master mgmt-bo state UP group default qlen 1000
link/ether e4:3d:1a:43:6d:44 brd ff:ff:ff:ff:ff:ff
altname enp25s0f0np0
4: eno2np1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
link/ether e4:3d:1a:43:6d:45 brd ff:ff:ff:ff:ff:ff
altname enp25s0f1np1
5: eno4: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether e4:3d:1a:43:6d:43 brd ff:ff:ff:ff:ff:ff
altname enp1s0f1
6: mgmt-br: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether e4:3d:1a:43:6d:44 brd ff:ff:ff:ff:ff:ff
inet 192.168.226.36/23 brd 192.168.227.255 scope global mgmt-br
valid_lft forever preferred_lft forever
7: mgmt-bo: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue master mgmt-br state UP group default qlen 1000
link/ether e4:3d:1a:43:6d:44 brd ff:ff:ff:ff:ff:ff
10: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default
link/ether 8e:d2:5c:0e:98:dc brd ff:ff:ff:ff:ff:ff
inet 10.52.1.0/32 scope global flannel.1
valid_lft forever preferred_lft forever
14: cali5324dbda990@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-d69977c6-9fad-089d-9494-6d9ee988ec71
16: calief77afc0611@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-1a00b884-9db1-195c-32e0-9528dc035d4d
17: cali6c749091891@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-b7a8945a-57f5-ac17-3275-2656f2ab572d
18: calia1a4fa4fa43@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-21ca0f74-10cf-2240-91ba-ceefee22f1c2
19: cali228f289367c@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-88c19ae3-a0df-0737-ecd0-b2a00ec01eab
20: caliaacb25b0c43@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-5b1e66fe-ba1e-31f7-9042-cf21adf8b5e8
21: cali4722fb31728@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-b0e3baa4-715d-104d-e9c5-884e3334118c
22: calida1aac2c3a3@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-042fb428-7ff1-7c37-75c9-3cca2072cc02
23: calidd90ed9348b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-5b336815-9107-88df-203c-8914c43b4528
77: cali685f2a67d9b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-726a70c1-9ca1-8433-70ae-33d8038830c0
80: cali89b23d281ba@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-b55e3f51-f88c-02fa-8d0e-61411b9dd26b
81: cali2d923128859@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-c78db810-2a5d-0daf-53ed-d37899b36c74
86: calidb774742a75@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-505967be-d98c-aad9-7bad-c457439ed9c8
87: calic8d3cdba063@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-91c15436-8d7a-d830-ee04-4486c46d9bf8
88: calic55dc719ade@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-77496162-42d4-b673-68fb-b773f84418ad
89: cali64601ff138e@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-182c4d95-2a50-ae30-44fc-fb49a0c93c4f
90: cali68e89200b36@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-c410a049-d00f-2026-2cf8-a5516fb489b5
91: calic40b25563f5@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-fbe6c84b-351b-7c95-31e9-34e30c02534c
92: cali519725c218b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-35bbb1fe-835a-a638-922b-adaf0b9c546c
93: cali0c37d337f56@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-c7a0b592-dc3c-9713-cf77-8466ccc1e8bb
94: cali6e86d5d5395@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-eb33c411-909e-b03f-339d-0f9ddff19656
95: calia2b6edc3a06@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-4b63f626-04a3-790f-e8b8-93ddb7c968b9
96: cali1b548065acb@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-fedff3a9-5b01-d126-9d40-acafd87ac527
97: cali6ed7cfe8715@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-760b735a-ceef-8b58-0806-c8d084f9c06a
98: cali394b3d657dc@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-a1577e90-e4c2-315f-ee4a-b8d7107319d9
99: cali0d384ed8889@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-ec52c349-0d1a-2323-6264-6ef6789c3d19
100: veth8ed292fe@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master mgmt-br state UP group default
link/ether fa:5f:bb:70:f3:cf brd ff:ff:ff:ff:ff:ff link-netns cni-ec52c349-0d1a-2323-6264-6ef6789c3d19
101: calibf7a0cfd81f@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-a7b30c67-d796-8a9f-eb98-66e9a7699736
102: veth60a9edf7@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master mgmt-br state UP group default
link/ether 8a:2c:9f:80:ab:2f brd ff:ff:ff:ff:ff:ff link-netns cni-a7b30c67-d796-8a9f-eb98-66e9a7699736
103: cali58198ee4cf5@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-b4da8342-ce5e-5324-42dd-22395148312c
104: caliec923649e5e@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-b25fa076-deb2-f0f7-edab-2bec7cdf4aab
105: cali0c7da1f03ad@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-eb6802a2-6d66-fa27-090a-9ddb13a08e78
106: cali4371ed33ace@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-7ec59d37-ef37-dbb2-babf-0686b2348170
107: cali94233d61402@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-298ebe16-1f43-17af-775f-11731355243f
108: califb984cf6b44@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-212b8933-d77b-9693-a0c2-091070ba4af9
109: calif838a0a056b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-8465168f-eaf3-d491-7a23-31116e82c5db
111: cali61aa616e9ba@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-6b11a42e-5472-09ae-18a7-96af0655ff73
113: cali17948ce1d0b@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-83d33408-6107-7152-c675-a9eb88c6403a
114: calib7cd418feb7@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default
link/ether ee:ee:ee:ee:ee:ee brd ff:ff:ff:ff:ff:ff link-netns cni-b5eef315-d025-e096-ef7a-a0c336771f74
r

red-king-19196

05/05/2023, 1:10 AM
Thanks for providing this information. I’d like to know the switch port config, too. Is it configured to something like “access/untagged vlan 962”? About VLAN 1, if 
install.management_interface.vlan_id
is unset, it actually associates with VLAN 1 (the default VLAN ID).
And if you want to make the VM network segment the same as the host was in (192.168.226.0/23), the VLAN ID of the VM network should be set to 1, not 962.
l

lively-zoo-40381

05/05/2023, 5:08 AM
Hey @red-king-19196 Yes, the switch ports config is without trunk, it’s access mode in Vlan 962 only. So you mean I should change the VLAN ID in the VM network to 1 ? Thanks again for your help
@red-king-19196 I changed the VM network to VlanID 1 and i think it indeed solved the issue ! Created new cluster and all Vms on all nodes seem to work correctly now
🙌 1
r

red-king-19196

05/05/2023, 5:34 AM
Glad that helped! Your switch port is set to “access VLAN 962”, so it accepts packets without tags and associates them internally with VLAN 962. On the Harvester side, the management and VM networks should use VLAN 1 (meaning “untagged”). But I’m still curious why some of the VMs can have Internet connectivity previously 🤔
l

lively-zoo-40381

05/05/2023, 5:36 AM
Yea, it was not clear to me as well how some VMs worked before, but at least now I understand the logic which makes sense and it works correctly. Thanks again !
👍 1
🎉 1
#harvesterJoin Slack
Powered by