https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# rke2
a
This message was deleted.
c
Don't use the ULA range like that. Iirc the kernel literally won't let you add routes to ula space because it's not valid to use it like that.
s
Can you help me understand how I’m using it incorrectly?
What should I be doing in stead, if I don’t have a real/public IPv6 allocation to test with?
c
You're using it at all
s
Since ULA is meant to be use in some circumstances, I presume you mean the issue is that I’m using it at all in this circumstance. I do not understand why it is incorrect, since the documentation that I have read compares ULA to RFC 1918 private addresses, which are used in this circumstance for IPv4. So if 10.0.0.0/8 addresses are appropriate, but ULA addresses are not, I’d like to understand why. But, regardless, I don’t know what I should be doing in stead, if I don’t have a real IPv6 allocation to test with.
h
if you do not have real ipv6 addresses, you could use the documentation prefix 3fff::/20 or 2001db8:/32 and pick some prefixes from that. It is made for internal labs and tests. but are not routed on the internet.
a
That's incorrect, actually.
Those are documentation prefixes. For documentation.
ULA is the proper thing to use here.
h
well the reason 3fff::/20 is made, is that 2001db8:/32 was too small for large labs
a
I am aware. I was part of the WG for that update.
there's nothing inherently wrong with using ULA addresses for such internal services or testing scenarios.
h
ULA have a different characteristic and is almost never the right thing to use. eg it is not used if there are ipv4 addresses in play
are there public GUA addresses for lab's tho ? i often see Documentation prefix in used on labs. while probably not strictly correct
a
I mean 
2001:cafe
is 100% wrong, so.
Fixing that would be a good start.
h
and if rke2 does not allow the ULA prefixes to be used that way as @creamy-pencil-82913 said. it is not really possible to use it either.
a
Then RKE2 should be fixed to not break standards.
h
having proper RKE2 ipv6 documentation would be a huge boon
a
Agreed
h
agree compleatly
i am as @stocky-kilobyte-40414 just trying to get it to work. but that is an uphill battle
was there not a 0200::/7 lab prefix rfx ? or did it expire ?
a
I understand, and appreciate your input too. Mostly, I wanted to join here and correct misinformation and not-very-helpful statements like, "you're using it at all" which perpetuate that ULA is unusable and has no real world purpose. ULA is not perfect, correct, but there is ongoing work (e.g.) to address the operational concerns of ULA.
that RFC was never formally adopted afaik
(wrong number, bad memory)
c
Sorry, just took a second look at this. It’s failing because your service CIDR mask is too large. There is a max size on that, set by Kubernetes, to prevent the bitmap of consumed IPv6 service addresses from growing too large.
Use the masks suggested by the docs and it should work.
s
Very good to understand. Thank you for the second look!
a
What is the concern over the bitmap of consumed addresses? like, the actual trie ?
c
yes
a
hm. Is that due to the k8s implementation of v6, or something upstream? a /64 is the smallest subnet usually used for ipv6, so I want to understand the reason for a smaller slice to see if there's a way it can be solved. smaller than /64 is usually not preferred due to potential incompatibility with devices which expect the last 64 bits to be host-only (slaac, e.g.). though, i guess that might not be a concern in this specific use case. even point to point links now are recommended to just use a /64 even though only two addresses are ever used
c
this is a kubernetes thing
a
interesting... thanks for the link there! I'll send it to my nerd friends 😉
Very glad there is a simple fix! Hopefully Jonathon and I (and others) can help document RKE v6 and improve the experience for future users.
c
feel free to suggest a change to the docs to use different subnets. It does work as documented, though.
s
imo, the documentation probably should use an appropriately-sized subnet from the documentation range; and then document the constraints on the subnet sizes.
Now that I know what to look for, this is how Azure documents it, fwiw:
Copy code
--pod-cidrs: Takes a comma-separated list of CIDR notation IP ranges to assign pod IPs from.
The count and order of ranges in this list must match the value provided to --ip-families.
If no values are supplied, the default value 10.244.0.0/16,fd12:3456:789a::/64 is used.
--service-cidrs: Takes a comma-separated list of CIDR notation IP ranges to assign service IPs from.
The count and order of ranges in this list must match the value provided to --ip-families.
If no values are supplied, the default value 10.0.0.0/16,fd12:3456:789a:1::/108 is used.
The IPv6 subnet assigned to --service-cidrs can be no larger than a /108.
(But, once I’ve confirmed I can get it working on my side with this new knowledge, I’d be happy to figure out how to suggest documentation edits!)
c
the mask constraints are enforced by Kubernetes. Unfortunately their docs don’t do a good job of calling them out.
👍 2
s
Understood.
h
@stocky-kilobyte-40414 do you deploy rke2 from the rancher ui. or manually/cli ? i have a different issue. where the cluster using ula space internally (the right sizes) works with kubectl locally. but does not register properly on the rancher interface.
s
Just manually (with https://get.rke2.io)
One layer of complexity at a time! 😅
Thanks again, @creamy-pencil-82913, for helping me to understand what’s going on.
(Now I’m also curious to find the log entry that would’ve told me this if I’d known where to look, too.)
c
it’d be in the controller-manager log I believe
possibly apiserver too, I can’t remember off the top of my head
Itll say something like 
Controller: Invalid --cluster-cidr, mask size of cluster CIDR must be less than or equal to --node-cidr-mask-size configured for CIDR family
I can’t remember what the message is for the too large service cidr
s
Thanks!
I confirm that using
Copy code
cluster-cidr: 10.42.0.0/16,fd58:faa1:bd97:42::/56
service-cidr: 10.43.0.0/16,fd58:faa1:bd97:43::/112
works. Thanks, again!
5 Views
Open in Slack
PreviousNext
Powered by