https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# k3s
a
This message was deleted.
h
check this link: https://docs.k3s.io/release-notes/v1.24.X I do not run k3s anymore so I do not have anything to add as far as experience. but I do not recommend to jump from 1.24 to 1.26. anyways check the upgrade nodes
🙌 1
b
watch out for the PSP removal in 1.25
l
thanks
b
also, only upgrade 1 minor at a time 1.24 -> 1.25 -> 1.26. Kubernetes upgrades are designed to be sequential. skipping releases can break stuff.
💯 1
l
good idea, thanks @bulky-sunset-52084 did not think about it at all,
b
@bulky-sunset-52084 when you say watch out for it; if a cluster is not running any end-user created psps, can we expect that all managed charts will be upgraded by now and thus have no issues?
@late-eve-82257 I second only one minor version at a time. You might want to review the skew policy as well
b
@brave-afternoon-4801 yes vanilla k3s charts will upgrade without any problem - if it doesn't please file a bug.
but rancher as an example needs a flag added to the install to NOT install the PSPs https://github.com/rancher/rancher/blob/release/v2.8/chart/values.yaml#L178 Rancher is not the only application where this consideration is made
b
@bulky-sunset-52084 so if rancher is being installed as a HelmChart resource, is there a specific order that should happen for the 1.25 upgrade?
b
https://www.suse.com/support/kb/doc/?id=000021053 It's going to be the same steps downstream and upstream
start with downstream - then do upstream last
b
@bulky-sunset-52084 small follow up. there is a mention in this doc:
For a smooth upgrade to Kubernetes v1.25, you must remove PodSecurityPolicy resources that were deployed with Helm. To do so, upgrade each of your charts’ installations to the newest version v102.0.0, making sure to set the PodSecurityPolicy switch global.cattle.psp.enabled value to false.
I just tested with the rancher-monitoring chart and it does not seem to delete any existing PSP, but does seem to not create them
h
if I understand you correctly, before upgrading cluster running Rancher to 1.25, we should upgrade downstream clusters to 1.25?
b
@brave-afternoon-4801 helm should be tracking that if the resources were created as part of a helm chart. Can you check the annotations on the psps to validate if they are being tracked by helm? If they are not helm resources then you may have to delete them manually. @hundreds-evening-84071 that is the method I've had the most luck with personally. It really shouldn't matter the order so long as everything stays with the support matrix
b
I can see that indeed they are helm tracked but not removed. further, i can look at the BundleDeployments and verify that 
global.cattle.psp.enabled: false
, yet the PSP resource is still there.
b
Well 
rancher-monitoring
is not 
rancher
so that explains why updating the rancher chart wouldn't effect the monitoring resources.
b
maybe wires got crossed - this follow up is only about 
rancher-monitoring
my rancher install either never installed any psps or did clean them up already
b
Oh sorry -thought you were talking about rancher
Lol alright let me dig through the chart really quick what version are you on?
Oh duh says in the picture
b
rancher-monitoring added the same toggle as promised by that linked doc.. for a second i thought the resources were orphaned but the resources are tracked
I'm on the newest 
102.0.2+up40.1.2
b
Hmm interesting the helm controller should clean up unreferenced resources when they are no longer referenced by the chart 🤔 this is k3s correct?
b
yeah, v1.24.17+k3s1
it's not unreferenced though. look at the screenshot annotations. it's been updated from the 100.x.x chart it was using previously
b
They are dereferenced (not created) when you set global.cattle.psp.enabled=false. Helm should clean this up when you install a new version of the chart that no longer referenced those psps.
b
that is what you would think should happen, yes, but it would seem like they were indeed rendered
thats what i mean - the version in the annotation would still be the old version if helm didn't think it should still render this resource and manage it
wouldn't all users of this chart be missing these by default if this toggle was working as expected?
um, is it possible that this value is getting screwed with by the subchart.. like, should the lookup by absolute and not relative? (
$.Values
)
I take that back - it's not a subchart, just a template in a subdir
Yeah this is seeming really fishy. If it is set to 
false
by default, I shouldn't have had to do anything to make these disappear. simply upgrading chart versions should have done it
I can only surmise that this isn't working as expected, or fleet pulled the wrong chart, but i don't think it's the latter
@bulky-sunset-52084 I just fully deleted the monitoring chart and installed it fresh. I'm pretty sure this is a bug at this point.
I think it's because the chart is relying on truthiness instead of 
true
even setting it to 
null
is useless
worse yet, I cant reproduce this locally, which makes me think the error is between fleet and helm
okay, i've more or less narrowed this down to a fleet bug
i take that back. it was transient. still lost
2 Views
Open in Slack
PreviousNext
Powered by