https://rancher.com/ logo
Powered by
#rke2
Title
n

numerous-country-20400

10/18/2022, 8:58 PM
Hello. I'am currently running rke2 1.24.4 and most recently run into several helm chart failing installing due to seccomp issues. Basically i get told that the contain seccomp settings cannot be applied 
forbidden seccomp may not be set
- for example the new cert-manager 1.10 introduces https://artifacthub.io/packages/helm/cert-manager/cert-manager/1.10.0#default-security-contexts a new default security context - thus i cannot install it on my rke2 cluster. Same goes with bitnami-wordpress start 15.2.0 which also introduce RuntimeDefault as their default runtime. Is there anything missing in my rke2 configuration or do i miss the point entirely?
does rke2 ship with a restricted security profile ? https://kubernetes.io/docs/concepts/security/pod-security-standards/ by default? But if yes, RuntimeDefault seems rather to be what should work in the first place
This change on the cert-manager chart is now leading to those kind of issues https://github.com/cert-manager/cert-manager/commit/cbb476929eb2fc336a76e703217f41b7133b4a18#diff-719152aa560e986139083[…]938a36e334d066bf084daf1716 .. but it all got stricted, not less strict. So i wonder why i would get into trouble with those. I would rather have expected to run into those when running "less restricted"
c

creamy-pencil-82913

10/18/2022, 9:21 PM
It depends on whether or not you started your rke2 servers with a --profile set…
If you’re on 1.24 you’re not using PSS yet, just PSP
n

numerous-country-20400

10/18/2022, 9:23 PM
iam looking at https://docs.rke2.io/security/policies/ already, but kot able to make a clue what i have and what is blocking
what do you mean by PSS?
ah default pod security standards
fairly sure i did not use any --profile when boostrapping rke. i used plain 
curl -sfL <https://get.rke2.io> | sh -
with a fairly simple default config https://gist.github.com/EugenMayer/cdf7ac12b02280fcd1fa885018994568
so i assume i used whatever default was there
I assume when running 
kubectl get psp -A
i see 
global-restricted-psp
and 
global-unrestricted-psp
- i guess when cert-manager 1.10 is installed and using 
RuntimeDefault
as the default, the the PSP is switched to 
global-restricted-psp
right. And something in there is not allowed
i could also image that applying the new PSP to the deployment, the old pods are not really conform with that and thaus violate it. But i'am really swimming here 🙂
c

creamy-pencil-82913

10/18/2022, 9:38 PM
n

numerous-country-20400

10/18/2022, 9:46 PM
i'am not using rkes 2 cert-manager helm chart, using the public one. Also not using rancher (yet, stopped with rancher1, we kind of do not need rancher2 somehow right now, terraform kind of replaced most of the use cases for it for now. We'll see)
i have see 2156 but could not identify how app armor was related to the changes of cert-manager or bitname wordpress - that is the entire reason i'am researching right now
i guess 
<http://seccomp.security.alpha.kubernetes.io/allowedProfileNames|seccomp.security.alpha.kubernetes.io/allowedProfileNames>: '*'
somewhat means, what profiles can be used (like "RuntimeDefault") ?
c

creamy-pencil-82913

10/18/2022, 9:49 PM
yeah it means anything
Our PSP does have that attribute on it now, it should be fine on all current releases
are you installing the charts yourself, or are you using the rke2 embedded helm controller?
n

numerous-country-20400

10/18/2022, 9:52 PM
i'am installing basically all charts myself. From nginx to CSI to cert-manager
So longhorn, localpath, calico are all installed by me
c

creamy-pencil-82913

10/18/2022, 9:53 PM
hmm.
n

numerous-country-20400

10/18/2022, 9:54 PM
"PSP has the attribute now" - are you refering to 1.25 only?
c

creamy-pencil-82913

10/18/2022, 9:54 PM
no, in 1.25 there are no more PSPs at all. I linked you to 1.24.
The release notes for cert-manager 1.10 aren’t even there yet… https://cert-manager.io/docs/installation/supported-releases/
Can you stick with 1.9.1 for now?
n

numerous-country-20400

10/18/2022, 9:55 PM
interesting. so you would expect it to be fixed with 1.24.4? or is is 12.4.6 required (AFAICS .7 is rc right now)
c

creamy-pencil-82913

10/18/2022, 9:56 PM
yeah we added that annotation several releases ago
n

numerous-country-20400

10/18/2022, 9:56 PM
i can for sure stick with 1.9.1 for now. But e.g. the bitnami chart (Wordpres) is already broken for about 3 weeks now. I assume those things will now fly in very soon broadly. Esp in bitnami charts, which we use often
c

creamy-pencil-82913

10/18/2022, 9:57 PM
so you’re just running 
helm install
and it comes back with an error where?
n

numerous-country-20400

10/18/2022, 9:57 PM
in the deployment
c

creamy-pencil-82913

10/18/2022, 9:57 PM
Can you open an issue on GH and specifically document what you’re installing and how?
n

numerous-country-20400

10/18/2022, 9:57 PM
c

creamy-pencil-82913

10/18/2022, 9:58 PM
What UI is that?
n

numerous-country-20400

10/18/2022, 9:58 PM
sure, of course. I assume https://github.com/rancher/rke2/issues ?
that is lens
c

creamy-pencil-82913

10/18/2022, 9:58 PM
yes
ah ok
n

numerous-country-20400

10/18/2022, 10:00 PM
after adding 
<http://seccomp.security.alpha.kubernetes.io/allowedProfileNames|seccomp.security.alpha.kubernetes.io/allowedProfileNames>: "*"
to the global unrestricted PSP, it does deploy just fine
with that in, you still want me to open that issue? Seems like it might would be either "outdated" or a duplicate
the question is, is it expected that the annotation would have been landed as an upgrade on my 1.24.4 cluster or not - that might be a potential source of an unexpected bug
c

creamy-pencil-82913

10/18/2022, 10:01 PM
yes, issue please
n

numerous-country-20400

10/18/2022, 10:01 PM
Should i go after the "shouldnt it be on 1.24.4 already" part? Or the supprise that this annotation is needed in the first place?
c

creamy-pencil-82913

10/18/2022, 10:03 PM
yeah I think the issue is that it’s not fixed on upgrades.
It was fixed in https://github.com/rancher/rke2/releases/tag/v1.23.4+rke2r1 but if you upgraded your cluster from a version before that, it isn’t added.
n

numerous-country-20400

10/18/2022, 10:04 PM
yeah i was pre 1.23.4 for sure, most probably 1.22 or earlier even
so this is actually expected then
Then actually, with all that being said - i really do not get what my issue should be on about. It is expected that the annotation is missing for me. It is known the annotation causes it and is required to be present and why. It has been fixed, even though it is not auto-upgraded (by design)
(beside all that, thank you a LOT for helping!)
c

creamy-pencil-82913

10/18/2022, 10:40 PM
yeah I don’t think we intended to not have it fixed on upgrade. If you want to open an issue about that, it would probably get that fixed.
n

numerous-country-20400

10/18/2022, 10:42 PM
I open an issue for that discussion, what ever comes of it, is fine. Thanks
opened https://github.com/rancher/rke2/issues/3475 - good night, it's bed time here :P)
12 Views
Powered by