Hello I am currently running rke2 1 24 4 and most recently r rancher-users #rke2

Title

numerous-country-20400

10/18/2022, 8:58 PM

Hello. I'am currently running rke2 1.24.4 and most recently run into several helm chart failing installing due to seccomp issues. Basically i get told that the contain seccomp settings cannot be applied

forbidden seccomp may not be set

- for example the new cert-manager 1.10 introduces https://artifacthub.io/packages/helm/cert-manager/cert-manager/1.10.0#default-security-contexts a new default security context - thus i cannot install it on my rke2 cluster. Same goes with bitnami-wordpress start 15.2.0 which also introduce RuntimeDefault as their default runtime. Is there anything missing in my rke2 configuration or do i miss the point entirely?

does rke2 ship with a restricted security profile ? https://kubernetes.io/docs/concepts/security/pod-security-standards/ by default? But if yes, RuntimeDefault seems rather to be what should work in the first place

This change on the cert-manager chart is now leading to those kind of issues https://github.com/cert-manager/cert-manager/commit/cbb476929eb2fc336a76e703217f41b7133b4a18#diff-719152aa560e986139083[…]938a36e334d066bf084daf1716 .. but it all got stricted, not less strict. So i wonder why i would get into trouble with those. I would rather have expected to run into those when running "less restricted"

creamy-pencil-82913

10/18/2022, 9:21 PM

It depends on whether or not you started your rke2 servers with a --profile set…

https://docs.rke2.io/security/policies/

If you’re on 1.24 you’re not using PSS yet, just PSP

numerous-country-20400

10/18/2022, 9:23 PM

iam looking at https://docs.rke2.io/security/policies/ already, but kot able to make a clue what i have and what is blocking

what do you mean by PSS?

ah default pod security standards

fairly sure i did not use any --profile when boostrapping rke. i used plain

curl -sfL <https://get.rke2.io> | sh -

with a fairly simple default config https://gist.github.com/EugenMayer/cdf7ac12b02280fcd1fa885018994568

so i assume i used whatever default was there

I assume when running

kubectl get psp -A

i see

global-restricted-psp

and

global-unrestricted-psp

- i guess when cert-manager 1.10 is installed and using

RuntimeDefault

as the default, the the PSP is switched to

global-restricted-psp

right. And something in there is not allowed

These are my PSP right now https://gist.github.com/EugenMayer/e2e6e16d5957857fde69315fc74065f6

i could also image that applying the new PSP to the deployment, the old pods are not really conform with that and thaus violate it. But i'am really swimming here 🙂

creamy-pencil-82913

10/18/2022, 9:38 PM

If you’re planning on using Rancher with RKE2, I don’t think it supports 1.10 yet. https://docs.ranchermanager.rancher.io/getting-started/installation-and-upgrade/resources/upgrade-cert-manager#cert-manager-api-[…]nd-data-migration

This sounds a lot like https://github.com/rancher/rke2/issues/2156

numerous-country-20400

10/18/2022, 9:46 PM

i'am not using rkes 2 cert-manager helm chart, using the public one. Also not using rancher (yet, stopped with rancher1, we kind of do not need rancher2 somehow right now, terraform kind of replaced most of the use cases for it for now. We'll see)

i have see 2156 but could not identify how app armor was related to the changes of cert-manager or bitname wordpress - that is the entire reason i'am researching right now

i guess

<http://seccomp.security.alpha.kubernetes.io/allowedProfileNames|seccomp.security.alpha.kubernetes.io/allowedProfileNames>: '*'

somewhat means, what profiles can be used (like "RuntimeDefault") ?

creamy-pencil-82913

10/18/2022, 9:49 PM

yeah it means anything

Our PSP does have that attribute on it now, it should be fine on all current releases

https://github.com/rancher/rke2/blob/release-1.24/pkg/rke2/psp_templates.go#L94

are you installing the charts yourself, or are you using the rke2 embedded helm controller?

numerous-country-20400

10/18/2022, 9:52 PM

i'am installing basically all charts myself. From nginx to CSI to cert-manager

So longhorn, localpath, calico are all installed by me

creamy-pencil-82913

10/18/2022, 9:53 PM

hmm.

numerous-country-20400

10/18/2022, 9:54 PM

"PSP has the attribute now" - are you refering to 1.25 only?

creamy-pencil-82913

10/18/2022, 9:54 PM

no, in 1.25 there are no more PSPs at all. I linked you to 1.24.

The release notes for cert-manager 1.10 aren’t even there yet… https://cert-manager.io/docs/installation/supported-releases/

Can you stick with 1.9.1 for now?

numerous-country-20400

10/18/2022, 9:55 PM

interesting. so you would expect it to be fixed with 1.24.4? or is is 12.4.6 required (AFAICS .7 is rc right now)

creamy-pencil-82913

10/18/2022, 9:56 PM

yeah we added that annotation several releases ago

numerous-country-20400

10/18/2022, 9:56 PM

i can for sure stick with 1.9.1 for now. But e.g. the bitnami chart (Wordpres) is already broken for about 3 weeks now. I assume those things will now fly in very soon broadly. Esp in bitnami charts, which we use often

creamy-pencil-82913

10/18/2022, 9:57 PM

so you’re just running

helm install

and it comes back with an error where?

numerous-country-20400

10/18/2022, 9:57 PM

in the deployment

creamy-pencil-82913

10/18/2022, 9:57 PM

Can you open an issue on GH and specifically document what you’re installing and how?

numerous-country-20400

10/18/2022, 9:57 PM

creamy-pencil-82913

10/18/2022, 9:58 PM

What UI is that?

numerous-country-20400

10/18/2022, 9:58 PM

sure, of course. I assume https://github.com/rancher/rke2/issues ?

that is lens

creamy-pencil-82913

10/18/2022, 9:58 PM

yes

ah ok

numerous-country-20400

10/18/2022, 10:00 PM

after adding

<http://seccomp.security.alpha.kubernetes.io/allowedProfileNames|seccomp.security.alpha.kubernetes.io/allowedProfileNames>: "*"

to the global unrestricted PSP, it does deploy just fine

with that in, you still want me to open that issue? Seems like it might would be either "outdated" or a duplicate

the question is, is it expected that the annotation would have been landed as an upgrade on my 1.24.4 cluster or not - that might be a potential source of an unexpected bug

creamy-pencil-82913

10/18/2022, 10:01 PM

yes, issue please

numerous-country-20400

10/18/2022, 10:01 PM

Should i go after the "shouldnt it be on 1.24.4 already" part? Or the supprise that this annotation is needed in the first place?

creamy-pencil-82913

10/18/2022, 10:03 PM

yeah I think the issue is that it’s not fixed on upgrades.

It was fixed in https://github.com/rancher/rke2/releases/tag/v1.23.4+rke2r1 but if you upgraded your cluster from a version before that, it isn’t added.

numerous-country-20400

10/18/2022, 10:04 PM

yeah i was pre 1.23.4 for sure, most probably 1.22 or earlier even

so this is actually expected then

i guess it's this then https://github.com/rancher/rke2/pull/2449

Then actually, with all that being said - i really do not get what my issue should be on about. It is expected that the annotation is missing for me. It is known the annotation causes it and is required to be present and why. It has been fixed, even though it is not auto-upgraded (by design)

(beside all that, thank you a LOT for helping!)

creamy-pencil-82913

10/18/2022, 10:40 PM

yeah I don’t think we intended to not have it fixed on upgrade. If you want to open an issue about that, it would probably get that fixed.

numerous-country-20400

10/18/2022, 10:42 PM

I open an issue for that discussion, what ever comes of it, is fine. Thanks

opened https://github.com/rancher/rke2/issues/3475 - good night, it's bed time here :P)

8 Views

#rke2 Join Slack