< creamy pencil 82913> does this mean the ~image~ binary ins rancher-users #k3s

Title

broad-farmer-70498

03/13/2023, 6:55 PM

@creamy-pencil-82913 does this mean the ~~image~~ binary installs the crd now when launched? https://github.com/k3s-io/helm-controller/commit/09dedbf504bdaa722b99d9e7ea8bd67fba787bd2

creamy-pencil-82913

03/13/2023, 6:58 PM

The controller always created the CRDs itself when started

That just added an auto-generated CRD to the build artifacts, as opposed to having a stale one that was checked into git

broad-farmer-70498

03/13/2023, 6:59 PM

am I missing it in the build artifacts?

creamy-pencil-82913

03/13/2023, 7:00 PM

its included in the top of both of the yaml files in the release assets

broad-farmer-70498

03/13/2023, 7:00 PM

aaaaahh

sorry I missed that...lastly, is there a comprehensive list of rbac needed in case I don't run it in kube-system?

I'm after cluster-wide but installing in a non kube-system ns

creamy-pencil-82913

03/13/2023, 7:07 PM

all the RBAC should be included in the manifest, I think you’d just need to update the namespaces to wherever you’re deploying it?

broad-farmer-70498

03/13/2023, 7:08 PM

I don't see any rbac in the manifest (this time I swear I've looked at it 🙂 )

the cluster wide has 2 crds and the deployment

creamy-pencil-82913

03/13/2023, 7:37 PM

Ahh right. It’s been a while since I looked at this and I’m not the primary maintainer of that project, but iirc it just uses the default serviceaccount which has permission to create Jobs. The Jobs in turn have full cluster access so that they can create whatever resources are in the Helm chart.

I don’t think we have an example of using custom RBAC and a separate SA for the controller.

because in either case it essentially needs full cluster admin for Helm to be able to do what it needs to do. The only difference between the namespaced and cluster-scoped installations is where the controller watches for HelmChart resources.

broad-farmer-70498

03/13/2023, 7:54 PM

right, but implicitly running in kube-system means full access no?

creamy-pencil-82913

03/13/2023, 8:35 PM

no, there’s nothing special about that namespace privilege-wise

the Helm job pod itself runs with a ServiceAccount that is bound to the cluster-admin built-in role https://github.com/k3s-io/helm-controller/blob/master/pkg/controllers/chart/chart.go#L517

thats where the full access comes from. not from the controller itself

broad-farmer-70498

03/13/2023, 8:38 PM

ok, but how would the controller be able to watch the crds cluster-wide without rbac if not running in kube-system

creamy-pencil-82913

03/13/2023, 8:44 PM

I’d have to go poke through the cluster RBAC to refresh my memory

broad-farmer-70498

03/13/2023, 8:45 PM

no worries, it may be easier to deploy to kube-system, but I'll dump it in my preferred ns and see how bad it blows up 😄

see if I can't get the proper rbac squared away

Copy code

I0313 20:48:00.594811       1 leaderelection.go:248] attempting to acquire leader lease kube-system/helm-controller-lock...
E0313 20:48:00.596531       1 leaderelection.go:330] error retrieving resource lock kube-system/helm-controller-lock: configmaps "helm-controller-lock" is forbidden: User "system:serviceaccount:operators:default" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
E0313 20:48:04.062603       1 leaderelection.go:330] error retrieving resource lock kube-system/helm-controller-lock: configmaps "helm-controller-lock" is forbidden: User "system:serviceaccount:operators:default" cannot get resource "configmaps" in API group "" in the namespace "kube-system"

seems to be pretty hard-coded to some things

I'll just move it I suppose

even in kube-system..

Copy code

I0313 21:07:18.118032       1 controllers.go:94] Starting helm controller with no namespace
I0313 21:07:18.118051       1 leaderelection.go:248] attempting to acquire leader lease kube-system/helm-controller-lock...
E0313 21:07:18.121038       1 leaderelection.go:330] error retrieving resource lock kube-system/helm-controller-lock: configmaps "helm-controller-lock" is forbidden: User "system:serviceaccount:kube-system:default" cannot get resource "configmaps" in API group "" in the namespace "kube-system"

creamy-pencil-82913

03/13/2023, 9:43 PM

honestly I’m not sure how much this gets used as a standalone controller. It was built for use in K3s/RKE2 where it is embedded into the supervisor as a basic way to get the packaged components deployed into the cluster at startup or during upgrade. If you’re trying to deploy a standalone Helm controller there are better more feature-complete ones out there.

broad-farmer-70498

03/13/2023, 9:44 PM

I predominantly run rke/rke2 and don't see it in any of the clusters

creamy-pencil-82913

03/13/2023, 9:44 PM

yes its built in and runs in the supervisor

broad-farmer-70498

03/13/2023, 9:44 PM

I don't deploy with rancher but they are joined

creamy-pencil-82913

03/13/2023, 9:45 PM

If you’re running K3s or RKE2 you don’t need to install anything, its already there

just deploy a HelmChart manifest and it’ll do its thing

broad-farmer-70498

03/13/2023, 9:45 PM

there is no such crd in my clusters

creamy-pencil-82913

03/13/2023, 9:46 PM

did you start them with --disable-helm-controller or something?

broad-farmer-70498

03/13/2023, 9:46 PM

creamy-pencil-82913

03/13/2023, 9:46 PM

what version of RKE2 are you running?

It’s not in RKE, only K3s/RKE2

If you look at a K3s or RKE2 cluster, it’ll be there

broad-farmer-70498

03/13/2023, 9:48 PM

ok, most of them are rke but indeed I do see it in any rke2 cluster

I do not however see a controller pod running (it may be handled differently)

creamy-pencil-82913

03/13/2023, 9:48 PM

thats what I mean by built in to the supervisor

👍 1

broad-farmer-70498

03/13/2023, 9:50 PM

I'll just mess around with it and get it running..I'll have clusters not managed by rke2 that will probably need it at some point as well

creamy-pencil-82913

03/13/2023, 9:53 PM

it looks like the tests at least do set up a one-off SA and ClusterRoleBinding for it. When it’s deployed as part of RKE2/K3s it uses the supervisor’s admin RBAC. If you want to open an issue to include RBAC in the example manifests we can try to do that at some point.

broad-farmer-70498

03/13/2023, 9:55 PM

I'll just do some trial/error and get it nailed down

3 Views