Channels
academy
amazon
arm
azure
cabpr
chinese
ci-cd
danish
deutsch
developer
elemental
epinio
espanol
events
extensions
fleet
français
gcp
general
harvester
harvester-dev
hobbyfarm
hypper
japanese
k3d
k3os
k3s
k3s-contributor
kim
kubernetes
kubewarden
lima
logging
longhorn-dev
longhorn-storage
masterclass
mesos
mexico
nederlands
neuvector-security
office-hours
one-point-x
onlinemeetup
onlinetraining
opni
os
ozt
phillydotnet
portugues
rancher-desktop
rancher-extensions
rancher-setup
rancher-wrangler
random
rfed_ara
rio
rke
rke2
russian
s3gw
service-mesh
storage
submariner
supermicro-sixsq
swarm
terraform-controller
terraform-provider-rancher2
terraform-provider-rke
theranchcast
training-0110
training-0124
training-0131
training-0207
training-0214
training-1220
ukranian
v16-v21-migration
vsphere
windows
Title
b
broad-farmer-70498
03/13/2023, 6:55 PM@creamy-pencil-82913 does this mean the image binary installs the crd now when launched? https://github.com/k3s-io/helm-controller/commit/09dedbf504bdaa722b99d9e7ea8bd67fba787bd2
c
creamy-pencil-82913
03/13/2023, 6:58 PMThe controller always created the CRDs itself when started
That just added an auto-generated CRD to the build artifacts, as opposed to having a stale one that was checked into git
b
broad-farmer-70498
03/13/2023, 6:59 PMam I missing it in the build artifacts?
c
creamy-pencil-82913
03/13/2023, 7:00 PMits included in the top of both of the yaml files in the release assets
b
broad-farmer-70498
03/13/2023, 7:00 PMaaaaahh
sorry I missed that...lastly, is there a comprehensive list of rbac needed in case I don't run it in kube-system?
I'm after cluster-wide but installing in a non kube-system ns
c
creamy-pencil-82913
03/13/2023, 7:07 PMall the RBAC should be included in the manifest, I think you’d just need to update the namespaces to wherever you’re deploying it?
b
broad-farmer-70498
03/13/2023, 7:08 PMI don't see any rbac in the manifest (this time I swear I've looked at it 🙂 )
the cluster wide has 2 crds and the deployment
c
creamy-pencil-82913
03/13/2023, 7:37 PMAhh right. It’s been a while since I looked at this and I’m not the primary maintainer of that project, but iirc it just uses the default serviceaccount which has permission to create Jobs. The Jobs in turn have full cluster access so that they can create whatever resources are in the Helm chart.
I don’t think we have an example of using custom RBAC and a separate SA for the controller.
because in either case it essentially needs full cluster admin for Helm to be able to do what it needs to do. The only difference between the namespaced and cluster-scoped installations is where the controller watches for HelmChart resources.
b
broad-farmer-70498
03/13/2023, 7:54 PMright, but implicitly running in kube-system means full access no?
c
creamy-pencil-82913
03/13/2023, 8:35 PMno, there’s nothing special about that namespace privilege-wise
the Helm job pod itself runs with a ServiceAccount that is bound to the cluster-admin built-in role
https://github.com/k3s-io/helm-controller/blob/master/pkg/controllers/chart/chart.go#L517
thats where the full access comes from. not from the controller itself
b
broad-farmer-70498
03/13/2023, 8:38 PMok, but how would the controller be able to watch the crds cluster-wide without rbac if not running in kube-system
c
creamy-pencil-82913
03/13/2023, 8:44 PMI’d have to go poke through the cluster RBAC to refresh my memory
b
broad-farmer-70498
03/13/2023, 8:45 PMno worries, it may be easier to deploy to kube-system, but I'll dump it in my preferred ns and see how bad it blows up 😄
see if I can't get the proper rbac squared away
I0313 20:48:00.594811 1 leaderelection.go:248] attempting to acquire leader lease kube-system/helm-controller-lock...
E0313 20:48:00.596531 1 leaderelection.go:330] error retrieving resource lock kube-system/helm-controller-lock: configmaps "helm-controller-lock" is forbidden: User "system:serviceaccount:operators:default" cannot get resource "configmaps" in API group "" in the namespace "kube-system"
E0313 20:48:04.062603 1 leaderelection.go:330] error retrieving resource lock kube-system/helm-controller-lock: configmaps "helm-controller-lock" is forbidden: User "system:serviceaccount:operators:default" cannot get resource "configmaps" in API group "" in the namespace "kube-system"seems to be pretty hard-coded to some things
I'll just move it I suppose
even in kube-system..
I0313 21:07:18.118032 1 controllers.go:94] Starting helm controller with no namespace
I0313 21:07:18.118051 1 leaderelection.go:248] attempting to acquire leader lease kube-system/helm-controller-lock...
E0313 21:07:18.121038 1 leaderelection.go:330] error retrieving resource lock kube-system/helm-controller-lock: configmaps "helm-controller-lock" is forbidden: User "system:serviceaccount:kube-system:default" cannot get resource "configmaps" in API group "" in the namespace "kube-system"c
creamy-pencil-82913
03/13/2023, 9:43 PMhonestly I’m not sure how much this gets used as a standalone controller. It was built for use in K3s/RKE2 where it is embedded into the supervisor as a basic way to get the packaged components deployed into the cluster at startup or during upgrade. If you’re trying to deploy a standalone Helm controller there are better more feature-complete ones out there.
b
broad-farmer-70498
03/13/2023, 9:44 PMI predominantly run rke/rke2 and don't see it in any of the clusters
c
creamy-pencil-82913
03/13/2023, 9:44 PMyes its built in and runs in the supervisor
b
broad-farmer-70498
03/13/2023, 9:44 PMI don't deploy with rancher but they are joined
c
creamy-pencil-82913
03/13/2023, 9:45 PMIf you’re running K3s or RKE2 you don’t need to install anything, its already there
just deploy a HelmChart manifest and it’ll do its thing
b
broad-farmer-70498
03/13/2023, 9:45 PMthere is no such crd in my clusters
c
creamy-pencil-82913
03/13/2023, 9:46 PMdid you start them with --disable-helm-controller or something?
b
broad-farmer-70498
03/13/2023, 9:46 PMno
c
creamy-pencil-82913
03/13/2023, 9:46 PMwhat version of RKE2 are you running?
It’s not in RKE, only K3s/RKE2
If you look at a K3s or RKE2 cluster, it’ll be there
b
broad-farmer-70498
03/13/2023, 9:48 PMok, most of them are rke but indeed I do see it in any rke2 cluster
I do not however see a controller pod running (it may be handled differently)
c
creamy-pencil-82913
03/13/2023, 9:48 PMthats what I mean by built in to the supervisor
👍 1
b
broad-farmer-70498
03/13/2023, 9:50 PMI'll just mess around with it and get it running..I'll have clusters not managed by rke2 that will probably need it at some point as well
c
creamy-pencil-82913
03/13/2023, 9:53 PMit looks like the tests at least do set up a one-off SA and ClusterRoleBinding for it. When it’s deployed as part of RKE2/K3s it uses the supervisor’s admin RBAC. If you want to open an issue to include RBAC in the example manifests we can try to do that at some point.
b
broad-farmer-70498
03/13/2023, 9:55 PMI'll just do some trial/error and get it nailed down