Channels
extensions
rancher-extensions
supermicro-sixsq
ozt
os
one-point-x
rio
onlinemeetup
v16-v21-migration
events
onlinetraining
training-1220
training-0110
training-0124
k3s-contributor
submariner
storage
k3os
windows
ci-cd
theranchcast
rfed_ara
nederlands
ukranian
danish
training-0131
masterclass
training-0207
training-0214
terraform-controller
epinio
phillydotnet
swarm
rancher-wrangler
deutsch
russian
chinese
mesos
français
japanese
arm
longhorn-dev
terraform-provider-rke
service-mesh
azure
gcp
academy
random
elemental
espanol
lima
harvester-dev
portugues
kubernetes
kim
hypper
kubewarden
opni
logging
neuvector-security
rancher-setup
developer
office-hours
mexico
cabpr
rke
vsphere
longhorn-storage
k3s
k3d
terraform-provider-rancher2
fleet
amazon
harvester
rke2
s3gw
hobbyfarm
rancher-desktop
general
Title
s
strong-tomato-67726
07/25/2022, 5:13 PM--kube-apiserver-arg="enable-admission-plugins=NodeRestriction,PodSecurityPolicy,ServiceAccount"
how to install with following options
h
high-waitress-66594
07/25/2022, 5:44 PMThis is not working as expected? Is there an error you can share? Otherwise, please describe how what you've shared is not working.
I've messed with kubelet args before but never admission plugins
My hunch is that it might be only picking up the first plugin for enablement? @strong-tomato-67726
s
strong-tomato-67726
07/25/2022, 5:51 PMexport INSTALL_K3S_EXEC="server --token test --cluster-init --write-kubeconfig-mode 644 --secrets-encryption --kube-apiserver-arg='enable-admission-plugins=NodeRestriction,PodSecurityPolicy,NamespaceLifecycle,ServiceAccount' "sudo curl -sfL <https://get.k3s.io> | INSTALL_K3S_EXEC="$INSTALL_K3S_EXEC" sh -❯ sudo curl -sfL <https://get.k3s.io> | INSTALL_K3S_EXEC="$INSTALL_K3S_EXEC" sh -
[INFO] Finding release for channel stable
[INFO] Using v1.24.3+k3s1 as release
[INFO] Downloading hash <https://github.com/k3s-io/k3s/releases/download/v1.24.3+k3s1/sha256sum-amd64.txt>
[INFO] Skipping binary downloaded, installed k3s matches hash
[INFO] Skipping installation of SELinux RPM
[INFO] Skipping /usr/local/bin/kubectl symlink to k3s, already exists
[INFO] Skipping /usr/local/bin/crictl symlink to k3s, already exists
[INFO] Skipping /usr/local/bin/ctr symlink to k3s, already exists
[INFO] Creating killall script /usr/local/bin/k3s-killall.sh
[INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh
[INFO] env: Creating environment file /etc/systemd/system/k3s.service.env
[INFO] systemd: Creating service file /etc/systemd/system/k3s.service
[INFO] systemd: Enabling k3s unit
Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.
[INFO] systemd: Starting k3s
Job for k3s.service failed because the control process exited with error code.
See "systemctl status k3s.service" and "journalctl -xeu k3s.service" for details.Jul 25 17:52:27 ip-172-31-14-144 k3s[3286187]: time="2022-07-25T17:52:27Z" level=info msg="Connecting to proxy" url="<wss://127.0.0.1:6443/v1-k3s/connect>"
Jul 25 17:52:27 ip-172-31-14-144 k3s[3286187]: time="2022-07-25T17:52:27Z" level=info msg="Running kubelet --address=0.0.0.0 --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=cgroupfs --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43>
Jul 25 17:52:27 ip-172-31-14-144 k3s[3286187]: time="2022-07-25T17:52:27Z" level=info msg="Handling backend connection request [ip-172-31-14-144]"
Jul 25 17:52:27 ip-172-31-14-144 k3s[3286187]: time="2022-07-25T17:52:27Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: <https://127.0.0.1:6443/v1-k3s/readyz>: 500 Internal Server Error"
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.344Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"acf9a069ddce4765 is starting a new election at term 3"}
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.344Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"acf9a069ddce4765 became pre-candidate at term 3"}
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.344Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"acf9a069ddce4765 received MsgPreVoteResp from acf9a069ddce4765 at term 3"}
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.344Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"acf9a069ddce4765 became candidate at term 4"}
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.344Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"acf9a069ddce4765 received MsgVoteResp from acf9a069ddce4765 at term 4"}
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.344Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"acf9a069ddce4765 became leader at term 4"}
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.344Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"raft.node: acf9a069ddce4765 elected leader acf9a069ddce4765 at term 4"}
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.346Z","caller":"etcdserver/server.go:2044","msg":"published local member to cluster through raft","local-member-id":"acf9a069ddce4765","local-member-attributes":"{Name:ip-172-31-14-144-67582285 ClientURLs:[<https://172.31.14.144:2379>]}","request-path":">
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.346Z","caller":"embed/serve.go:98","msg":"ready to serve client requests"}
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.346Z","caller":"embed/serve.go:98","msg":"ready to serve client requests"}
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.348Z","caller":"embed/serve.go:188","msg":"serving client traffic securely","address":"172.31.14.144:2379"}
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.349Z","caller":"embed/serve.go:188","msg":"serving client traffic securely","address":"127.0.0.1:2379"}
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: time="2022-07-25T17:52:31Z" level=info msg="Tunnel server egress proxy waiting for runtime core to become available"
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: time="2022-07-25T17:52:31Z" level=info msg="Defragmenting etcd database"
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.365Z","caller":"v3rpc/maintenance.go:89","msg":"starting defragment"}
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.368Z","caller":"backend/backend.go:497","msg":"defragmenting","path":"/var/lib/rancher/k3s/server/db/etcd/member/snap/db","current-db-size-bytes":1613824,"current-db-size":"1.6 MB","current-db-size-in-use-bytes":1609728,"current-db-size-in-use":"1.6 MB>
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.393Z","caller":"backend/backend.go:549","msg":"finished defragmenting directory","path":"/var/lib/rancher/k3s/server/db/etcd/member/snap/db","current-db-size-bytes-diff":0,"current-db-size-bytes":1613824,"current-db-size":"1.6 MB","current-db-size-in-u>
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: {"level":"info","ts":"2022-07-25T17:52:31.393Z","caller":"v3rpc/maintenance.go:95","msg":"finished defragment"}
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: time="2022-07-25T17:52:31Z" level=info msg="etcd data store connection OK"
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: time="2022-07-25T17:52:31Z" level=info msg="Waiting for API server to become available"
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: Error: unknown flag: --enable-admission-plugins\
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: time="2022-07-25T17:52:31Z" level=fatal msg="apiserver exited: unknown flag: --enable-admission-plugins\\"
Jul 25 17:52:31 ip-172-31-14-144 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: <http://www.ubuntu.com/support>
░░
░░ An ExecStart= process belonging to unit k3s.service has exited.
░░
░░ The process' exit code is 'exited' and its exit status is 1.
Jul 25 17:52:31 ip-172-31-14-144 systemd[1]: k3s.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: <http://www.ubuntu.com/support>
░░
░░ The unit k3s.service has entered the 'failed' state with result 'exit-code'.
Jul 25 17:52:31 ip-172-31-14-144 systemd[1]: k3s.service: Unit process 3284059 (containerd-shim) remains running after unit stopped.
Jul 25 17:52:31 ip-172-31-14-144 systemd[1]: k3s.service: Unit process 3284140 (containerd-shim) remains running after unit stopped.
Jul 25 17:52:31 ip-172-31-14-144 systemd[1]: k3s.service: Unit process 3284173 (containerd-shim) remains running after unit stopped.
Jul 25 17:52:31 ip-172-31-14-144 systemd[1]: k3s.service: Unit process 3285150 (containerd-shim) remains running after unit stopped.
Jul 25 17:52:31 ip-172-31-14-144 systemd[1]: k3s.service: Unit process 3285229 (containerd-shim) remains running after unit stopped.
Jul 25 17:52:31 ip-172-31-14-144 systemd[1]: Failed to start Lightweight Kubernetes.
░░ Subject: A start job for unit k3s.service has failed
░░ Defined-By: systemd
░░ Support: <http://www.ubuntu.com/support>
░░
░░ A start job for unit k3s.service has finished with a failure.
░░
░░ The job identifier is 24748 and the job result is failed.
Jul 25 17:52:31 ip-172-31-14-144 systemd[1]: k3s.service: Consumed 6.549s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: <http://www.ubuntu.com/support>
░░
░░ The unit k3s.service completed and consumed the indicated resources.Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: Error: unknown flag: --enable-admission-plugins\
Jul 25 17:52:31 ip-172-31-14-144 k3s[3286187]: time="2022-07-25T17:52:31Z" level=fatal msg="apiserver exited: unknown flag: --enable-admission-plugins\\"
Jul 25 17:52:31 ip-172-31-14-144 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE❯ cat k3s.service
[Unit]
Description=Lightweight Kubernetes
Documentation=<https://k3s.io>
Wants=network-online.target
After=network-online.target
[Install]
WantedBy=multi-user.target
[Service]
Type=notify
EnvironmentFile=-/etc/default/%N
EnvironmentFile=-/etc/sysconfig/%N
EnvironmentFile=-/etc/systemd/system/k3s.service.env
KillMode=process
Delegate=yes
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
TimeoutStartSec=0
Restart=always
RestartSec=5s
ExecStartPre=/bin/sh -xc '! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service'
ExecStartPre=-/sbin/modprobe br_netfilter
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/k3s \
server \
'--token' \
'test' \
'--cluster-init' \
'--write-kubeconfig-mode' \
'644' \
'--secrets-encryption' \
'--kube-apiserver-arg=enable-admission-plugins\=NodeRestriction,PodSecurityPolicy,NamespaceLifecycle,ServiceAccount' \h
high-waitress-66594
07/25/2022, 5:55 PMShould be no need to shell-escape the
=The hint is here:
Jul 25 17:52:31 ip-172-31-14-144 k3s[<tel:3286187|3286187>]: Error: unknown flag: --enable-admission-plugins\
Jul 25 17:52:31 ip-172-31-14-144 k3s[<tel:3286187|3286187>]: time="2022-07-25T17:52:31Z" level=fatal msg="apiserver exited: unknown flag: --enable-admission-plugins\\"
Jul 25 17:52:31 ip-172-31-14-144 systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE✅ 1