#k3s how do I create the k3s cluster with 3 master and 3 worker nodes with a single command line? do we have any configuration file (having all the node details) and pass it as argument to k3s .. please let me know !!

hello. I'm trying to link k3s running rootless together with nerdctl. I can get k3s rootless going pretty easily and I have nerdctl working rootless using their containerd-rootless-setuptool.sh. That's where the success ends.

If I don't do the nerctl rootless stuff and use nsenter to run nerdctl I can connect to the k3s containerd and use nerdctl ps and images. I can get buildkit started with nsenter buildkitd and other settings but when it starts building I get errors when the Dockerfile does a COPY

anyone have any experience with this or can give me some pointers? end goal is to be able to use nerdctl to start and build containers while I transition my teams development environment from docker to k8s but also be able to use k3s.

Is it possible to use the upgrade image for upgrading single-node cluster? Tried with the approach described here and here, but it doesn’t seem to work as expected. Upgrade image is executed, but

kubectl version

still shows old server version.

(message removed)

Hi people! I need a hand to understand why calico installation job fails when I install it via helm-controller. I'm trying to deploy this helmchart resource:

apiVersion: <http://helm.cattle.io/v1|helm.cattle.io/v1>
kind: HelmChart
metadata:
  name: cilium
  namespace: kube-system
spec:
  bootstrap: True
  chart: cilium/cilium
  version: 1.12.1
  targetNamespace: kube-system
  valuesContent: |-
    operator:
      replicas: 2
      image:
        useDigest: false
    tunnel: disabled
    autoDirectNodeRoutes: true
    kubeProxyReplacement: strict
    loadBalancer:
      standalone: false
      mode: dsr
    k8sServiceHost: 10.130.42.39
    k8sServicePort: 6443
    nativeRoutingCIDR: 10.0.0.0/16
    image: 
      useDigest: false
      pullPolicy: IfNotPresent

but the job pod fails with this error:

+ echo 'Installing helm_v3 chart'
+ helm_v3 install --namespace kube-system --version 1.12.1 cilium cilium/cilium --values /config/values-01_HelmChart.yaml
Error: INSTALLATION FAILED: failed to download "cilium/cilium" at version "1.12.1"

if I try do install the chart manually (

helm install cilium cilium/cilium ...

), it works

Hey friends, here's what the k3s team worked on this week: https://github.com/k3s-io/k3s/discussions/6149

What would the process look like if I had 1 master+worker, 3 worker nodes and I wanted to swap to 3 master+worker, 1 worker? Would there be anything to it more than just draining the two existing nodes, deleting them with

k delete node exampleNode

, uninstalling k3s on them, then following https://rancher.com/docs/k3s/latest/en/installation/ha/#2-launch-server-nodes

Hey there folks! I'd like to ask a question about creating service accounts in k3s and their related access tokens - I'm using Argo Workflows locally (for development purposes) and somehow I can't seem to get the token I need for accessing my local Argo programatically. More on this thread.

-> #rke2 The containerd binary, and others, are downloaded and extracted from the rke2-runtime docker image when rke2 starts. They are not included in the rpm. If the extract fails it should retry, but it's possible some error caused things to be partially created and that is causing problems? What do the full logs say? Have you tried running rke2-uninstall.sh and trying again from scratch?

k3s doesn’t have masters/workers, instead has servers/agents

#k3s have anybody installed elastic stack on K3s in airgap environment ?

Hi All Pen test reported that weak & invalid ssl communication is in use by one k3s server. Literally, i am a very new to this k3's. Could some one point me on how to fix the below issue

openssl s_client -connect kube1001:6443 2>&1 | grep issuer
verify error:num=20:unable to get local issuer certificate
issuer=/CN=k3s-server-ca@1649270763

openssl s_client -connect kube1001:443 2>&1 | grep issuer
verify error:num=20:unable to get local issuer certificate
issuer=/CN=k3s-server-ca@1649270763

openssl s_client -connect kube1001:80 2>&1 | grep issuer
verify error:num=20:unable to get local issuer certificate
issuer=/CN=k3s-server-ca@1649270763

Hey! We would like to understand what k3s users are choosing as flannel backend. We created a one-question survey and we would be very thankful if you could answer it: https://github.com/k3s-io/k3s/discussions/6170

Hello <!channel> we rarely use the

channel

but this survey helps us drive direction and is critically important to the k3s team, sorry and thank you for your support.

@bland-account-99790 and @clever-analyst-23771 we dont use flannel but are working on a much faster fabric for gpus from cloud to iot. ML use cases are very diff. we are working towards something like the AWS elastic fabric adapter.

I’ve never had to stray from the default vxlan

been using Wireguard as I have servers in the cloud and agents in the edge

IIUC K3s uses traefik by default. I knew that there is an annotation to redirect to SSL ports on the ingress. Did this change recently? The annoation used to be

'<http://traefik.ingress.kubernetes.io/redirect-entry-point|traefik.ingress.kubernetes.io/redirect-entry-point>': 'https',

Howdy, I am working on getting my first k3s cluster up and running and was wanted to expose some services externally with metallb. Has anyone been able to get the latest version of metallb running with a rancher deployed k3s cluster? I found a few references to metallb and k3s in the search, but nothing yet that seemed to be hitting my particular issue. Thanks for any help,

Hi team, all my dev environments are k3s based. I use vcluster to spin up k3s clusters. Now I am testing istio-cni and I can see the following error with Calico and Cilium

Error: failed to create ambient informer service: error getting host IP: <nil>

I do not expect anybody to understand that, but at least to point me what could be te reason the the process cannot get the host IP. Thanks!

I'm experimenting with an idea of hosting the control plane of my K3s based cluster away from home - in another Kubernetes cluster hosted in the cloud. I've created the needed manifests and got it all working using the

--disable-agent

flag so that the control plane pod is not registering itself as a node. The control plane environment is accessed by a public IP which works well, I can get pod logs and shell from my test workstation and from a side container on the control plane node I can curl to services inside the cluster which confirms the built in proxying is working - however, when I deploy gatekeeper, the control plane is not able to execute the webhook (which is strange because I can curl that webhook from the control plane pod sidecar just fine).

Error from server (InternalError): error when creating "<https://raw.githubusercontent.com/metallb/metallb/v0.12.1/manifests/namespace.yaml>": Internal error occurred: failed calling webhook "check-ignore-label.gatekeeper.sh": failed to call webhook: Post "<https://gatekeeper-webhook-service.gatekeeper-system.svc:443/v1/admitlabel?timeout=3s>": context deadline exceeded

The above is the error and below is the curl from the CP side car:

/ # curl "<https://gatekeeper-webhook-service.gatekeeper-system.svc:443/v1/admit?timeout=3s>" -Ivk
*   Trying 10.0.17.2:443...
* Connected to gatekeeper-webhook-service.gatekeeper-system.svc (10.0.17.2) port 443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN: server accepted h2
* Server certificate:
*  subject: CN=gatekeeper-webhook-service.gatekeeper-system.svc
*  start date: Aug 23 11:45:37 2022 GMT
*  expire date: Aug 23 11:55:37 2024 GMT
*  issuer: CN=gatekeeper-ca
*  SSL certificate verify result: unable to get local issuer certificate (20), continuing anyway.

The connectivity is definitely there, but somehow something isn't connecting. The k3s cp process logs:

k3s I0924 11:53:04.418026      46 trace.go:205] Trace[729582201]: "Proxy via http_connect protocol over tcp" address:10.42.0.8:8443 (24-Sep-2022 11:50:54.455) (total time: 129962ms):
k3s Trace[729582201]: [2m9.962952731s] [2m9.962952731s] END
k3s I0924 11:53:04.418106      46 trace.go:205] Trace[975991258]: "Proxy via http_connect protocol over tcp" address:10.42.0.7:8443 (24-Sep-2022 11:50:53.454) (total time: 130963ms):
k3s Trace[975991258]: [2m10.963777266s] [2m10.963777266s] END
k3s I0924 11:53:04.418387      46 trace.go:205] Trace[1687549248]: "Proxy via http_connect protocol over tcp" address:10.42.0.7:8443 (24-Sep-2022 11:50:53.454) (total time: 130964ms):
k3s Trace[1687549248]: [2m10.964212472s] [2m10.964212472s] END
k3s I0924 11:53:04.418121      46 trace.go:205] Trace[1695746900]: "Proxy via http_connect protocol over tcp" address:10.42.0.9:8443 (24-Sep-2022 11:50:54.944) (total time: 129473ms):
k3s Trace[1695746900]: [2m9.473126709s] [2m9.473126709s] END
k3s I0924 11:53:04.418155      46 trace.go:205] Trace[1578279874]: "Proxy via http_connect protocol over tcp" address:10.42.0.8:8443 (24-Sep-2022 11:50:54.457) (total time: 129961ms):
k3s Trace[1578279874]: [2m9.961131206s] [2m9.961131206s] END
k3s I0924 11:53:04.418160      46 trace.go:205] Trace[406850929]: "Proxy via http_connect protocol over tcp" address:10.42.0.7:8443 (24-Sep-2022 11:50:53.944) (total time: 130473ms):
k3s Trace[406850929]: [2m10.473654339s] [2m10.473654339s] END
k3s I0924 11:53:04.418208      46 trace.go:205] Trace[1213933259]: "Proxy via http_connect protocol over tcp" address:10.42.0.9:8443 (24-Sep-2022 11:50:54.455) (total time: 129962ms):
k3s Trace[1213933259]: [2m9.962336223s] [2m9.962336223s] END
k3s I0924 11:53:04.418239      46 trace.go:205] Trace[1402600249]: "Proxy via http_connect protocol over tcp" address:10.42.0.7:8443 (24-Sep-2022 11:50:55.121) (total time: 129297ms):
k3s Trace[1402600249]: [2m9.297126293s] [2m9.297126293s] END
k3s I0924 11:53:04.418302      46 trace.go:205] Trace[1277249521]: "Proxy via http_connect protocol over tcp" address:10.42.0.8:8443 (24-Sep-2022 11:50:54.119) (total time: 130298ms):
k3s Trace[1277249521]: [2m10.298506336s] [2m10.298506336s] END
k3s I0924 11:53:04.418305      46 trace.go:205] Trace[688495020]: "Proxy via http_connect protocol over tcp" address:10.42.0.8:8443 (24-Sep-2022 11:50:53.454) (total time: 130964ms):
k3s Trace[688495020]: [2m10.964143771s] [2m10.964143771s] END
k3s I0924 11:53:05.376142      46 trace.go:205] Trace[1922244545]: "Call mutating webhook" configuration:gatekeeper-mutating-webhook-configuration,webhook:mutation.gatekeeper.sh,resource:/v1, Resource=configmaps,subresource:,operation:UPDATE,UID:bfb30054-8598-496e-9cd7-cbb4765fa8e1 (24-Sep-2022 11:53:04.375) (total time: 1000ms):
k3s Trace[1922244545]: [1.000712635s] [1.000712635s] END
k3s W0924 11:53:05.376198      46 dispatcher.go:180] Failed calling webhook, failing open mutation.gatekeeper.sh: failed calling webhook "mutation.gatekeeper.sh": failed to call webhook: Post "<https://gatekeeper-webhook-service.gatekeeper-system.svc:443/v1/mutate?timeout=1s>": context deadline exceeded
k3s E0924 11:53:05.376218      46 dispatcher.go:184] failed calling webhook "mutation.gatekeeper.sh": failed to call webhook: Post "<https://gatekeeper-webhook-service.gatekeeper-system.svc:443/v1/mutate?timeout=1s>": context deadline exceeded

Any ideas why the proxying wouldn't work for the webhook but works for logs and curl?

Hi, I'm using k3s and have mysql as backend server and have now run into a issue where I have reached the maximum number for the

id

field. I receive the error

Duplicate entry '2147483647' for key 'PRIMARY'

from the database. Any hints on how to fix this?

Does rancher/suse provide addon license for k3s ?

Hi everyone, I posted this in #general but I think this is more appropriate. I'm hoping I can get some help standing up a k3s cluster on bare metal: I'm using k3sup, and I've setup 3 servers and 3 workers, they all show as ready.

flannel-backend=host-gw

All nodes are reachable between each other and from workers, I see the cni0 interface and I can ping pods from the host. On server I cannot ping pods form the host, and there is no cni0 interface. Servers have a

--node-taint='k3s-controlplane=true:NoSchedule'

taint, which I assumed meant that non system workloads aren't scheduled, but looks like no workloads are scheduled on workers at all, including coredns, etc. I ran out of places to look for a cause, does anyone have any insight? Thanks in advance

Is Postgres a commonly used backend? Does anyone have any performance data vs etcd?

I have a question about the "pending" status for my cluster, am I able to re-register a cluster with the same name as one I deleted? I attempted to test system upgrade controller on the cluster but it became unreachable so I had to delete it and launch a new cluster with the same name. The new cluster is stuck in the pending state and I am not sure how to get it active.

I know there is an open issue out there with a little bit of action every once in a while https://github.com/rancher/rancher/issues/36463 But just curious if any one out here has any info on if arm64 is going to be supported from the rancher UI when creating clusters.

where does k3s store the client-ca and in how much pain would I be if I rotate it?