Hi. I'm building a brand new k3s cluster. Currently I have it up and running along with Ceph. I will be installing a variety of opensource services (e.g. NGINX). I'll also be setting up custom built services. In a past K3S and Vanilla K8S clusters I also instantiated a private docker registry within the cluster to serve custom images for those custom services. I know in recent releases k8s has stepped away from the docker toolsuite and is solidifying around containerd, and CRI-O. Since I have flexibility to build the exact cluster I want, I am looking for advice. Questions I have? Should I use containerd, or CRI-O? Does k3s support both? What tools (other than docker) should I consider for building compliant container images? Rather than deploy a private docker registry in my k3s cluster is there an alternative recommended registry that I should consider using?

Hi. I've got a simple public image in dockerhub. I have no problem pushing or pulling this image using docker. I'm trying to use it in my k3s cluster. I see the following error:

Events:
  Type     Reason     Age   From               Message
  ----     ------     ----  ----               -------
  Normal   Scheduled  7s    default-scheduler  Successfully assigned default/test-59697b49db-tgmw2 to worker1
  Normal   Pulling    105s  kubelet            Pulling image "dsargrad/test:0.1.0"
  Warning  Failed     103s  kubelet            Failed to pull image "dsargrad/test:0.1.0": rpc error: code = Unknown desc = failed to pull and unpack image "<http://docker.io/dsargrad/test:0.1.0|docker.io/dsargrad/test:0.1.0>": failed to copy: httpReadSeeker: failed open: failed to do request: Get "<https://production.cloudflare.docker.com/registry-v2/docker/registry/v2/blobs/sha256/c8/c852233570204549536abaa9a881803c38ef6192a370d7791bd480e7102d7560/data?verify=1654961268-HeAOL1x8aj7nmL5RUXQv1ApRkvY%3D>": x509: certificate has expired or is not yet valid: current time 2022-06-10T13:32:23-04:00 is before 2022-06-11T00:00:00Z
  Warning  Failed     103s  kubelet            Error: ErrImagePull
  Normal   BackOff    102s  kubelet            Back-off pulling image "dsargrad/test:0.1.0"
  Warning  Failed     102s  kubelet            Error: ImagePullBackOff

hello! I’ve got a question about the k3s images: is there supposed to be a

v1.21.13-k3s1

image? it doesn’t seem to exist: https://hub.docker.com/r/rancher/k3s/tags?page=1&name=1.21.13-k3s1

Hi, I was wondering how I can set up K3s on a cluster where the nodes are on different networks. I'm using K3s in a project where my worker nodes are running on edge devices, and my control node is running in the cloud.

Quick and probably easy question - can Rancher deploy k3s clusters? I know it can manage them but I’m wondering how people manage rollouts at scale. Thanks.

Hi, I have two interfaces, ens2 external and ens5 internal vpc, I wanted my cluster to use the internal network, and to achieve that I had to add flags, but I'm not sure if I'm missing something or if I'm making changes that are not required:

Server : not sure if it's needed to change the service-cidr  & cluster-dns

ExecStart=/usr/local/bin/k3s server --data-dir /var/lib/rancher/k3s 
            --flannel-iface "ens5" 
            --kube-scheduler-arg  address=${server_ip} 
            --kubelet-arg address=${server_ip} 
            --bind-address ${server_ip} 
            --cluster-cidr 172.17.0.0/16 
            --service-cidr 172.18.0.0/16 --cluster-dns 172.18.0.10 
            --advertise-address ${server_ip} --tls-san value ${server_ip}

nodes : only defined the interface i wanted it to use

ExecStart=/usr/local/bin/k3s agent --server https://{{ k3s_server_address }}:6443 --token {{token}} --flannel-iface "ens5"

Does it look like I upgraded right? But It's stil 1.23? I deployed update to newest stable version. I don't know if 1.24 is? Btw I am very green

Has anyone had any luck configuring specific cipher suites for traefik v1 in K3s? The default list of ciphers includes

TLS_RSA_WITH_3DES_EDE_CBC_SHA

which comes with this security warning:

64-bit block cipher 3DES vulnerable to SWEET32 attack

. By setting the list of ciphers in HelmChartConfig, checking my ciphers with

nmap

comes back with nothing, instead of an expected list of ciphers. No errors in traefik logs. This is my HelmChartConfig:

apiVersion: <http://helm.cattle.io/v1|helm.cattle.io/v1>
kind: HelmChartConfig
metadata:
  name: traefik
  namespace: kube-system
spec:
  valuesContent: |-
    ssl:
      enabled: true
      tlsMinVersion: VersionTLS12
      cipherSuites:
        - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
        - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
        - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305

k3s version v1.20.13+k3s1 (b8a1f455)

Good morning. Hey, I need to set

disable_apparmor

for containerd on an embedded project I am working on. Can anyone tell me how to do it, so k3s keeps my change?

[plugins.cri]
...
  disable_apparmor = true

/var/lib/rancher/k3s/agent/etc/containerd# ls config.toml config.toml.tmpl I know I need to make a tmpl file in that folder, but if I do, it seems to break the resulting config.toml. With like the GUID k3s install path and such…. I see the GO template https://github.com/k3s-io/k3s/blob/master/pkg/agent/templates/templates_linux.go

Hi Folks. I'd like to control my k3s cluster from outside the cluster. The documentation provided here, provides part of the answer. However it doesnt describe how to install kubectl. The machine that I'd like to connect to the cluster from is a Fedora 36 machine. I am using k3s version

[root@M ~]# kubectl version
Client Version: <http://version.Info|version.Info>{Major:"1", Minor:"23", GitVersion:"v1.23.6+k3s1", GitCommit:"418c3fa858b69b12b9cefbcff0526f666a6236b9", GitTreeState:"clean", BuildDate:"2022-04-28T22:16:18Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: <http://version.Info|version.Info>{Major:"1", Minor:"23", GitVersion:"v1.23.6+k3s1", GitCommit:"418c3fa858b69b12b9cefbcff0526f666a6236b9", GitTreeState:"clean", BuildDate:"2022-04-28T22:16:18Z", GoVersion:"go1.17.5", Compiler:"gc", Platform:"linux/amd64"}

How do I install just kubectl on my fedora 36 machine?

Simple question for the k3s experts. I've been using k3s for about a half year now. Prior to that I had setup my prior kubernetes clusters using "vanilla kubernetes".. all my efforts so far have been non-production. I know k3s is designed to be lightweight. Does this mean that it is potentially not a good platform for "load heavy" production platforms? How would you characterize when to choose an alternative k8s environment? I'll be driving towards making production decisions in the future. I like k3s very much. But I need to keep my eyes open to all system architectural and performance concerns. Any thoughts would be appreciated.

Hi. Quick Technical Question I've deployed a fresh k3s server.. using klipper. I've got an nginx service up and running and its exposed on an external ip.

(base) [dsargrad@localhost nginx]$ k get services

NAME         TYPE           CLUSTER-IP    EXTERNAL-IP                     PORT(S)          AGE

kubernetes   ClusterIP      10.43.0.1     <none>                          443/TCP          6h31m

nginx        LoadBalancer   10.43.228.3   192.168.56.133,192.168.56.134   9080:32741/TCP   31m

For some odd reason I can only access it on 192.168.56.134 (the worker). I cant access it on the master: 192.168.56.133.

(base) [dsargrad@localhost nginx]$ curl 192.168.56.133:9080

curl: (7) Failed to connect to 192.168.56.133 port 9080 after 0 ms: No route to host

(base) [dsargrad@localhost nginx]$ curl 192.168.56.134:9080

<!DOCTYPE html>

<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>

</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="<http://nginx.org/>"><http://nginx.org|nginx.org></a>.<br/>
Commercial support is available at
<a href="<http://nginx.com/>"><http://nginx.com|nginx.com></a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Any suggestions on why this is?

@creamy-pencil-82913 I have a mysql8.0.29 server setup on my aws ec2 which has been configured with require_secure_transport = ON and ssl related files. I have an user configured for k3s with REQUIRE SSL option. Now what steps I need to take to make a secure TLS connection from my k3s master node to this database server? I am a bit confused and clueless here.

Hello. I am looking into doing an air-gapped install of k3s containers (coredns, localpath provisioner, etc), but want to generate the containers tarball using custom containers. Is it possible to generate our own tarball that ctr with import the containers of on boot up rather than one of the releases as explained in the docs? Is there documentation on the expected format of the tarball? It looks like there is a manifest and registries file a part of the released airgap images. Do I need to create both a registry.yaml file along with the tarball?

Is there any way to get klipper to make use of letsencrypt?

Getting these in my k3s installation log file. Anyone can help?

Hello! I upgraded to 1.23.7-k3s1 from 1.23.6, now all my pods are constantly restarting with the message "Pod sandbox changed, it will be killed and re-created."

Any hints as to how to get that secret assigned? I'm using an internally generated cert from a local CA (and have had this working on an older version of rancher). I have my tls-ca secret and ingress secret set, but I don't know where this cattle-system/rancher secret is supposed to be assigned (certainly isn't listed in the docs)

looks like get.k3s.io and update.k3s.io are down

Hi I have a question which I need help with. I need to understand what is the support policy for older releases of k3s. Does k3s drop support for older releases once a new version is released or are previous versions supported but only till a given major release ? Thanks

export INSTALL_K3S_EXEC=--"https-listen-port xxxx --tls-san xxxxx --node-external-ip x.x.x.x --secrets-encryption true --advertise-address x.x.x.x --cluster-cidr x.x.x.x/16 --service-cidr x.x.x.x/16 --cluster-dns x.x.x.x --flannel-backend wireguard-native --disable traefik --protect-kernel-defaults=true --kube-apiserver-arg='audit-log-path=/var/lib/rancher/k3s/server/logs/audit.log' --kube-apiserver-arg='audit-policy-file=/var/lib/rancher/k3s/server/audit.yaml' --kube-apiserver-arg='audit-log-maxage=30' --kube-apiserver-arg='audit-log-maxbackup=10' --kube-apiserver-arg='audit-log-maxsize=100' --kube-apiserver-arg='request-timeout=300s' --kube-apiserver-arg='service-account-lookup=true' --kube-apiserver-arg='enable-admission-plugins=NodeRestriction,PodSecurityPolicy,NamespaceLifecycle,ServiceAccount' --kube-controller-manager-arg='terminated-pod-gc-threshold=10' --kube-controller-manager-arg='use-service-account-credentials=true' --kubelet-arg='streaming-connection-idle-timeout=5m' --kubelet-arg='make-iptables-util-chains=true'"

K3S install script is ignoring the whole environment variable for some reason. Anyone can help?

INSTALL_K3S_EXEC="--secrets-encryption"

or

INSTALL_K3S_EXEC="--secrets-encryption=true"

or

INSTALL_K3S_EXEC="--secrets-encryption true"

which one is the correct syntax to enable secrets encryption? The k3s docs have a mixed combination of the first two. A bit confused here to which one is actually working.

Hi, I installed rancher on my k3s cluster and it created an ingress rule on its own but something seems wrong. When I try to reach the dashboard, I get the Nginx 404 page. This is the output of the ingress:

NAME      CLASS    HOSTS                     ADDRESS   PORTS     AGE
rancher   <none>   <http://rancher.mydomain.eu|rancher.mydomain.eu>                 80, 443   24m

I'm not sure if it should have a class and why the address is blank. Any ideas?

I want to try out gRPC probes on v1.23.6 of k3s. But, I'm struggling with how to enable this alpha feature. Using the

kubelet-api-server-arg

parameter is not doing the job. Have anyone tried and successfully enabled the gRPC probe alpha feature on a k3s distribution? Thanks!!! 🙏

@high-waitress-66594 any idea how to check all the arguements that my k3s server is starting with in openrc system?

I'm not sure this question makes sense, but I'll try: Is there a way to list the current traefik ingress controller configuration? After it has configured itself from the various ingress'es in my cluster? I ask because I'm trying to debug why my ingress configuration isn't working as I expect.

Hello, I have setup k3s and facing the challenge when providing K3s_URL as the .local hostname. I have understood the @creamy-pencil-82913 comment. But by any chance is it possible to resolve it.. Or in upcoming updates team is planning to resolve it? https://github.com/k3s-io/k3s/issues/1395#issuecomment-702871386

Is SUSE/Rancher still involved with k3s development and is there any roadmap for RISCV support?

k3s worker node not joining master node Can someone help me with this issue ?????? Please I am stuck on this for past 2 days