Hi, I have a k8s cluster managed by harvester, the cluster only has 1 node. I had to reboot the machine and would like to start the cluster again. I run the command

systemctl start rancher-system-agent.service && systemctl start rke2-agent.service && systemctl start rke2-server.service

but

rke2-agent

fails to start journalctl says:

Apr 28 12:03:30 nid003204 systemd[1]: rke2-agent.service: Failed with result 'exit-code'.
Apr 28 12:03:30 nid003204 systemd[1]: Failed to start Rancher Kubernetes Engine v2 (agent).
Apr 28 12:03:36 nid003204 systemd[1]: rke2-agent.service: Scheduled restart job, restart counter is at 42.
Apr 28 12:03:36 nid003204 systemd[1]: Stopped Rancher Kubernetes Engine v2 (agent).
Apr 28 12:03:36 nid003204 systemd[1]: Starting Rancher Kubernetes Engine v2 (agent)...
Apr 28 12:03:36 nid003204 sh[134363]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Apr 28 12:03:36 nid003204 sh[134364]: Failed to get unit file state for nm-cloud-setup.service: No such file or directory
Apr 28 12:03:36 nid003204 rke2[134367]: time="2023-04-28T12:03:36+02:00" level=warning msg="Unknown flag --agent-token found in config.yaml, skipping\n"
Apr 28 12:03:36 nid003204 rke2[134367]: time="2023-04-28T12:03:36+02:00" level=warning msg="Unknown flag --cni found in config.yaml, skipping\n"
Apr 28 12:03:36 nid003204 rke2[134367]: time="2023-04-28T12:03:36+02:00" level=warning msg="Unknown flag --disable found in config.yaml, skipping\n"
Apr 28 12:03:36 nid003204 rke2[134367]: time="2023-04-28T12:03:36+02:00" level=warning msg="Unknown flag --disable-kube-proxy found in config.yaml, skipping\n"
Apr 28 12:03:36 nid003204 rke2[134367]: time="2023-04-28T12:03:36+02:00" level=warning msg="Unknown flag --etcd-expose-metrics found in config.yaml, skipping\n"
Apr 28 12:03:36 nid003204 rke2[134367]: time="2023-04-28T12:03:36+02:00" level=warning msg="Unknown flag --etcd-snapshot-retention found in config.yaml, skipping\n"
Apr 28 12:03:36 nid003204 rke2[134367]: time="2023-04-28T12:03:36+02:00" level=warning msg="Unknown flag --etcd-snapshot-schedule-cron found in config.yaml, skipping\n"
Apr 28 12:03:36 nid003204 rke2[134367]: time="2023-04-28T12:03:36+02:00" level=warning msg="Unknown flag --kube-controller-manager-arg found in config.yaml, skipping\n"
Apr 28 12:03:36 nid003204 rke2[134367]: time="2023-04-28T12:03:36+02:00" level=warning msg="Unknown flag --kube-controller-manager-arg found in config.yaml, skipping\n"
Apr 28 12:03:36 nid003204 rke2[134367]: time="2023-04-28T12:03:36+02:00" level=warning msg="Unknown flag --kube-scheduler-arg found in config.yaml, skipping\n"
Apr 28 12:03:36 nid003204 rke2[134367]: time="2023-04-28T12:03:36+02:00" level=warning msg="Unknown flag --kube-scheduler-arg found in config.yaml, skipping\n"
Apr 28 12:03:36 nid003204 rke2[134367]: time="2023-04-28T12:03:36+02:00" level=warning msg="not running in CIS mode"
Apr 28 12:03:36 nid003204 rke2[134367]: time="2023-04-28T12:03:36+02:00" level=fatal msg="--server is required"
Apr 28 12:03:36 nid003204 systemd[1]: rke2-agent.service: Main process exited, code=exited, status=1/FAILURE
Apr 28 12:03:36 nid003204 systemd[1]: rke2-agent.service: Failed with result 'exit-code'.
Apr 28 12:03:36 nid003204 systemd[1]: Failed to start Rancher Kubernetes Engine v2 (agent).

Any idea?

Hi 👋 team anyone can help me with this issue? https://rancher-users.slack.com/archives/C3ASABBMH/p1682613848166019 😞 the pod logs get up to

time="2023-04-28T11:14:09Z" level=info msg="Requesting kubelet certificate regeneration"

and is just stuck there.

Quick question...does SUSE have an opinion on RHEL vs SUSE for hosting Rancher if there is already a rather large in-house RHEL presence??

meaning SLES vs RHEL specifically...

macOS arm64 here: I am trying to create an ingress with the default traefik ingress controller, but all I am getting is

Bad Gateway

. And I am out of ideas. I have a fresh installation of rancher-desktop and I can see

traefik

running in the

kube-system

namespace. I have applied an http demo deployment (comes from ingress-nginx, but the deployment should be agnostic) and the ingress file below:

kubectl apply -f <https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/docs/examples/http-svc.yaml>

---
apiVersion: <http://networking.k8s.io/v1|networking.k8s.io/v1>
kind: Ingress
metadata:
  name: demo-ingress
spec:
  rules: 
  - http:
      paths:
        - pathType: Prefix
          path: "/"
          backend:
            service:
              name: http-svc
              port:
                number: 80
---

I end up with this ingress:

kubectl get <http://ingresses.networking.k8s.io|ingresses.networking.k8s.io>
NAME           CLASS     HOSTS   ADDRESS           PORTS   AGE
demo-ingress   traefik   *       192.168.130.178   80      24m

However, all requests to localhost or 192.168.130.178 return bad gateway:

# same with <http://192.168.130.178>
curl <http://127.0.0.1:80>
Bad Gateway

Anyone installed Harvester on a Dell Poweredge server? I can't get the install to complete.

Is it possible to identify all the available additional environment variables that can be passed to the Rancher pod by utilizing the "extraEnv" field in the values.yaml file ?

I’ve somehow got into a weird situation where I accidentally registered the local cluster on a downstream cluster. Everything seems to be corrected with one small issue in that downstream cluster will show the wrong cluster information intermittently. Anyone have any ideas what to check?

hello, please someone look at this issue https://rancher-users.slack.com/archives/C3ASABBD1/p1682329051244029?thread_ts=1681813524.796119&cid=C3ASABBD1

Hello, We are currently running rancher 2.6.9 and we would like to upgrade to 2.7.x Is direct upgrade supported or shall we better go first to 2.6.12 which was released 4 days ago.

Hey guys, does

globalRoleBinds

creates a

clusterRole

in each one of the downstream clusters?

I want to create a

globalClusterRole

which will be assigned to Okta group, but lookds like it dosn't work because the

clusterRole

is not being created in the downstrewam cluster

Can I Create Users Through API ?

FYI --> There is a NeuVector K8S Security Rodeo tomorrow, May 2, 2023 | 9:15 AM - 12:15 PM (CST)

Is there a way to tell my Longhorn storage architecture is badly designed? We are seeing frequent node crashes. Also the cluster is shared for storage and running the application PODS not sure if it is a good design to have a dedicated Storage cluster for Longhorn?

hey guys network issue post upgrade

I cant get it to either create new networks or delete old networkss

Hi anyone with experience with “rancher-vmware.iso” - when I boot the ISO the disk provided is like an overlay and I am not sure how to install the image to disk?

Is there any pro/cons using rancheros vs a ubuntu cloud image?

How to generate kubeconfig for ad user group as SA secret? For downstream clusters

Hello everyone, i need help to upgrade kubernates cluster , my kubernates cluster version is 1.22.9 and i want 1.22.17 , currently my rancher version is 2.6.6 , I have tried to upgrade rancher to 2.6.10 but after that cluster upgrade is failing

Hi, can anybody help out? Where is Rancher keeping the templates for the Roles? I see in the the custom roles/permissions in UI but cannot find any k8s object that would contain the data

Can Rancher be configured with an external identity provider using environment variables?

Reading over the documentation for Rancher server disaster recovery it seems like there is not a good method for running Rancher (server) in a multi-site setup. Is the only option to have a standby cluster that backups get sync'd to ( or a remove store ) and then recovered and Rancher reinstalled via Helm on the secondary site ? Is anyone else running Rancher in a multi site setup at all ?

Hello everyone 🙂 Is there a possibility to the following config in a rke2 downstream-cluster?

kubelet-arg:
  - container-log-max-files=3
  - container-log-max-size=2Mi

I found this issue (https://github.com/rancher/rke2/discussions/3002#discussioncomment-2862822) but it seems this only works in the upstrean-cluster. Thanks and regards

Hi all, i wondering if it’s possible to avoid Rancher automatically creating a secret associated with a serviceAccount ( for user management purpose).

kubectl

is showing a Warning at each running command :

kubectl get nodes
Warning: Use tokens from the TokenRequest API or manually created secret-based tokens instead of auto-generated secret-based tokens.
NAME                         STATUS   ROLES           AGE   VERSION
...
...

If anyone has a workaround ? 😉

Hello, I just installed Rancher Desktop and running into an issue running kubectl version. I am getting following error. WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short. Use --output=yaml|json to get the full version. Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.3", GitCommit:"434bfd82814af038ad94d62ebe59b133fcb50506", GitTreeState:"clean", BuildDate:"2022-10-12T105726Z", GoVersion:"go1.19.2", Compiler:"gc", Platform:"linux/amd64"} Kustomize Version: v4.5.7 Error from server (NotFound): the server could not find the requested resource

My Rancher is waiting for a VM to be deleted that isn’t present in Azure any longer. How can I clean up this mess ? I’m using the Azure Cloud provider.

Hello team, There seems to be an issue with the k3s uninstallation on an airgapped environment(no internet) with the k3s-uninstall.sh script. The uninstallation hangs for too long at the

yum remove -y k3s-selinux

step. Since there is no internet this step times out after a long time(5+mins I guess). Shldn't we handle this case specifically for the airgap uninstallation? Tried searching, but cldn't find any issue in reference to this. Would appreciate any help on this.

hello everyone i have a issue while trying to enable the extentions i don’t see in the docs how to enable it, I have upgraded from v2.6.3 to 2.7.3 anyone can tell me how to enable extentions ?