Hello, cluster v1.21.5+k3s2 is installed, after restarting the service ( k3s) it applies the default kernel parameters ( net.netfilter.nf_conntrack_max). Ubuntu 18.04.5 LTS root@drm-set1-master01:~# sysctl -p net.core.somaxconn = 65535 net.ipv4.ip_local_port_range = 1024 65535 net.nf_conntrack_max = 4194304 net.netfilter.nf_conntrack_max = 4194304 fs.file-max = 2097152 net.ipv4.tcp_syncookies = 1 net.ipv4.tcp_max_syn_backlog = 65535 net.ipv4.ip_forward = 1 net.ipv4.ip_local_reserved_ports = 30000-32767 net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-arptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 root@drm-set1-master01:~# systemctl restart k3s root@drm-set1-master01:~# sysctl -a | grep net.netfilter.nf_conntrack_max sysctl: reading key "net.ipv6.conf.all.stable_secret" sysctl: reading key "net.ipv6.conf.cni0.stable_secret" sysctl: reading key "net.ipv6.conf.default.stable_secret" sysctl: reading key "net.ipv6.conf.ens160.stable_secret" sysctl: reading key "net.ipv6.conf.flannel/1.stable_secret" sysctl: reading key "net.ipv6.conf.kube-ipvs0.stable_secret" sysctl: reading key "net.ipv6.conf.lo.stable_secret" sysctl: reading key "net.ipv6.conf.veth000ece3e.stable_secret" sysctl: reading key "net.ipv6.conf.veth1412c3ee.stable_secret" sysctl: reading key "net.ipv6.conf.veth3d3b54df.stable_secret" net.netfilter.nf_conntrack_max = 131072

Hello! I am starting to look into setting up Rancher Management in HA mode. What is the best way for the management cluster setup if starting from scratch ? RCE2 as its newest, or something else ?

i ve just install rancher desktop in my pc and i use wsl with linux, i have this alert, how can i already have some config clusterin in the .kube/config how can i solve it?

Am i missing something, or why is

release-v2.7

branch of the Rancher Charts https://github.com/rancher/charts/tree/release-v2.7/charts still using extremely old upstream charts? Just looking at the

rancher-monitoring

chart it still uses

101.0.0+up19.0.3

as it's latest version,

19.0.3

is about 18 months old by now?? https://artifacthub.io/packages/helm/prometheus-community/kube-prometheus-stack/19.0.3

Hi all. We are running Rancher 2.6.9 and upgraded on of the clusters managed by rancher from 1.23.15 to 1.24.10 We saw a lot of things going wrong. Nodes had trouble communicating to each other, Some storage mounts ( Ceph block storage ) didn't want to unmount. We basically had to restart all our nodes one by one to recover everything. Has anyone else seen such behaviour? We do know that the cri-docker replaced the docker shim part, could this be a root cause of it?

Hi all, I'm looking for a role template definition that allows a restricted user to import manifest via Rancher UI (see attached image). Does anybody have any idea which resource I need to include in the Role Template definition please ? FYI below is my Role Template definition :

administrative: false
apiVersion: <http://management.cattle.io/v3|management.cattle.io/v3>
builtin: false
clusterCreatorDefault: false
context: project
description: "Default tenant role attached to a project"
displayName: project-default-role
external: false
hidden: false
kind: RoleTemplate
locked: false
metadata:
  finalizers:
    - <http://controller.cattle.io/mgmt-auth-roletemplate-lifecycle|controller.cattle.io/mgmt-auth-roletemplate-lifecycle>
  labels:
    <http://cattle.io/creator|cattle.io/creator>: norman
  name: project-default-role
projectCreatorDefault: false
roleTemplateNames: []
rules:
  - apiGroups:
      - ''
    nonResourceURLs: []
    resourceNames: []
    resources:
      - namespaces
    verbs:
      - get
  - apiGroups:
      - apps
    nonResourceURLs: []
    resourceNames: []
    resources:
      - deployments
    verbs:
      - create
      - delete
      - list
      - update
  - apiGroups:
      - ''
    nonResourceURLs: []
    resourceNames: []
    resources:
      - pods
    verbs:
      - create
      - delete
      - get
      - list
      - patch
      - update
      - watch
  - apiGroups:
      - ''
    nonResourceURLs: []
    resourceNames: []
    resources:
      - services
    verbs:
      - list
      - create
      - delete
      - update
  - apiGroups:
      - <http://events.k8s.io|events.k8s.io>
    nonResourceURLs: []
    resourceNames: []
    resources:
      - events
    verbs:
      - list
  ########################
  # Add here an apiGroup that allow user to import manifest
  ......

Thanks in advance ! 🙂

does anyone have a tl;dr on if i have multiple clusters on the same host machine (seperate vms) if i need to explicitly set the subnet blocks or will the two clusters work fine on the default?

i assume because they are seperate vm's that would make it fine, given how flannel etc set up vxlan

Hello All, I have IAM account in Account-A and EKS cluster and few Kubernetes clusters on EC2 in Account-B. We would like to have IAM account in Account-A. Is there a way for Rancher to create and/or manage the cluster through IAM assume role?

Hello Everyone, We lost for a problem with our DC our Rancher Server. The clusters fortunately were in a different DC. For some reason the backups are not working... If I create a brand new rancher server and Import the clusters. Would it work? Thanks ahead!

Hello, I don’t have nfs in storage class provisioner list.

image.png

Rancher v2.7 installed as docker container.

image.png

It should be enabled by default?

Nfs support for container mounting is built into k8s now, no need for an external provisioner

Hi all ; i am trying to add new worker node in my existing rancher cluster but getting below error . i really dont have any clue .connectivity is ok openssl s_client -connect rancher.lab:443 (no issue with ssl connectiity) Mar 23 043957 pol1.lab rancher-system-agent[11037]: time="2023-03-23T043957Z" level=debug msg="[Prober] (kubelet) running probe" Mar 23 043957 pol1.lab rancher-system-agent[11037]: time="2023-03-23T043957Z" level=debug msg="[Prober] (kubelet) retrieving existing probe status from map if existing" Mar 23 043957 pol1.lab rancher-system-agent[11037]: time="2023-03-23T043957Z" level=debug msg="Probe timeout duration: 5 seconds" Mar 23 043957 pol1.lab rancher-system-agent[11037]: time="2023-03-23T043957Z" level=debug msg="Probe output was Get \"http://127.0.0.1:10248/healthz\": dial tcp 127.0.0.110248 connect: connection refused" Mar 23 043957 pol1.lab rancher-system-agent[11037]: time="2023-03-23T043957Z" level=debug msg="Setting success threshold to 1" Mar 23 043957 pol1.lab rancher-system-agent[11037]: time="2023-03-23T043957Z" level=debug msg="Setting failure threshold to 2" Mar 23 043957 pol1.lab rancher-system-agent[11037]: time="2023-03-23T043957Z" level=debug msg="Probe failed" Mar 23 043957 pol1.lab rancher-system-agent[11037]: time="2023-03-23T043957Z" level=debug msg="[Prober] (kubelet) writing probe status to map" Mar 23 043957 pol1.lab rancher-system-agent[11037]: time="2023-03-23T043957Z" level=debug msg="[K8s] Enqueueing after 5.000000 seconds" Mar 23 043957 pol1.lab rancher-system-agent[11037]: time="2023-03-23T043957Z" level=debug msg="[K8s] secret data/string-data did not change, not updating secret"

Hi all, We are trying to install rancher 2.7 on k3s cluster(v1.23.17+k3s1) in a single node ec2 instance(t3.medium) We selected external tls and one replica. After the installation is done we are getting 404 page not found if we try to access rancher. I see some errors in the rancher logs

2023/03/23 03:36:04 [INFO] Starting <http://rke.cattle.io/v1|rke.cattle.io/v1>, Kind=RKECluster controller
2023/03/23 03:36:04 [ERROR] failed to start controller for <http://cluster.x-k8s.io/v1alpha3|cluster.x-k8s.io/v1alpha3>, Kind=Cluster: failed to wait for caches to sync
2023/03/23 03:36:04 [ERROR] failed to start controller for <http://cluster.x-k8s.io/v1alpha3|cluster.x-k8s.io/v1alpha3>, Kind=MachineHealthCheck: failed to wait for caches to sync
2023/03/23 03:36:04 [ERROR] failed to start controller for <http://cluster.x-k8s.io/v1alpha3|cluster.x-k8s.io/v1alpha3>, Kind=MachineSet: failed to wait for caches to sync
E0323 03:36:04.152482      33 gvks.go:69] failed to sync schemas: failed to sync cache for <http://rke-machine-config.cattle.io/v1|rke-machine-config.cattle.io/v1>, Kind=DigitaloceanConfig
2023/03/23 03:36:04 [INFO] Watching metadata for <http://rke-machine-config.cattle.io/v1|rke-machine-config.cattle.io/v1>, Kind=Amazonec2Config
2023/03/23 03:36:04 [INFO] Starting <http://cluster.x-k8s.io/v1alpha3|cluster.x-k8s.io/v1alpha3>, Kind=MachineDeployment controller
2023/03/23 03:36:04 [INFO] [CleanupOrphanBindingsDone] orphan bindings cleanup has already run, skipping
2023/03/23 03:36:04 [INFO] checking configmap cattle-system/admincreated to determine if orphan bindings cleanup needs to run
2023/03/23 03:36:04 [INFO] duplicate bindings cleanup has already run, skipping
2023/03/23 03:36:04 [INFO] Watching metadata for <http://rke-machine-config.cattle.io/v1|rke-machine-config.cattle.io/v1>, Kind=AzureConfig
2023/03/23 03:36:04 [INFO] [clean-catalog-orphan-bindings] cleaning up orphaned catalog bindings
2023/03/23 03:36:04 [INFO] Starting <http://rke-machine-config.cattle.io/v1|rke-machine-config.cattle.io/v1>, Kind=Amazonec2Config controller
2023/03/23 03:36:04 [INFO] [clean-catalog-orphan-bindings] Processing 2 rolebindings
2023/03/23 03:36:04 [INFO] [clean-catalog-orphan-bindings] Deleting orphaned role global-catalog
2023/03/23 03:36:04 [INFO] Watching metadata for <http://rke-machine-config.cattle.io/v1|rke-machine-config.cattle.io/v1>, Kind=DigitaloceanConfig
2023/03/23 03:36:04 [INFO] Starting <http://rke-machine-config.cattle.io/v1|rke-machine-config.cattle.io/v1>, Kind=AzureConfig controller
2023/03/23 03:36:04 [WARNING] [clean-catalog-orphan-bindings] Error when deleting role global-catalog, <http://roles.rbac.authorization.k8s.io|roles.rbac.authorization.k8s.io> "global-catalog" not found
2023/03/23 03:36:04 [WARNING] [CleanupOrphanCatalogBindingsDone] error during orphan binding cleanup: <http://roles.rbac.authorization.k8s.io|roles.rbac.authorization.k8s.io> "global-catalog" not found
2023/03/23 03:36:04 [ERROR] failed to cleanup orphan catalog bindings
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine-config.cattle.io/v1|rke-machine-config.cattle.io/v1>, Kind=HarvesterConfig
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine-config.cattle.io/v1|rke-machine-config.cattle.io/v1>, Kind=DigitaloceanConfig controller
2023/03/23 03:36:05 [INFO] driverMetadata: refreshing data from upstream <https://releases.rancher.com/kontainer-driver-metadata/release-v2.7/data.json>
2023/03/23 03:36:05 [INFO] Retrieve data.json from local path /var/lib/rancher-data/driver-metadata/data.json
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine-config.cattle.io/v1|rke-machine-config.cattle.io/v1>, Kind=LinodeConfig
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine-config.cattle.io/v1|rke-machine-config.cattle.io/v1>, Kind=VmwarevsphereConfig
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=VmwarevsphereMachine
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=AzureMachineTemplate
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=HarvesterMachineTemplate
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=Amazonec2Machine
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=DigitaloceanMachineTemplate
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=Amazonec2MachineTemplate
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=LinodeMachineTemplate
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=LinodeMachine
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=AzureMachine
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=DigitaloceanMachine
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=VmwarevsphereMachineTemplate
2023/03/23 03:36:05 [INFO] Watching metadata for <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=HarvesterMachine
2023/03/23 03:36:05 [INFO] Watching metadata for <http://cluster.x-k8s.io/v1alpha3|cluster.x-k8s.io/v1alpha3>, Kind=Cluster
2023/03/23 03:36:05 [INFO] Watching metadata for <http://cluster.x-k8s.io/v1alpha3|cluster.x-k8s.io/v1alpha3>, Kind=MachineHealthCheck
2023/03/23 03:36:05 [INFO] Watching metadata for <http://cluster.x-k8s.io/v1alpha3|cluster.x-k8s.io/v1alpha3>, Kind=MachineSet
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine-config.cattle.io/v1|rke-machine-config.cattle.io/v1>, Kind=HarvesterConfig controller
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine-config.cattle.io/v1|rke-machine-config.cattle.io/v1>, Kind=LinodeConfig controller
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine-config.cattle.io/v1|rke-machine-config.cattle.io/v1>, Kind=VmwarevsphereConfig controller
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=VmwarevsphereMachine controller
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=AzureMachineTemplate controller
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=HarvesterMachineTemplate controller
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=Amazonec2Machine controller
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=DigitaloceanMachineTemplate controller
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=Amazonec2MachineTemplate controller
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=LinodeMachineTemplate controller
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=LinodeMachine controller
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=AzureMachine controller
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=DigitaloceanMachine controller
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=VmwarevsphereMachineTemplate controller
2023/03/23 03:36:05 [INFO] Starting <http://rke-machine.cattle.io/v1|rke-machine.cattle.io/v1>, Kind=HarvesterMachine controller
2023/03/23 03:36:05 [INFO] Starting <http://cluster.x-k8s.io/v1alpha3|cluster.x-k8s.io/v1alpha3>, Kind=Cluster controller
2023/03/23 03:36:05 [INFO] Starting <http://cluster.x-k8s.io/v1alpha3|cluster.x-k8s.io/v1alpha3>, Kind=MachineHealthCheck controller
2023/03/23 03:36:05 [INFO] Starting <http://cluster.x-k8s.io/v1alpha3|cluster.x-k8s.io/v1alpha3>, Kind=MachineSet controller
2023/03/23 03:36:08 [INFO] Loaded configuration from <https://releases.rancher.com/kontainer-driver-metadata/release-v2.7/data.json> in [0x6d10098]
2023/03/23 03:36:08 [INFO] Loaded configuration from <https://releases.rancher.com/kontainer-driver-metadata/release-v2.7/data.json> in [0x6d10098]
2023/03/23 03:36:08 [INFO] kontainerdriver amazonelasticcontainerservice listening on address 127.0.0.1:42731
2023/03/23 03:36:08 [INFO] kontainerdriver googlekubernetesengine listening on address 127.0.0.1:41457
2023/03/23 03:36:08 [INFO] kontainerdriver azurekubernetesservice listening on address 127.0.0.1:45945
2023/03/23 03:36:08 [INFO] kontainerdriver amazonelasticcontainerservice stopped
2023/03/23 03:36:08 [INFO] dynamic schema for kontainerdriver amazonelasticcontainerservice updating
2023/03/23 03:36:08 [INFO] kontainerdriver azurekubernetesservice stopped
2023/03/23 03:36:08 [INFO] dynamic schema for kontainerdriver azurekubernetesservice updating
2023/03/23 03:36:08 [INFO] kontainerdriver googlekubernetesengine stopped
2023/03/23 03:36:08 [INFO] dynamic schema for kontainerdriver googlekubernetesengine updating

I'm a new to rancher and tried almost every solution that i found on google..

ya'll need to learn to put the log paste in a thread under the question 🐿️

Hello all, hope you are doing well, I'm not sure I'm communicating through the right channel I just joined the workspace, I'm trying to monitor a list of static targets , URLs, through cattle-monitoring-system, I can't find any information on the documentation on blackbox-exporter, unlike node-exporter doesn't have a subchart already installed, could you please advise on the best practice on monitoring static targets through cattle-monitoring-system ?

I am using Rancher v2.7.1 in a docker container and import an existing Kubernetes v1.24.8 (remote location). The setup works and I have a question. The App->Respository (git) The question is the git must be accessible from the Kubernetes location but not the location of the Rancher docker. Am I right to assume this?

Hi All, I have a question about Legacy Features. I need to disable this feature flag, but I don't see any documentation about what this enables or how I can audit to make sure I don't break an environment I have inherited. Outside of looking at the legacy features section in the UI, for each cluster, is there anything else this feature flag does?

hi, we have a few clusters within rancher, and 2 of them in the status "Updating", 2 of 3 master nodes remains in the state "Waiting for probes: kube-controller-manager, kube-scheduler" and i have no idea where to look since all services are up and running in those clusters. Rechecked scheduler and controller-manager for errors but there is nothing. Please help.

Has anyone deploy kubernetes cluster with multiple zones using rke 1.3.x?

Does anyone know if azure dev ops can connect to rke2 as a repo ?

Hi all, We have a Rancher cluster (AKS) as Management cluster with public LoadBalancer and we would like to expose a service internally to the downstream Rancher launched K8s clusters (AKS). Is there a way to achieve that without having to create an extra internal Ingress Controller ? Thanks!

I want to avoid exposing my DNS server credentials to every individual cluster, but still be able to use hostnames instead of IPs for each cluster node. so my idea was to generate them using the resources in the cluster rancher is hosted in. I was thought about using external-dns's CRD support to do that, but I'm not sure if that works with rancher's CRDs. has anyone ever done that or is there a better way maybe?

Opinion Post: I'm heartbroken over the fact that the Rancher Cluster Tools UI is gone. The new Cluster Manager is not user friendly. I'm trying to teach a group of people how to navigate and where to find things, some that aren't extremely technical and getting bad responses. It was a great UX and I don't understand the reason behind revamping it. It was very good as it was

I’m trying to set up rancher backups for my cluster. I’m running the “rancher backup crd” and the “rancher backup” helm chart with terraform. Both of those seem to install fine. When I try to apply a backup config file to set up the recurring backups, I get this error: no kind “Backup” is registered for version │ “resources.cattle.io/v1”. I had this working before I upgraded to rancher 2.7

Hello Everyone, We lost for a problem with our DC our Rancher Server. The clusters fortunately were in a different DC. For some reason the backups are not working... If I create a brand new rancher server and Import the clusters. Would it work? Thanks ahead!