https://rancher.com/ logo
Join the conversationJoin Slack
Channels
academy
amazon
arm
azure
cabpr
chinese
ci-cd
danish
deutsch
developer
elemental
epinio
espanol
events
extensions
fleet
français
gcp
general
harvester
harvester-dev
hobbyfarm
hypper
japanese
k3d
k3os
k3s
k3s-contributor
kim
kubernetes
kubewarden
lima
logging
longhorn-dev
longhorn-storage
masterclass
mesos
mexico
nederlands
neuvector-security
office-hours
one-point-x
onlinemeetup
onlinetraining
opni
os
ozt
phillydotnet
portugues
rancher-desktop
rancher-extensions
rancher-setup
rancher-wrangler
random
rfed_ara
rio
rke
rke2
russian
s3gw
service-mesh
storage
submariner
supermicro-sixsq
swarm
terraform-controller
terraform-provider-rancher2
terraform-provider-rke
theranchcast
training-0110
training-0124
training-0131
training-0207
training-0214
training-1220
ukranian
v16-v21-migration
vsphere
windows
Powered by Linen
general
  • a

    ancient-mechanic-72639

    12/10/2022, 4:02 PM
    however, I am trying to do it a little differently, I am trying to have a load balancer on my bare metal cluster to expose everything, and I am following this guide here https://vmguru.com/2021/04/how-to-install-rancher-on-k3s/
  • a

    ancient-mechanic-72639

    12/10/2022, 4:02 PM
    but when i apply the daemonset for kube-vip, i am met with an infinite crash back off loop, and i have been scratching my head trying to figure it out
  • a

    ancient-mechanic-72639

    12/10/2022, 4:03 PM
    the logs say it cannot find interface "ens192", which i set in my kubeconfig
  • a

    ancient-mechanic-72639

    12/10/2022, 4:04 PM
    i am new to this and am still very much experimenting, so forgive me if i am missing something simple. does anyone see something that i am missing? or would this be a better suited question for a different channel? thanks!
  • b

    billions-truck-68762

    12/11/2022, 5:42 AM
    I use minikube to start a Kubernetes cluster, after i install flannel, the node not ready because of cni config uninitialized.
  • b

    billions-truck-68762

    12/11/2022, 5:42 AM
    could some get the same situation or know why?
  • b

    billions-truck-68762

    12/11/2022, 5:44 AM
    kubectl get node -o yaml contain the below information:
    conditions:
        - lastHeartbeatTime: "2022-12-11T05:38:06Z"
          lastTransitionTime: "2022-12-11T05:38:06Z"
          message: Flannel is running on this node
          reason: FlannelIsUp
          status: "False"
          type: NetworkUnavailable
        - lastHeartbeatTime: "2022-12-11T05:40:15Z"
          lastTransitionTime: "2022-12-11T04:30:51Z"
          message: kubelet has sufficient memory available
          reason: KubeletHasSufficientMemory
          status: "False"
          type: MemoryPressure
        - lastHeartbeatTime: "2022-12-11T05:40:15Z"
          lastTransitionTime: "2022-12-11T04:30:51Z"
          message: kubelet has no disk pressure
          reason: KubeletHasNoDiskPressure
          status: "False"
          type: DiskPressure
        - lastHeartbeatTime: "2022-12-11T05:40:15Z"
          lastTransitionTime: "2022-12-11T04:30:51Z"
          message: kubelet has sufficient PID available
          reason: KubeletHasSufficientPID
          status: "False"
          type: PIDPressure
        - lastHeartbeatTime: "2022-12-11T05:40:15Z"
          lastTransitionTime: "2022-12-11T04:30:51Z"
          message: 'container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady
            message:docker: network plugin is not ready: cni config uninitialized'
          reason: KubeletNotReady
          status: "False"
          type: Ready
  • b

    billions-truck-68762

    12/11/2022, 5:45 AM
    and flannel logs like this:
  • b

    billions-truck-68762

    12/11/2022, 5:45 AM
    [root@VM-8-2-centos ~]# kubectl -n kube-flannel logs kube-flannel-ds-hrlp9
    Defaulted container "kube-flannel" out of: kube-flannel, install-cni-plugin (init), install-cni (init)
    I1211 05:38:04.531603       1 main.go:204] CLI flags config: {etcdEndpoints:<http://127.0.0.1:4001>,<http://127.0.0.1:2379> etcdPrefix:/coreos.com/network etcdKeyfile: etcdCertfile: etcdCAFile: etcdUsername: etcdPassword: version:false kubeSubnetMgr:true kubeApiUrl: kubeAnnotationPrefix:<http://flannel.alpha.coreos.com|flannel.alpha.coreos.com> kubeConfigFile: iface:[] ifaceRegex:[] ipMasq:true ifaceCanReach: subnetFile:/run/flannel/subnet.env publicIP: publicIPv6: subnetLeaseRenewMargin:60 healthzIP:0.0.0.0 healthzPort:0 iptablesResyncSeconds:5 iptablesForwardRules:true netConfPath:/etc/kube-flannel/net-conf.json setNodeNetworkUnavailable:true}
    W1211 05:38:04.531676       1 client_config.go:617] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
    I1211 05:38:04.540479       1 kube.go:126] Waiting 10m0s for node controller to sync
    I1211 05:38:04.631479       1 kube.go:431] Starting kube subnet manager
    I1211 05:38:04.633428       1 kube.go:452] Creating the node lease for IPv4. This is the n.Spec.PodCIDRs: [10.244.0.0/24]
    I1211 05:38:05.631564       1 kube.go:133] Node controller sync successful
    I1211 05:38:05.631583       1 main.go:224] Created subnet manager: Kubernetes Subnet Manager - vm-8-2-centos
    I1211 05:38:05.631587       1 main.go:227] Installing signal handlers
    I1211 05:38:05.631659       1 main.go:467] Found network config - Backend type: vxlan
    I1211 05:38:05.631696       1 match.go:206] Determining IP address of default interface
    I1211 05:38:05.632363       1 match.go:259] Using interface with name eth0 and address 10.0.8.2
    I1211 05:38:05.632394       1 match.go:281] Defaulting external address to interface address (10.0.8.2)
    I1211 05:38:05.632466       1 vxlan.go:138] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false
    I1211 05:38:05.639390       1 main.go:416] Current network or subnet (10.244.0.0/16, 10.244.0.0/24) is not equal to previous one (0.0.0.0/0, 0.0.0.0/0), trying to recycle old iptables rules
    I1211 05:38:05.640425       1 kube.go:452] Creating the node lease for IPv4. This is the n.Spec.PodCIDRs: [10.244.0.0/24]
    I1211 05:38:05.935150       1 main.go:342] Setting up masking rules
    I1211 05:38:05.937659       1 main.go:364] Changing default FORWARD chain policy to ACCEPT
    I1211 05:38:06.031834       1 main.go:379] Wrote subnet file to /run/flannel/subnet.env
    I1211 05:38:06.031856       1 main.go:383] Running backend.
    I1211 05:38:06.032031       1 vxlan_network.go:61] watching for new subnet leases
    I1211 05:38:06.132099       1 main.go:404] Waiting for all goroutines to exit
    I1211 05:38:06.333454       1 iptables.go:270] bootstrap done
    I1211 05:38:06.335965       1 iptables.go:270] bootstrap done
  • s

    straight-actor-37028

    12/11/2022, 7:55 AM
    Hello All, I was trying to configure external haproxy loadbalancer for Rancher, when i edit 50-rancher.yaml with LB IP, its getting overwritten by Rancher, how can i make it permanent.?
    {
    "node-label": [ "cattle.io/os=linux", "rke.cattle.io/machine=35570e9e-4d18-4f14-b6f2-5e3dee9dec51" ], "private-registry": "/etc/rancher/rke2/registries.yaml", "profile": "cis-1.6", "protect-kernel-defaults": true, "server": "https://10.200.9.4:9345", "token": ""
  • c

    cuddly-breakfast-76667

    12/11/2022, 8:36 AM
    Hello All; i am using rancher and deployed kuberentes successfully but when try to get any logs from custom .net image i recieve below error failed to try resolving symlinks in path "/var/log/pods/mail-6ff69cf96-c2pm9_88b40a03-48ff-4198-b1ae-25e8934791d7/mail/0.log": lstat /var/log/pods_m ail-6ff69cf96-c2pm9_88b40a03-48ff-4198-b1ae-25e8934791d7/mail/0.log: no such file or directory[
  • l

    lively-gpu-91507

    12/11/2022, 12:05 PM
    hello, when I try to add a system target in suse manager ,all is well and salt key was acceptted but the system does not listed in the systems , there no systems so, anyone any idea?
    w
    • 2
    • 1
  • b

    breezy-ram-80329

    12/12/2022, 7:49 AM
    1.Does the rancher server needs to talk to the k8s api server other than installing the cluster agent ? 2. can we do the cluster agent installation manually, because my cluster api server is not exposed to the public ?
  • b

    breezy-ram-80329

    12/12/2022, 8:27 AM
    can we authenticate rancher server to the downstream cluster api server through oidc proxy ?
  • m

    magnificent-balloon-44813

    12/12/2022, 8:28 AM
    s/dnd
  • a

    astonishing-rain-93930

    12/12/2022, 9:26 AM
    Hi! Im trying to install Rancher on one of our clusters, but getting the following error from the 3 rancher-fdb454d45 pods:
    [FATAL] <http://clusters.management.cattle.io|clusters.management.cattle.io> is forbidden: User "system:serviceaccount:cattle-system:rancher" cannot list resource "clusters" in API group "<http://management.cattle.io|management.cattle.io>" at the cluster scope
    Any idea what this might be about? Thanks!
    d
    • 2
    • 3
  • s

    swift-wall-53633

    12/12/2022, 12:51 PM
    Hi! I have installed Rancher Desktop, and clicked on the option:
    Expose Rancher Desktop's Kubernetes configuration and Docker socket to Windows Subsystem for Linux (WSL) distros
    But I have to run as root, or else I get :
    Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "<http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/json>": dial unix /var/run/docker.sock: connect: permission denied
    Because
    $ ls -l /var/run/docker.sock
    srwxr-xr-x 1 root root 0 Dec 12 13:46 /var/run/docker.sock
    I'd like to run with some group membership, so I can run docker cli as a normal user
    w
    • 2
    • 1
  • s

    square-alarm-76296

    12/12/2022, 12:51 PM
    hi, does anyone know how to create this same bash script for project role https://gist.github.com/superseb/cad9b87c844f166b9c9bf97f5dea1609
  • d

    dazzling-jewelry-63449

    12/12/2022, 1:29 PM
    Hi, it might be a stupid question, but I want to add a new node to my Rancher cluster, according to the documentation I need to edit the cluster.yml to do so, but where do I find this file?
  • d

    damp-magician-5939

    12/12/2022, 1:45 PM
    Hello everyone, I am trying to do something but can't really find much docs, maybe I'm looking in the wrong place. My team is using Rancher to manage 3 clusters and we have setup the Rancher App for monitoring (Basic installation) and while we already have a good amount of alerts available on Prometheus, we have some specific ones that we need to add (Disk pressure metrics, some network stuff) and Only thing I can find in Rancher is adding the Alert by hand, but I'm not really sure if 1) I am doing it the correct way and 2) If there's a way to setup a bunch of alerts in a yaml and use it somehow. Is there any docs or can someone provide some guidance on this proccess?
  • a

    aloof-glass-99040

    12/12/2022, 4:02 PM
    Hello everyone! 👋 I had a couple of questions about rancher's integration with cloud providers: On DigitalOcean Kubernetes, we have observed that we have quite a few users using rancher. 🙂 They install the rancher related webhooks in the cattle-system namespace and some of these webhooks have rules for basically all objects in the kube-system namespace. We have observed for some time now that this causes the cluster to break during upgrades especially because the rancher webhook prevents managed components that DigitalOcean installs in the kube-system namespace from coming up. I was wondering if this issue is something that is common to other cloud providers too? The mitigation so far has been to edit the webhook to set FailurePolicy of
    Ignore
    , finish the upgrade. At the end of it, the webhook configuration is reset back to the original failurePolicy of Fail. We usually advise the users to exclude the managed components in the kube-system ns from the webhook configuration, but I suspect that they probably use helm or something provided from rancher and just apply. I was wondering if there any mitigation strategies you would recommend? Thank you! 🙂
  • b

    blue-jewelry-65830

    12/12/2022, 7:27 PM
    Hi All,
  • b

    blue-jewelry-65830

    12/12/2022, 7:28 PM
    The question is about the completed installation of Rancher version 2.7.0 in the AirGapped environment. We noticed the unavailability of vSphere CSI helm chart. Has anyone encountered a similar problem ?
  • c

    creamy-accountant-88363

    12/12/2022, 7:46 PM
    What is the use of the
    tls-rancher-internal-ca
    secret and the
    /cacerts
    endpoint? I have noticed when using Rancher with a private CA, this secret has a "dynamiclistener" Rancher-generated secret, instead of the user provided TLS CA + Key. This can cause an issue if you're using RKE2, since the RKE2 provisioning jobs will check the
    /cacerts
    endpoint, get the invalid CA, and fail. Manually updating the
    tls-rancher-internal-ca
    secret will fix this issue if you're using private CA as a workaround. Any thoughts? Thanks. FYI I am using Rancher 2.6.8, with the Rancher helm chart.
  • w

    white-yacht-56857

    12/12/2022, 7:51 PM
    Hello guys, I have two clusters created via RKE that have "Unavailable" status in Rancher. I'm looking for a way to understand why this problem happened. But I would like to know if there is any way to add this previously created cluster to Rancher?
    s
    • 2
    • 1
  • q

    quiet-house-51252

    12/12/2022, 8:02 PM
    Hello, Can anyone help me how to get the user email id of the user in Rancher dashboard, when someone login rancher server with either GitHub, google, or another Identity service provider account
  • s

    stocky-vr-48505

    12/12/2022, 9:22 PM
    Hi gang. Often, when I open up my laptop after it goes to sleep Rancher Desktop seems "dead". Using k9s the cluster cpu/memory is 0. Multiple deployments fail to restart properly and I usually need to completely wipe and recreate the cluster. Is there a guide on how to avoid this?
    w
    • 2
    • 1
  • m

    most-crowd-3167

    12/12/2022, 9:50 PM
    I'm using Rancher to deploy RKE2 on EC2 instances. Is there any way to attach multiple network interfaces to each node in the cluster (i.e. one interface per subnet in a VPC)
  • e

    early-solstice-46134

    12/13/2022, 7:40 AM
    Any experts on the Rancher API around? I got some weird behaviour and before hunting bugs for hours, maybe knows something: I'm deploying the monitoring application into the cluster in which Rancher is installed via API without any problems. When I try to deploy the monitoring app to a cluster managed by Rancher using the exact same approach (with an adjusted API call, of course), this does not work. Rancher responds to the call with a
    201
    response code, but never actually starts rolling out the monitoring stack to the managed cluster.
    • 1
    • 1
  • s

    steep-window-46329

    12/13/2022, 10:39 AM
    Hello everyone. We are experiencing stability issues with downstream clusters where memory usage is high on worker nodes. This seems to be expected because there is no resource-reservation (
    kube-reserved
    ,
    system-reserved
    ) set per default on kubernetes and rancher. So there are evictions happening and in most cases just the OOM-Killer of Systemd running wild. We've set a custom value for
    eviction-hard
    and
    system-reserved
    and this seems to improve the stability of the worker-nodes and in case of low memory, the correct pods are getting killed instead of random system processes. Is there a general recommendation from Rancher for how much memory and CPU should be reserved on downstream-clusters based on the size of the worker-node? It would help a lot to get some general guidelines. Thanks! Reserve Compute Resources for System Daemons https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/
Powered by Linen
Title
s

steep-window-46329

12/13/2022, 10:39 AM
Hello everyone. We are experiencing stability issues with downstream clusters where memory usage is high on worker nodes. This seems to be expected because there is no resource-reservation (
kube-reserved
,
system-reserved
) set per default on kubernetes and rancher. So there are evictions happening and in most cases just the OOM-Killer of Systemd running wild. We've set a custom value for
eviction-hard
and
system-reserved
and this seems to improve the stability of the worker-nodes and in case of low memory, the correct pods are getting killed instead of random system processes. Is there a general recommendation from Rancher for how much memory and CPU should be reserved on downstream-clusters based on the size of the worker-node? It would help a lot to get some general guidelines. Thanks! Reserve Compute Resources for System Daemons https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/
View count: 42