rancher-users #general

ancient-mechanic-72639

12/10/2022, 4:03 PM

the logs say it cannot find interface "ens192", which i set in my kubeconfig

ancient-mechanic-72639

12/10/2022, 4:04 PM

i am new to this and am still very much experimenting, so forgive me if i am missing something simple. does anyone see something that i am missing? or would this be a better suited question for a different channel? thanks!

billions-truck-68762

12/11/2022, 5:42 AM

I use minikube to start a Kubernetes cluster, after i install flannel, the node not ready because of cni config uninitialized.

billions-truck-68762

12/11/2022, 5:42 AM

could some get the same situation or know why?

billions-truck-68762

12/11/2022, 5:44 AM

kubectl get node -o yaml contain the below information:

Copy code

conditions:
    - lastHeartbeatTime: "2022-12-11T05:38:06Z"
      lastTransitionTime: "2022-12-11T05:38:06Z"
      message: Flannel is running on this node
      reason: FlannelIsUp
      status: "False"
      type: NetworkUnavailable
    - lastHeartbeatTime: "2022-12-11T05:40:15Z"
      lastTransitionTime: "2022-12-11T04:30:51Z"
      message: kubelet has sufficient memory available
      reason: KubeletHasSufficientMemory
      status: "False"
      type: MemoryPressure
    - lastHeartbeatTime: "2022-12-11T05:40:15Z"
      lastTransitionTime: "2022-12-11T04:30:51Z"
      message: kubelet has no disk pressure
      reason: KubeletHasNoDiskPressure
      status: "False"
      type: DiskPressure
    - lastHeartbeatTime: "2022-12-11T05:40:15Z"
      lastTransitionTime: "2022-12-11T04:30:51Z"
      message: kubelet has sufficient PID available
      reason: KubeletHasSufficientPID
      status: "False"
      type: PIDPressure
    - lastHeartbeatTime: "2022-12-11T05:40:15Z"
      lastTransitionTime: "2022-12-11T04:30:51Z"
      message: 'container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady
        message:docker: network plugin is not ready: cni config uninitialized'
      reason: KubeletNotReady
      status: "False"
      type: Ready

billions-truck-68762

12/11/2022, 5:45 AM

and flannel logs like this:

billions-truck-68762

12/11/2022, 5:45 AM

Copy code

[root@VM-8-2-centos ~]# kubectl -n kube-flannel logs kube-flannel-ds-hrlp9
Defaulted container "kube-flannel" out of: kube-flannel, install-cni-plugin (init), install-cni (init)
I1211 05:38:04.531603       1 main.go:204] CLI flags config: {etcdEndpoints:<http://127.0.0.1:4001>,<http://127.0.0.1:2379> etcdPrefix:/coreos.com/network etcdKeyfile: etcdCertfile: etcdCAFile: etcdUsername: etcdPassword: version:false kubeSubnetMgr:true kubeApiUrl: kubeAnnotationPrefix:<http://flannel.alpha.coreos.com|flannel.alpha.coreos.com> kubeConfigFile: iface:[] ifaceRegex:[] ipMasq:true ifaceCanReach: subnetFile:/run/flannel/subnet.env publicIP: publicIPv6: subnetLeaseRenewMargin:60 healthzIP:0.0.0.0 healthzPort:0 iptablesResyncSeconds:5 iptablesForwardRules:true netConfPath:/etc/kube-flannel/net-conf.json setNodeNetworkUnavailable:true}
W1211 05:38:04.531676       1 client_config.go:617] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I1211 05:38:04.540479       1 kube.go:126] Waiting 10m0s for node controller to sync
I1211 05:38:04.631479       1 kube.go:431] Starting kube subnet manager
I1211 05:38:04.633428       1 kube.go:452] Creating the node lease for IPv4. This is the n.Spec.PodCIDRs: [10.244.0.0/24]
I1211 05:38:05.631564       1 kube.go:133] Node controller sync successful
I1211 05:38:05.631583       1 main.go:224] Created subnet manager: Kubernetes Subnet Manager - vm-8-2-centos
I1211 05:38:05.631587       1 main.go:227] Installing signal handlers
I1211 05:38:05.631659       1 main.go:467] Found network config - Backend type: vxlan
I1211 05:38:05.631696       1 match.go:206] Determining IP address of default interface
I1211 05:38:05.632363       1 match.go:259] Using interface with name eth0 and address 10.0.8.2
I1211 05:38:05.632394       1 match.go:281] Defaulting external address to interface address (10.0.8.2)
I1211 05:38:05.632466       1 vxlan.go:138] VXLAN config: VNI=1 Port=0 GBP=false Learning=false DirectRouting=false
I1211 05:38:05.639390       1 main.go:416] Current network or subnet (10.244.0.0/16, 10.244.0.0/24) is not equal to previous one (0.0.0.0/0, 0.0.0.0/0), trying to recycle old iptables rules
I1211 05:38:05.640425       1 kube.go:452] Creating the node lease for IPv4. This is the n.Spec.PodCIDRs: [10.244.0.0/24]
I1211 05:38:05.935150       1 main.go:342] Setting up masking rules
I1211 05:38:05.937659       1 main.go:364] Changing default FORWARD chain policy to ACCEPT
I1211 05:38:06.031834       1 main.go:379] Wrote subnet file to /run/flannel/subnet.env
I1211 05:38:06.031856       1 main.go:383] Running backend.
I1211 05:38:06.032031       1 vxlan_network.go:61] watching for new subnet leases
I1211 05:38:06.132099       1 main.go:404] Waiting for all goroutines to exit
I1211 05:38:06.333454       1 iptables.go:270] bootstrap done
I1211 05:38:06.335965       1 iptables.go:270] bootstrap done

straight-actor-37028

12/11/2022, 7:55 AM

Hello All, I was trying to configure external haproxy loadbalancer for Rancher, when i edit 50-rancher.yaml with LB IP, its getting overwritten by Rancher, how can i make it permanent.?

"node-label": [ "cattle.io/os=linux", "rke.cattle.io/machine=35570e9e-4d18-4f14-b6f2-5e3dee9dec51" ], "private-registry": "/etc/rancher/rke2/registries.yaml", "profile": "cis-1.6", "protect-kernel-defaults": true, "server": "https://10.200.9.4:9345", "token": ""

cuddly-breakfast-76667

12/11/2022, 8:36 AM

Hello All; i am using rancher and deployed kuberentes successfully but when try to get any logs from custom .net image i recieve below error failed to try resolving symlinks in path "/var/log/pods/mail-6ff69cf96-c2pm9_88b40a03-48ff-4198-b1ae-25e8934791d7/mail/0.log": lstat /var/log/pods_m ail-6ff69cf96-c2pm9_88b40a03-48ff-4198-b1ae-25e8934791d7/mail/0.log: no such file or directory[

lively-gpu-91507

12/11/2022, 12:05 PM

hello, when I try to add a system target in suse manager ,all is well and salt key was acceptted but the system does not listed in the systems , there no systems so, anyone any idea?

breezy-ram-80329

12/12/2022, 7:49 AM

1.Does the rancher server needs to talk to the k8s api server other than installing the cluster agent ? 2. can we do the cluster agent installation manually, because my cluster api server is not exposed to the public ?

breezy-ram-80329

12/12/2022, 8:27 AM

can we authenticate rancher server to the downstream cluster api server through oidc proxy ?

magnificent-balloon-44813

12/12/2022, 8:28 AM

~~s/dnd~~

astonishing-rain-93930

12/12/2022, 9:26 AM

Hi! Im trying to install Rancher on one of our clusters, but getting the following error from the 3 rancher-fdb454d45 pods:

Copy code

[FATAL] <http://clusters.management.cattle.io|clusters.management.cattle.io> is forbidden: User "system:serviceaccount:cattle-system:rancher" cannot list resource "clusters" in API group "<http://management.cattle.io|management.cattle.io>" at the cluster scope

Any idea what this might be about? Thanks!

swift-wall-53633

12/12/2022, 12:51 PM

Hi! I have installed Rancher Desktop, and clicked on the option:

Copy code

Expose Rancher Desktop's Kubernetes configuration and Docker socket to Windows Subsystem for Linux (WSL) distros

But I have to run as root, or else I get :

Copy code

Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "<http://%2Fvar%2Frun%2Fdocker.sock/v1.24/containers/json>": dial unix /var/run/docker.sock: connect: permission denied

Because

Copy code

$ ls -l /var/run/docker.sock
srwxr-xr-x 1 root root 0 Dec 12 13:46 /var/run/docker.sock

I'd like to run with some group membership, so I can run docker cli as a normal user

square-alarm-76296

12/12/2022, 12:51 PM

hi, does anyone know how to create this same bash script for project role https://gist.github.com/superseb/cad9b87c844f166b9c9bf97f5dea1609

dazzling-jewelry-63449

12/12/2022, 1:29 PM

Hi, it might be a stupid question, but I want to add a new node to my Rancher cluster, according to the documentation I need to edit the cluster.yml to do so, but where do I find this file?

damp-magician-5939

12/12/2022, 1:45 PM

Hello everyone, I am trying to do something but can't really find much docs, maybe I'm looking in the wrong place. My team is using Rancher to manage 3 clusters and we have setup the Rancher App for monitoring (Basic installation) and while we already have a good amount of alerts available on Prometheus, we have some specific ones that we need to add (Disk pressure metrics, some network stuff) and Only thing I can find in Rancher is adding the Alert by hand, but I'm not really sure if 1) I am doing it the correct way and 2) If there's a way to setup a bunch of alerts in a yaml and use it somehow. Is there any docs or can someone provide some guidance on this proccess?

aloof-glass-99040

12/12/2022, 4:02 PM

Hello everyone! 👋 I had a couple of questions about rancher's integration with cloud providers: On DigitalOcean Kubernetes, we have observed that we have quite a few users using rancher. 🙂 They install the rancher related webhooks in the cattle-system namespace and some of these webhooks have rules for basically all objects in the kube-system namespace. We have observed for some time now that this causes the cluster to break during upgrades especially because the rancher webhook prevents managed components that DigitalOcean installs in the kube-system namespace from coming up. I was wondering if this issue is something that is common to other cloud providers too? The mitigation so far has been to edit the webhook to set FailurePolicy of

Ignore

, finish the upgrade. At the end of it, the webhook configuration is reset back to the original failurePolicy of Fail. We usually advise the users to exclude the managed components in the kube-system ns from the webhook configuration, but I suspect that they probably use helm or something provided from rancher and just apply. I was wondering if there any mitigation strategies you would recommend? Thank you! 🙂

blue-jewelry-65830

12/12/2022, 7:27 PM

Hi All,

blue-jewelry-65830

12/12/2022, 7:28 PM

The question is about the completed installation of Rancher version 2.7.0 in the AirGapped environment. We noticed the unavailability of vSphere CSI helm chart. Has anyone encountered a similar problem ?

creamy-accountant-88363

12/12/2022, 7:46 PM

What is the use of the

tls-rancher-internal-ca

secret and the

/cacerts

endpoint? I have noticed when using Rancher with a private CA, this secret has a "dynamiclistener" Rancher-generated secret, instead of the user provided TLS CA + Key. This can cause an issue if you're using RKE2, since the RKE2 provisioning jobs will check the

/cacerts

endpoint, get the invalid CA, and fail. Manually updating the

tls-rancher-internal-ca

secret will fix this issue if you're using private CA as a workaround. Any thoughts? Thanks. FYI I am using Rancher 2.6.8, with the Rancher helm chart.

white-yacht-56857

12/12/2022, 7:51 PM

Hello guys, I have two clusters created via RKE that have "Unavailable" status in Rancher. I'm looking for a way to understand why this problem happened. But I would like to know if there is any way to add this previously created cluster to Rancher?

quiet-house-51252

12/12/2022, 8:02 PM

Hello, Can anyone help me how to get the user email id of the user in Rancher dashboard, when someone login rancher server with either GitHub, google, or another Identity service provider account

stocky-vr-48505

12/12/2022, 9:22 PM

Hi gang. Often, when I open up my laptop after it goes to sleep Rancher Desktop seems "dead". Using k9s the cluster cpu/memory is 0. Multiple deployments fail to restart properly and I usually need to completely wipe and recreate the cluster. Is there a guide on how to avoid this?

most-crowd-3167

12/12/2022, 9:50 PM

I'm using Rancher to deploy RKE2 on EC2 instances. Is there any way to attach multiple network interfaces to each node in the cluster (i.e. one interface per subnet in a VPC)

early-solstice-46134

12/13/2022, 7:40 AM

Any experts on the Rancher API around? I got some weird behaviour and before hunting bugs for hours, maybe knows something: I'm deploying the monitoring application into the cluster in which Rancher is installed via API without any problems. When I try to deploy the monitoring app to a cluster managed by Rancher using the exact same approach (with an adjusted API call, of course), this does not work. Rancher responds to the call with a

response code, but never actually starts rolling out the monitoring stack to the managed cluster.

steep-window-46329

12/13/2022, 10:39 AM

Hello everyone. We are experiencing stability issues with downstream clusters where memory usage is high on worker nodes. This seems to be expected because there is no resource-reservation (

kube-reserved

system-reserved

) set per default on kubernetes and rancher. So there are evictions happening and in most cases just the OOM-Killer of Systemd running wild. We've set a custom value for

eviction-hard

and

system-reserved

and this seems to improve the stability of the worker-nodes and in case of low memory, the correct pods are getting killed instead of random system processes. Is there a general recommendation from Rancher for how much memory and CPU should be reserved on downstream-clusters based on the size of the worker-node? It would help a lot to get some general guidelines. Thanks! Reserve Compute Resources for System Daemons https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/

boundless-country-59971

12/13/2022, 1:56 PM

#CheckIn Today: • Resolve multifactor authentication for gitlab • Input finished work through Gitlab including security review for VPN on AWS • Get with Ashley regarding bucket policies with IAM policies • Go through to do list tasks on Gitlab

crooked-cat-21365

12/13/2022, 4:03 PM

Hi folks, I am trying to recover from an etcd database corruption of a cluster setup by Rancher. There is just a single control plane node and 4 workers. I found https://rancher.com/docs/rke/latest/en/etcd-snapshots/, saying to read the rke documentation for restore. This document recommends to run

Copy code

# ~/rke.v1.3.15 etcd snapshot-restore --config ~/kube002.config.yaml --name c-blwbg-rl-mxhdw_2022-12-13T05:18:35Z 
INFO[0000] Running RKE version: v1.3.15                 
INFO[0000] Checking if state file is included in snapshot file for [c-blwbg-rl-mxhdw_2022-12-13T05:18:35Z] 
FATA[0000] Cluster must have at least one etcd plane host: please specify one or more etcd in cluster config

Problem is, where do I get a working cluster.yaml from? I had expected that all necessary information is stored in the snapshot (next to rke.state and the etcd database). The cluster.yaml I saved via the GUI appears to be incomplete. Every helpful comment is highly appreciated