https://rancher.com/ logo
Docs
Join the conversationJoin Slack
Channels
academy
amazon
arm
azure
cabpr
chinese
ci-cd
danish
deutsch
developer
elemental
epinio
espanol
events
extensions
fleet
français
gcp
general
harvester
harvester-dev
hobbyfarm
hypper
japanese
k3d
k3os
k3s
k3s-contributor
kim
kubernetes
kubewarden
lima
logging
longhorn-dev
longhorn-storage
masterclass
mesos
mexico
nederlands
neuvector-security
office-hours
one-point-x
onlinemeetup
onlinetraining
opni
os
ozt
phillydotnet
portugues
rancher-desktop
rancher-extensions
rancher-setup
rancher-wrangler
random
rfed_ara
rio
rke
rke2
russian
s3gw
service-mesh
storage
submariner
supermicro-sixsq
swarm
terraform-controller
terraform-provider-rancher2
terraform-provider-rke
theranchcast
training-0110
training-0124
training-0131
training-0207
training-0214
training-1220
ukranian
v16-v21-migration
vsphere
windows
Powered by Linen
general
  • c

    crooked-cat-21365

    12/15/2022, 12:44 PM
    Trying to recover my broken etcd of a cluster setup via Rancher I get an error message in the GUI "Cluster must have at least one etcd plane host: failed to connect to the following etcd host(s) ...". How is this supposed to work? Rancher should have all information necessary to proceed. ???
  • f

    fancy-cricket-731

    12/15/2022, 3:00 PM
    hey, we are currently using the old app v1 to deploy our workloads in our clusters (using the rancher cli). We would like to move to app v2 since the other endpoints are lagacy now. Is there a way to: • migrate app v1 to app v2 • deploy the app v2 using a CLI tool like it was possible with rancher-cli (rancher app command)
    m
    • 2
    • 1
  • l

    loud-salesmen-91388

    12/15/2022, 4:15 PM
    Hi, i'm trying to install Rancher Server on my AKS cluster, but every time I try to access, my browser return with 404 or 503 error. I tried installing the aks-helloworld-demo and these were fine, i could access, but not rancher. Can someone please help me? Maybe sharing a ingress file as example (i'm pretty sure this is my root problem), i'm struggling for almost 3 weeks 😕
    c
    • 2
    • 3
  • c

    creamy-piano-98139

    12/15/2022, 5:31 PM
    Hi, I'm trying to add cluster CCE (Huawei Cloud Cluster) but it's timeout to download rancher driver. Can someone please help me ?
  • s

    salmon-shoe-87542

    12/15/2022, 8:45 PM
    I've installed Rancher 2.7.0 on top of a fresh k3s installation which is running on CentOS 7. I'm running into a couple of issues which may be related to my proxy settings. 1. I see errors like these in the logs of the Rancher pods: 2022/12/15 16:40:33 [ERROR] error syncing 'rancher-rke2-charts': handler helm-clusterrepo-ensure: git -C /var/lib/rancher-data/local-catalogs/v2/rancher-rke2-charts/675f1b63a0a83905972dcab2794479ed599a6f41b86cd6193d69472d0fa889c9 fetch origin 88c6633998c1b6e528f15a74428930c5df880908 error: exit status 128, detail: fatal: unable to access 'https://git.rancher.io/rke2-charts/': Received HTTP code 407 from proxy after CONNECT a. if I exec into the Rancher container and run "git config --global http.proxyAuthMethod 'basic'" these errors go away. Not sure if this should be considered a bug... 2. I'm also seeing errors downloading icons: time="2022-12-15T20:05:37Z" level=error msg="Unknown error: Get \"https://charts.rancher.io/assets/logos/logging.svg\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" a. not sure how to fix this one yet. I can exec into the container and "curl --insecure https://charts.rancher.io/assets/logos/logging.svg" just fine Any help is appreciated 🙂
  • l

    little-actor-95014

    12/15/2022, 9:54 PM
    I've got a few 2.7.0 clusters with imported RKE2 Downstream Clusters. I've noticed when navigating to the "Edit/Upgrade" screen for an installed helm app, it's blank. Anyone else running into this? I've got 3 instances I've been able to repo on, but want to make sure it isn't just me before starting to dig deep and opening issues
  • f

    few-spoon-45426

    12/15/2022, 10:33 PM
    do I need to do anything on Ubuntu's cloud-images to use them as templates for worker clusters on vSphere?
    m
    • 2
    • 7
  • b

    best-address-42882

    12/15/2022, 10:48 PM
    I've created a new Rancher 2.7.0 cluster with RKE2 on our air-gap environment. Now I need to install istio on it. I assume default https://git.rancher.io/partner-charts contain all necessary charts for it, but repo is not accessible directly. How I could download all necessary charts and images to have it on my personal repository? Or maybe some one had same challenges with air gaped env, will be appreciate for any help.
  • g

    gentle-spring-91845

    12/16/2022, 10:14 AM
    Hello everybody, i updated with rke kubernetes to version 1.24. when i run rke up to update i get this error`:`
    INFO[0070] Finding container [service-sidekick] on host [10.155.10.113], try #1
    INFO[0070] [sidekick] Sidekick container already created on host [10.155.10.113]
    INFO[0070] [healthcheck] Start Healthcheck on service [kubelet] on host [10.155.10.113]
    ERRO[0121] Failed to upgrade worker components on NotReady hosts, error: [Failed to verify healthcheck: Failed to check <http://localhost:10248/healthz> for service [kubelet] on host [10.155.10.113]: Get "<http://localhost:10248/healthz>": Unable to access the service on localhost:10248. The service might be still starting up. Error: ssh: rejected: connect failed (Connection refused), log: I1216 09:55:52.876292 1131786 docker_service.go:359] "Docker cri received runtime config" runtimeConfig="&RuntimeConfig{NetworkConfig:&NetworkConfig{PodCidr:,},}"]
    INFO[0121] [controlplane] Now checking status of node 10.155.10.113, try #1
    INFO[0126] [controlplane] Now checking status of node 10.155.10.113, try #2
    INFO[0131] [controlplane] Now checking status of node 10.155.10.113, try #3
    INFO[0136] [controlplane] Now checking status of node 10.155.10.113, try #4
    INFO[0141] [controlplane] Now checking status of node 10.155.10.113, try #5
    ERRO[0146] Host 10.155.10.113 failed to report Ready status with error: host 10.155.10.113 not ready
    INFO[0146] [controlplane] Processing controlplane hosts for upgrade 1 at a time
    INFO[0146] Processing controlplane host 10.155.10.113
    INFO[0146] [controlplane] Now checking status of node 10.155.10.113, try #1
    INFO[0151] [controlplane] Now checking status of node 10.155.10.113, try #2
    INFO[0156] [controlplane] Now checking status of node 10.155.10.113, try #3
    INFO[0161] [controlplane] Now checking status of node 10.155.10.113, try #4
    INFO[0166] [controlplane] Now checking status of node 10.155.10.113, try #5
    ERRO[0171] Failed to upgrade hosts: 10.155.10.113 with error [host 10.155.10.113 not ready]
    FATA[0171] [controlPlane] Failed to upgrade Control Plane: [[host 10.155.10.113 not ready]]
    On master1 I checked the kubelet docker container log
    [root@vit81-ldevsa00a ~]# docker logs kubelet --follow  --tail 10
          --tls-private-key-file string                              File containing x509 private key matching --tls-cert-file. (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See <https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/> for more information.)
          --topology-manager-policy string                           Topology Manager policy to use. Possible values: 'none', 'best-effort', 'restricted', 'single-numa-node'. (default "none") (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See <https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/> for more information.)
          --topology-manager-scope string                            Scope to which topology hints applied. Topology Manager collects hints from Hint Providers and applies them to defined scope to ensure the pod admission. Possible values: 'container', 'pod'. (default "container") (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See <https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/> for more information.)
      -v, --v Level                                                  number for the log level verbosity (default 0)
          --version version[=true]                                   Print version information and quit
          --vmodule pattern=N,...                                    comma-separated list of pattern=N settings for file-filtered logging (only works for text log format)
          --volume-plugin-dir string                                 The full path of the directory in which to search for additional third party volume plugins (default "/usr/libexec/kubernetes/kubelet-plugins/volume/exec/") (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See <https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/> for more information.)
          --volume-stats-agg-period duration                         Specifies interval for kubelet to calculate and cache the volume disk usage for all pods and volumes.  To disable volume calculations, set to a negative number. (default 1m0s) (DEPRECATED: This parameter should be set via the config file specified by the Kubelet's --config flag. See <https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/> for more information.)
    
    Error: failed to run Kubelet: unable to determine runtime API version: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial unix: missing address"
    [root@vit81-ldevsa00a ~]#
    I'm using rke version 1 1.3.10
    [root@vit81-ldevsa00a ~]# rke --version
    rke version v1.3.10
    in the cluster.yaml file I added this line:
    authorization:
      mode: rbac
      options: {}
    ignore_docker_version: null
    #enable_cri_dockerd: null
    enable_cri_dockerd: false
    #kubernetes_version: "v1.23.14-rancher1-1"
    kubernetes_version: "v1.24.8-rancher1-1" <<<----- THIS LINE ADDED
    private_registries: []
    ingress:
      provider: ""
      options: {}
  • g

    gentle-spring-91845

    12/16/2022, 10:52 AM
    Resolved. Using v1.4.2-rc1
  • b

    billions-garden-37463

    12/16/2022, 11:33 AM
    Hi everyone, so I’m trying to create a kubernetes cluster that can allow be add servers from different provider(aws, azure, DO) as worker nodes, has anyone tried creating a kubernetes cluster using a custom provider and then tried to add an AWS EC2 node as a worker node using the “sudo docker …” command? I tried it but it’s not working, Digital ocean was added successfully and works but not AWS EC2
    a
    • 2
    • 10
  • w

    wide-dog-6163

    12/16/2022, 1:08 PM
    Hi everyone, I am trying to provision an infrastructure-provided RKE2 Cluster on the OpenStack cloud using Rancher. I have cloud credentials ready, when I create a cluster from Rancher's GUI with OpenStack cloud, in machine pool form a lot of fields are given, but I can't determine which fields are required and which are optional. Also, I can't find any proper documentation to create clusters on the OpenStack cloud, I tried many combinations but nothing worked. Any help is appreciated.
  • a

    acceptable-dream-1782

    12/16/2022, 2:23 PM
    Hi Everyone, I'm having some issues with my downstreams clusters on RKE2. When I deploy any downstream cluster from Rancher (v2.6.8) GUI, the deploy stucks and the provisioning log tab shows this: [ERROR] Failed to set up SSH tunneling for host [10.1.76.18]: Can't retrieve Docker Info: error during connect. I accessed the cluster (In this case just for testing pupouse is a one node cluster doing control plane, worker and etcd) via ssh and do a "docker ps" and a "docker logs". This logs show: time="2022-12-16T13:55:09Z" level=error msg="Issuer of last certificate found in chain (CN=AAA Certificate Services,O=Comodo CA Limited,L=Salford,ST=Greater Manchester,C=GB) does not match with CA certificate Issuer (CN=dynamiclistener-ca@1662989077,O=dynamiclistener-org). Please check if the configured server certificate contains all needed intermediate certificates and make sure they are in the correct order (server certificate first, intermediates after)" time="2022-12-16T13:55:09Z" level=fatal msg="Server certificate is not valid, please check if the host has the correct time configured and if the server certificate has a notAfter date and time in the future. Certificate information is displayed above. error: Get \"https://xxx.xxx.xxx.xxx\": x509: certificate has expired or is not yet valid: current time 2022-12-16T13:55:09Z is after 2022-11-26T23:59:59Z" I am running Rancher in HA cluster with HAProxy in front of it. The downstream cluster creates VMs in VMware architecture and they all stuck at this point. I know that is a certificate issue but I don't know where I can find o replace that certificate. I will appreciate any kind of help. I'm trying to solve this issue since long time.
  • a

    acceptable-printer-7134

    12/16/2022, 2:45 PM
    can rancher cause
    eks node groups
    deletion or creation while they are just imported clusters? we are seeing some weird messages in eks node group section. FYI - we are just importing EKS clusters not creating by rancher. We see node groups creation in EKS that shows rancher trying to create.
    • 1
    • 8
  • q

    quaint-book-15301

    12/16/2022, 2:58 PM
    Good morning - I had some type of issue overnight that caused almost all of my volumes in Longhorn to become 'missing'. The pvc's show up in Rancher, but the pods wont deploy as the volume is missing when I look for them in Longhorn. I can see the PVC in /var/lib/longhorn/replica but I have no idea how the volume detached from the pvc. I have exported the volume from a single replica using the doc online but is there a way to recreate the volume and point it to the already existing PVC in /var/lib/longhorn/replica? Or, do I need to recreate the longhorn pv/pvc and copy the data in myself? Thanks!
  • b

    best-address-42882

    12/16/2022, 6:55 PM
    Hi, I've run Rancher 2.7.0 with CATTLE_SYSTEM_CATALOG=bundled option, and it works but when I push any buttons on App menu it loads and after couple min refreshes and returns back to menu without any result. In logs a lot of errors like that: [ERROR] Unknown error: Get "

    https://raw.githubusercontent.com/haproxytech/helm-charts/main/kubernetes-ingress/chart-icon.png▾

    ": context deadline exceeded (Client.Timeout exceeded while awaiting headers) Any ideas about it?
    ➕ 1
  • b

    best-address-42882

    12/16/2022, 6:56 PM
  • r

    rough-cricket-22538

    12/16/2022, 7:50 PM
    OK - getting frustrated - who knows how to delete the old, legacy rancher monitoring/logging from the cluster. I’ve tried deleting them from rancher apps, but they keep showing up
  • a

    ambitious-plastic-3551

    12/17/2022, 12:40 PM
    what taints should I use to disable running workloads on a control plane? I can't find the docs
  • a

    ambitious-plastic-3551

    12/17/2022, 12:42 PM
    node-taint: - "CriticalAddonsOnly=true:NoExecute" is this one the only one?
  • b

    bored-farmer-36655

    12/17/2022, 1:24 PM
    @ambitious-plastic-3551 from my code snippets I see
    kubectl taint nodes node-1 <http://node-role.kubernetes.io/controlplane=true:NoSchedule|node-role.kubernetes.io/controlplane=true:NoSchedule>
    would this help: https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
  • a

    ambitious-plastic-3551

    12/17/2022, 1:30 PM
    Thanks @bored-farmer-36655 will check it out
  • a

    ambitious-plastic-3551

    12/17/2022, 4:20 PM
    is it possible to expose hostPort with TCP and UDP at the same time
  • b

    bored-farmer-36655

    12/17/2022, 5:08 PM
    @ambitious-plastic-3551 Like this: https://jamesdefabia.github.io/docs/user-guide/kubectl/kubectl_expose/ or port forward?
  • a

    ambitious-plastic-3551

    12/17/2022, 6:08 PM
    well yaml version 😄 or even better helm chart...
  • a

    ambitious-plastic-3551

    12/17/2022, 6:09 PM
    I have to open github issue
  • c

    cuddly-vegetable-29975

    12/18/2022, 7:21 PM
    Hello. I just rolled my first Harvester installation and created my first VM on this. I only have one interface, so I added a bridged interface on another VLAN but still on the mgmt interface. The VM booted and successfully got a ip from my dhcp. I can connect to the VM from outside, but cannot connect to anything from inside the VM. Is it possible the have a bridge on the same physical interface as the mgmt network?
    w
    • 2
    • 2
  • b

    billions-bear-86898

    12/19/2022, 3:35 AM
    Hi I am new to this app. After installing this no command(docker,kubectl etc) is working in either in cmd or wsl
    c
    • 2
    • 1
  • l

    lively-night-78214

    12/19/2022, 5:14 AM
    is there a way to edit an ingress in a file locally and apply it to a remote? i promise i googled and tried finding a solution but can only see kubectl edit and never know if the formatting is right by manually editing on the shell
    h
    • 2
    • 3
  • s

    some-monkey-58167

    12/19/2022, 9:18 AM
    Hi, I'm trying to add new node to my Rancher k8s cluster. I have 1 rancher server(v2.6.9) running with docker container, 1 master node, 2 worker nodes, I want to add a new worker node to the cluster, but when I run registration command on new server, it show me below: # curl --insecure -fL https://192.168.0.57:8443/system-agent-install.sh | sudo sh -s - --server https://192.168.0.57:8443 --label 'cattle.io/os=linux' --token zxmdxmxdlwbgllpmf5zcwmr5rjg7bcb5nt7lv7k66rw6qz94k59tpd --ca-checksum 3a76abfb81778a09e15616d979bc298475dafab642b96daf23ff54b8c0a24b04 --worker % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 29798 0 29798 0 0 2645k 0 --:--:-- --:--:-- --:--:-- 2645k [INFO] Label: cattle.io/os=linux [INFO] Role requested: worker [INFO] Using default agent configuration directory /etc/rancher/agent [INFO] Using default agent var directory /var/lib/rancher/agent [INFO] Determined CA is necessary to connect to Rancher [INFO] Successfully downloaded CA certificate [INFO] Value from https://192.168.0.57:8443/cacerts is an x509 certificate [INFO] Successfully tested Rancher connection [INFO] Downloading rancher-system-agent binary from https://192.168.0.57:8443/assets/rancher-system-agent-amd64 [INFO] Successfully downloaded the rancher-system-agent binary. [INFO] Downloading rancher-system-agent-uninstall.sh script from https://192.168.0.57:8443/assets/system-agent-uninstall.sh [INFO] Successfully downloaded the rancher-system-agent-uninstall.sh script. [INFO] Generating Cattle ID curl: (28) Operation timed out after 60000 milliseconds with 0 bytes received [ERROR] 000 received while downloading Rancher connection information. Sleeping for 5 seconds and trying again curl: (28) Operation timed out after 60001 milliseconds with 0 bytes received [ERROR] 000 received while downloading Rancher connection information. Sleeping for 5 seconds and trying again curl: (28) Operation timed out after 60001 milliseconds with 0 bytes received [ERROR] 000 received while downloading Rancher connection information. Sleeping for 5 seconds and trying again curl: (28) Operation timed out after 60001 milliseconds with 0 bytes received [ERROR] 000 received while downloading Rancher connection information. Sleeping for 5 seconds and trying again
Powered by Linen
Title
s

some-monkey-58167

12/19/2022, 9:18 AM
Hi, I'm trying to add new node to my Rancher k8s cluster. I have 1 rancher server(v2.6.9) running with docker container, 1 master node, 2 worker nodes, I want to add a new worker node to the cluster, but when I run registration command on new server, it show me below: # curl --insecure -fL https://192.168.0.57:8443/system-agent-install.sh | sudo sh -s - --server https://192.168.0.57:8443 --label 'cattle.io/os=linux' --token zxmdxmxdlwbgllpmf5zcwmr5rjg7bcb5nt7lv7k66rw6qz94k59tpd --ca-checksum 3a76abfb81778a09e15616d979bc298475dafab642b96daf23ff54b8c0a24b04 --worker % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 29798 0 29798 0 0 2645k 0 --:--:-- --:--:-- --:--:-- 2645k [INFO] Label: cattle.io/os=linux [INFO] Role requested: worker [INFO] Using default agent configuration directory /etc/rancher/agent [INFO] Using default agent var directory /var/lib/rancher/agent [INFO] Determined CA is necessary to connect to Rancher [INFO] Successfully downloaded CA certificate [INFO] Value from https://192.168.0.57:8443/cacerts is an x509 certificate [INFO] Successfully tested Rancher connection [INFO] Downloading rancher-system-agent binary from https://192.168.0.57:8443/assets/rancher-system-agent-amd64 [INFO] Successfully downloaded the rancher-system-agent binary. [INFO] Downloading rancher-system-agent-uninstall.sh script from https://192.168.0.57:8443/assets/system-agent-uninstall.sh [INFO] Successfully downloaded the rancher-system-agent-uninstall.sh script. [INFO] Generating Cattle ID curl: (28) Operation timed out after 60000 milliseconds with 0 bytes received [ERROR] 000 received while downloading Rancher connection information. Sleeping for 5 seconds and trying again curl: (28) Operation timed out after 60001 milliseconds with 0 bytes received [ERROR] 000 received while downloading Rancher connection information. Sleeping for 5 seconds and trying again curl: (28) Operation timed out after 60001 milliseconds with 0 bytes received [ERROR] 000 received while downloading Rancher connection information. Sleeping for 5 seconds and trying again curl: (28) Operation timed out after 60001 milliseconds with 0 bytes received [ERROR] 000 received while downloading Rancher connection information. Sleeping for 5 seconds and trying again
View count: 29