https://rancher.com/ logo
Join the conversationJoin Slack
Channels
academy
amazon
arm
azure
cabpr
chinese
ci-cd
danish
deutsch
developer
elemental
epinio
espanol
events
extensions
fleet
français
gcp
general
harvester
harvester-dev
hobbyfarm
hypper
japanese
k3d
k3os
k3s
k3s-contributor
kim
kubernetes
kubewarden
lima
logging
longhorn-dev
longhorn-storage
masterclass
mesos
mexico
nederlands
neuvector-security
office-hours
one-point-x
onlinemeetup
onlinetraining
opni
os
ozt
phillydotnet
portugues
rancher-desktop
rancher-extensions
rancher-setup
rancher-wrangler
random
rfed_ara
rio
rke
rke2
russian
s3gw
service-mesh
storage
submariner
supermicro-sixsq
swarm
terraform-controller
terraform-provider-rancher2
terraform-provider-rke
theranchcast
training-0110
training-0124
training-0131
training-0207
training-0214
training-1220
ukranian
v16-v21-migration
vsphere
windows
Powered by Linen
general
  • b

    boundless-country-59971

    12/13/2022, 1:56 PM
    #CheckIn Today: • Resolve multifactor authentication for gitlab • Input finished work through Gitlab including security review for VPN on AWS • Get with Ashley regarding bucket policies with IAM policies • Go through to do list tasks on Gitlab
    w
    • 2
    • 1
  • c

    crooked-cat-21365

    12/13/2022, 4:03 PM
    Hi folks, I am trying to recover from an etcd database corruption of a cluster setup by Rancher. There is just a single control plane node and 4 workers. I found https://rancher.com/docs/rke/latest/en/etcd-snapshots/, saying to read the rke documentation for restore. This document recommends to run
    # ~/rke.v1.3.15 etcd snapshot-restore --config ~/kube002.config.yaml --name c-blwbg-rl-mxhdw_2022-12-13T05:18:35Z 
    INFO[0000] Running RKE version: v1.3.15                 
    INFO[0000] Checking if state file is included in snapshot file for [c-blwbg-rl-mxhdw_2022-12-13T05:18:35Z] 
    FATA[0000] Cluster must have at least one etcd plane host: please specify one or more etcd in cluster config
    Problem is, where do I get a working cluster.yaml from? I had expected that all necessary information is stored in the snapshot (next to rke.state and the etcd database). The cluster.yaml I saved via the GUI appears to be incomplete. Every helpful comment is highly appreciated
  • a

    acoustic-refrigerator-10961

    12/13/2022, 4:46 PM
    I'm trying to configure Rancher Desktop to work behind a proxy. To troubleshoot I have it setup to use mitmproxy. I followed steps here: https://github.com/rancher-sandbox/rancher-desktop/issues/2259#issuecomment-1136833849 I can see that dockerd and curl are correctly using the proxy when I run rdctl shell but I don't see any attempt from the Rancher Desktop client running on the host to use the proxy.
    • 1
    • 2
  • e

    eager-airport-76783

    12/13/2022, 5:30 PM
    @ancient-bear-48931 Anyone have experience in getting updated container images to address CVEs found during monthly scans?
  • f

    fierce-elephant-30846

    12/13/2022, 6:17 PM
    Hi, everyone! I've deployed sucessfully a k8s cluster in a rancher single node instance, using a custom certificate, but now I'm stucked in a continuous message in the Rancher's panel, that says: Waiting for API to be available anyone could to help me?
  • p

    polite-piano-74233

    12/13/2022, 8:10 PM
    so when you remove the default nginx ingress from rancher cluster management, it deletes the entire ingress-nginx namespace.... welp there goes my other nginx controllers i guess 😞
    c
    • 2
    • 2
  • p

    polite-piano-74233

    12/13/2022, 10:26 PM
    so when i install ranchers logging in cluster tools i get tons of dns errors, looks like the logging chart doesnt support custom cluster domains? Its slamming .svc.cluster.local even with mine not being that domain (staging.cluster.etc.etc)
  • w

    white-gpu-45500

    12/13/2022, 11:19 PM
    Is anything special required to use RD on Windows 10 with WSL2? I installed RD, uninstalled DD, restarted wsl2, but am getting this error when running docker inside the wsl2 Ubuntu container
    $ docker
    Command 'docker' not found, but can be installed with:
    sudo apt install docker.io # version 20.10.12-0ubuntu4, or
    sudo apt install podman-docker # version 3.4.4+ds1-1ubuntu1
    I can't remember if I needed to do
    sudo apt install <http://docker.io|docker.io>
    when I installed Docker Desktop. Running
    docker
    in PowerShell seems to be working fine.
    s
    • 2
    • 1
  • l

    lemon-restaurant-45446

    12/14/2022, 2:27 AM
    Hi all, I have one docker node run rancher and rke cluster. Where is kubeconfig file of rke cluster save at? I want set a cron job to backup same setting that require kubeconfig file, I know WEB UI can get the kubeconfig, but it will change after reboot.
  • i

    incalculable-cartoon-97789

    12/14/2022, 5:33 AM
    Hello, I have a cluster with 3 masters and 3 workers. When I reboot a master server, it just disappears from the "kubectl get nodes" output. I can't rejoin the master. I can't even add new master to the cluster. Has anyone have this problem, how should I approach it? My RKE2 version is v1.24.8
    c
    • 2
    • 2
  • s

    some-machine-69773

    12/14/2022, 6:51 AM
    hello everyone
  • s

    some-machine-69773

    12/14/2022, 6:52 AM
    I got a problem with rancher, suddenly all service can not access, return the 503 error code from nginx
    e
    • 2
    • 2
  • b

    billowy-minister-95880

    12/14/2022, 8:54 AM
    I'm using Rancher Desktop and have some problems running docker on windows. It works great in WSL2 but when I try to run basically any command from PowerShell, I get bellow err. message: I'm running PowerShell as Admin. PS C:\Users\fdrzewieck001> docker config ls error during connect: In the default daemon configuration on Windows, the docker client must be run with elevated privileges to connect.: Get "http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.24/configs": open //./pipe/docker_engine: The system cannot find the file specified. PS C:\Users\fdrzewieck001> docker run hello-world docker: error during connect: In the default daemon configuration on Windows, the docker client must be run with elevated privileges to connect.: Post "http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.24/containers/create": open //./pipe/docker_engine: The system cannot find the file specified.
    s
    • 2
    • 1
  • s

    silly-jordan-81965

    12/14/2022, 9:27 AM
    We are seeing in the UI that Rancher arent always able to to a proper inverntory of the etcd snapshots. And thus we dont get status or size on the actual snapshot in the UI. However checking locally on the control plane nodes we do see that the snapshots has gone through as expected. What can be the cause of this? https://github.com/rancher/rancher/issues/39703
    • 1
    • 1
  • l

    little-jordan-39170

    12/14/2022, 12:24 PM
    👋 Hi everyone!
  • l

    little-jordan-39170

    12/14/2022, 12:26 PM
    I am new to rancher. I have launch a docker container for rancher.. Rancher UI is available but when I am trying to import EKS it shows "Failed to communicate with cluster: Unauthorized" and in the container log it shows "2022/12/14 12:01:40 [ERROR] error syncing 'c-wbp29': handler eks-operator-controller: Unauthorized, requeuing 2022/12/14 12:01:41 [ERROR] error syncing '_all_': handler user-controllers-controller: failed to start user controllers for cluster c-wbp29: secrets "cattle-global-data/" not found, requeuing 2022/12/14 12:03:41 [ERROR] error syncing 'c-wbp29': handler eks-operator-controller: Unauthorized, requeuing 2022/12/14 12:03:41 [ERROR] error syncing '_all_': handler user-controllers-controller: failed to start user controllers for cluster c-wbp29: secrets "cattle-global-data/" not found, requeuing 2022/12/14 12:04:40 [INFO] checking cluster [c-wbp29] upstream state for changes 2022/12/14 12:04:41 [INFO] cluster [c-wbp29] currently updating, skipping spec sync 2022/12/14 12:04:41 [ERROR] error syncing 'c-wbp29': handler eks-operator-controller: Unauthorized, requeuing 2022/12/14 12:04:41 [ERROR] error syncing 'c-wbp29': handler eks-operator-controller: Unauthorized, requeuing 2022/12/14 12:05:41 [ERROR] error syncing '_all_': handler user-controllers-controller: failed to start user controllers for cluster c-wbp29: secrets "cattle-global-data/" not found, requeuing 2022/12/14 12:06:41 [ERROR] error syncing 'c-wbp29': handler eks-operator-controller: Unauthorized, requeuing 2022/12/14 12:07:41 [ERROR] error syncing '_all_': handler user-controllers-controller: failed to start user controllers for cluster c-wbp29: secrets "cattle-global-data/" not found, requeuing"
    c
    a
    • 3
    • 8
  • l

    little-jordan-39170

    12/14/2022, 12:27 PM
    Need some help to resolve this issue...I am stuck for a long and not able to find any solution
  • c

    creamy-accountant-88363

    12/14/2022, 4:26 PM
    I'm wondering the best way to get the kubeconfig passed down to a driver. I was looking at some existing drivers, and saw the harvester node driver accepts the kubeconfig as a flag. Is that avoidable? Using the in-cluster kubeconfig doesn't seem to work, unfortunately.
  • w

    wooden-planet-93714

    12/14/2022, 5:07 PM
    is there a way to update the status.nodePlan.plan on an old rke node? it is spawning old rancher-agent image which fails after OS upgrade
    • 1
    • 1
  • l

    loud-daybreak-83328

    12/14/2022, 5:59 PM
    Hi. I'm trying to get my head around NFS permissions for the external nfs storage class. We've set up the nfs-subdir-external-provisioner and have an NFS server set up and all that good stuff. I can deploy a workload and request a PV to be created and that works. What I'm wondering about is permission handling. Say we have an NFS server that we want to have available to users on Linux systems direct via NFS where people can put files in their PVCs, yet still have it functional in K8s. It seems like when the provisioner creates a volume, it's owned by 'nfsnobody' on our server, with 777 permissions on it. Meaning anyone who can access our nfs server can theoretically access any PVC inside of it. What I'd like ideally is that the user that's submitting a workload 'own' the NFS directory where the PVC resides. Has anyone done anything like this or dealt with closing down NFS permissions for this?
  • c

    cool-nightfall-8293

    12/14/2022, 7:04 PM
    Hello everyone, i'm trying put my AWS Credentials to Rancher Server (this are deployed on Docker Container), but, i'm getting the error: Authentication test failed, please check your credentials. But, i'm already test these credentials on my computer and in EC2, and i receive success in test these credentials. can anybody help me? Really thanks!
  • r

    ripe-actor-83292

    12/14/2022, 9:15 PM
    message has been deleted
  • c

    creamy-accountant-88363

    12/14/2022, 11:37 PM
    If I install CAPI providers onto a cluster with Rancher installed, what is the interaction? I would like Rancher to not interfere with the CAPI provider resources - i saw rancher has its own internal CAPI controller for RKE2
    r
    • 2
    • 6
  • r

    rich-account-85657

    12/15/2022, 2:32 AM
    Hello~ Does anyone knows why 'ImageGCFailed' happen in a cluster using RKE? The kubelet on this cluster has errors with log as below
    $ kubectl describe node master2
    
    ...
    Wanrning ImageGCFailed     5m        kubelet      wanted to free 1000000000 bytes, but freed 0 bytes space with errors in  image deletion: [rpc error: code = Unknown desc = Error response from daemon: conflict:
      unable to remove repository reference "<image registry url>/rancher/hyperkube:v1.24.4-rancher1"
      (must force) - container 4b51a3451d24 is using its referenced image 517766856680,
      rpc error: code = Unknown desc = Error response from daemon: conflict: unable to
      remove repository reference "<image registry url>/rancher/mirrored-coreos-etcd:v3.5.4"
      (must force) - container 9ef807f9a241 is using its referenced image 77b8864f9930,
      rpc error: code = Unknown desc = Error response from daemon: conflict: unable to
      remove repository reference "<image registry url>/rancher/rke-tools:v0.1.87"
      (must force) - container 0886469c9f2f is using its referenced image caffe885434d]
    Please let me know if you all have resolved error like this.
  • h

    happy-librarian-50007

    12/15/2022, 8:44 AM
    Hello everyone 🙂, Is there a way/solution to receive rancher metrics in prometheus of ResourceQuotas from a Project? I would like to get Cpu/Memory Limits and Utilization by projects
  • h

    handsome-doctor-17506

    12/15/2022, 9:03 AM
    Hi, how do I revoke a leaked user API Key?
  • b

    billowy-apple-60989

    12/15/2022, 9:26 AM
    Heya brain-trust. I’m trying to find out workloads on some of our clusters are using API’s that are being removed, from the kubernetes docs https://kubernetes.io/blog/2020/09/03/warnings/#deprecation-warnings i found this handy Prometheus query:
    apiserver_requested_deprecated_apis{removed_release="1.22"} * on(group,version,resource,subresource)
    group_right() apiserver_request_total
    However when i try it i get the following error:
    Error executing query: found duplicate series for the match group {group="<http://apiextensions.k8s.io|apiextensions.k8s.io>", resource="customresourcedefinitions", version="v1beta1"} on the left hand-side of the operation: [{__name__="apiserver_requested_deprecated_apis", endpoint="https", group="<http://apiextensions.k8s.io|apiextensions.k8s.io>", instance="172.30.101.215:6443", job="apiserver", namespace="default", removed_release="1.22", resource="customresourcedefinitions", service="kubernetes", version="v1beta1"}, {__name__="apiserver_requested_deprecated_apis", component="k3s-server", endpoint="metrics", group="<http://apiextensions.k8s.io|apiextensions.k8s.io>", instance="172.30.101.215:10250", job="k3s-server", metrics_path="/metrics", namespace="cattle-monitoring-system", pod="pushprox-k3s-server-client-6stpf", removed_release="1.22", resource="customresourcedefinitions", service="pushprox-k3s-server-client", version="v1beta1"}];many-to-many matching not allowed: matching labels must be unique on one side
    Being a novice in PromQL i don’t quite get how to resolve this.. any pointers? 🙂
  • c

    calm-exabyte-238

    12/15/2022, 10:29 AM
    Hello folks! Is it possible to change the font-size in the logs explorer and the shell tabs in the UI?
  • a

    ancient-minister-33499

    12/15/2022, 10:51 AM
    Regarding https://rancher-users.slack.com/archives/C3ASABBD1/p1671015319219489?thread_ts=1663437408.124819&amp;cid=C3ASABBD1 What about the metrics url https://<rancher-url>/metrics? This returns upto version 2.6.0 only standard go client metrics:
    go_**
    , `process_*`* and
    promhttp_metric_handler_requests_total
    Will this be extended in the future?
  • p

    powerful-energy-34627

    12/15/2022, 12:21 PM
    Hello folks! Is it possible to delete the namespace that are "stuck" in terminating mode? There is a lot of namespaces which are not deleted. I tried "kubectl delete namespace ... --force" but it didn't change anything.
Powered by Linen
Title
p

powerful-energy-34627

12/15/2022, 12:21 PM
Hello folks! Is it possible to delete the namespace that are "stuck" in terminating mode? There is a lot of namespaces which are not deleted. I tried "kubectl delete namespace ... --force" but it didn't change anything.
View count: 35