limited-planet-94551
01/26/2023, 11:45 PMcool-beard-20658
01/27/2023, 4:20 AMcool-beard-20658
01/27/2023, 4:21 AMcool-beard-20658
01/27/2023, 4:23 AMbrainy-whale-94197
01/27/2023, 8:54 AMgray-helicopter-82716
01/27/2023, 1:32 PMprehistoric-state-31789
01/27/2023, 2:37 PMaverage-waitress-96027
01/27/2023, 4:01 PMFailed to pull image "rancher/gitjob:v0.1.32-security1": rpc error: code = Unknown desc = failed to pull and unpack image "<http://docker.io/rancher/gitjob:v0.1.32-security1|docker.io/rancher/gitjob:v0.1.32-security1>": failed to resolve reference "<http://docker.io/rancher/gitjob:v0.1.32-security1|docker.io/rancher/gitjob:v0.1.32-security1>": pulling from host <http://registry-1.docker.io|registry-1.docker.io> failed with status code [manifests v0.1.32-security1]: 403 Forbidden
Any ideas please?average-waitress-96027
01/27/2023, 4:09 PMhelm -n cattle-fleet-system install --create-namespace --wait \
fleet <https://github.com/rancher/fleet/releases/download/v0.5.0/fleet-0.5.0.tgz>
limited-planet-94551
01/27/2023, 4:17 PMabundant-hair-58573
01/27/2023, 5:25 PM2023/01/27 17:04:31 [ERROR] failed to start cluster controllers c-mmcz2: context canceled
2023/01/27 17:04:55 [ERROR] error syncing 'c-mmcz2': handler cluster-deploy: Get "https://<ip-address>:6443/apis/apps/v1/namespaces/cattle-system/daemonsets/cattle-node-agent": cluster agent disconnected, requeuing
2023/01/27 17:06:26 [INFO] Stopping cluster agent for c-mmcz2
2023/01/27 17:06:26 [ERROR] failed to start cluster controllers c-mmcz2: context canceled
2023/01/27 17:06:35 [INFO] Stopping cluster agent for c-cssrh
2023/01/27 17:06:35 [ERROR] failed to start cluster controllers c-cssrh: context canceled
2023/01/27 17:06:41 [ERROR] error syncing 'c-cssrh': handler cluster-deploy: Get "https://<ip-address>:6443/apis/apps/v1/namespaces/cattle-system/daemonsets/cattle-node-agent": cluster agent disconnected, requeuing
I believe c-cssrh is our RKE cluster that rancher was managing before the crash, and that we're hoping can just rejoin Rancher once we get it back up. The ip-address it's trying to connect to is a control plane in that cluster. I'm not sure what c-mmcz2 is, the ip-address it's trying to connect to is no longer around and I don't know what it was, possibly an old rancher manager and that's the name of the rancher cluster in k3s that was corrupted? Is there somewhere else to look for more detailed logs?proud-father-19786
01/27/2023, 7:27 PMsquare-rainbow-74440
01/28/2023, 5:39 PMcolossal-lizard-57639
01/29/2023, 12:39 PMcrooked-cat-21365
01/29/2023, 4:52 PMError while applying agent YAML, it will be retried automatically: exit status 1, <http://clusterrole.rbac.authorization.k8s.io/proxy-clusterrole-kubeapiserver|clusterrole.rbac.authorization.k8s.io/proxy-clusterrole-kubeapiserver> unchanged <http://clusterrolebinding.rbac.authorization.k8s.io/proxy-role-binding-kubernetes-master|clusterrolebinding.rbac.authorization.k8s.io/proxy-role-binding-kubernetes-master> unchanged namespace/cattle-system unchanged serviceaccount/cattle unchanged <http://clusterrolebinding.rbac.authorization.k8s.io/cattle-admin-binding|clusterrolebinding.rbac.authorization.k8s.io/cattle-admin-binding> unchanged secret/cattle-credentials-713674f unchanged <http://clusterrole.rbac.authorization.k8s.io/cattle-admin|clusterrole.rbac.authorization.k8s.io/cattle-admin> unchanged daemonset.apps/kube-api-auth unchanged service/cattle-cluster-agent unchanged Error from server (Invalid): error when applying patch: {"metadata":{"annotations":{"<http://kubectl.kubernetes.io/last-applied-configuration|kubectl.kubernetes.io/last-applied-configuration>":"{\"apiVersion\":\"apps/v1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{\"<http://management.cattle.io/scale-available\|management.cattle.io/scale-available\>":\"2\"},\"name\":\"cattle-cluster-agent\",\"namespace\":\"cattle-system\"},\"spec\":{\"selector\":{\"matchLabels\":{\"app\":\"cattle-cluster-agent\"}},\"strategy\":{\"rollingUpdate\":{\"maxSurge\":1,\"maxUnavailable\":0},\"type\":\"RollingUpdate\"},\"template\":{\"metadata\":{\"labels\":{\"app\":\"cattle-cluster-agent\"}},\"spec\":{\"affinity\":{\"nodeAffinity\":{\"preferredDuringSchedulingIgnoredDuringExecution\":[{\"preference\":{\"matchExpressions\":[{\"key\":\"<http://node-role.kubernetes.io/controlplane\|node-role.kubernetes.io/controlplane\>",\"operator\":\"In\",\"values\":[\"true\"]}]},\"weight\":100},{\"preference\":{\"matchExpressions\":[{\"key\":\"<http://node-role.kubernetes.io/control-plane\|node-role.kubernetes.io/control-plane\>",\"operator\":\"In\",\"values\":[\"true\"]}]},\"weight\":100},{\"preference\":{\"matchExpressions\":[{\"key\":\"<http://node-role.kubernetes.io/master\|node-role.kubernetes.io/master\>",\"operator\":\"In\",\"values\":[\"true\"]}]},\"weight\":100},{\"preference\":{\"matchExpressions\":[{\"key\":\"<http://cattle.io/cluster-agent\|cattle.io/cluster-agent\>",\"operator\":\"In\",\"values\":[\"true\"]}]},\"weight\":1}],\"requiredDuringSchedulingIgnoredDuringExecution\":{\"nodeSelectorTerms\":[{\"matchExpressions\":[{\"key\":\"<http://beta.kubernetes.io/os\|beta.kubernetes.io/os\>",\"operator\":\"NotIn\",\"values\":[\"windows\"]}]}]}},\"podAntiAffinity\":{\"preferredDuringSchedulingIgnoredDuringExecution\":[{\"podAffinityTerm\":{\"labelSelector\":{\"matchExpressions\":[{\"key\":\"app\",\"operator\":\"In\",\"values\":[\"cattle-cluster-agent\"]}]},\"topologyKey\":\"<http://kubernetes.io/hostname\|kubernetes.io/hostname\>"},\"weight\":100}]}},\"containers\":[{\"env\":[{\"name\":\"CATTLE_FEATURES\",\"value\":\"embedded-cluster-api=false,fleet=false,monitoringv1=false,multi-cluster-management=false,multi-cluster-management-agent=true,provisioningv2=false,rke2=false\"},{\"name\":\"CATTLE_IS_RKE\",\"value\":\"true\"},{\"name\":\"CATTLE_SERVER\",\"value\":\"<https://rancher01.ac.aixigo.de>\"},{\"name\":\"CATTLE_CA_CHECKSUM\",\"value\":\"\"},{\"name\":\"CATTLE_CLUSTER\",\"value\":\"true\"},{\"name\":\"CATTLE_K8S_MANAGED\",\"value\":\"true\"},{\"name\":\"CATTLE_CLUSTER_REGISTRY\",\"value\":\"\"},{\"name\":\"CATTLE_SERVER_VERSION\",\"value\":\"v2.6.9\"},{\"name\":\"CATTLE_INSTALL_UUID\",\"value\":\"b55cfe32-1daa-4848-b5f5-a7532f5eacd2\"},{\"name\":\"CATTLE_INGRESS_IP_DOMAIN\",\"value\":\"<http://sslip.io|sslip.io>\"},{\"name\":\"\"}],\"image\":\"rancher/rancher-agent:v2.6.9\",\"imagePullPolicy\":\"IfNotPresent\",\"name\":\"cluster-register\",\"volumeMounts\":[{\"mountPath\":\"/cattle-credentials\",\"name\":\"cattle-credentials\",\"readOnly\":true}]}],\"serviceAccountName\":\"cattle\",\"tolerations\":[{\"effect\":\"NoSchedule\",\"key\":\"<http://node-role.kubernetes.io/controlplane\|node-role.kubernetes.io/controlplane\>",\"value\":\"true\"},{\"effect\":\"NoSchedule\",\"key\":\"<http://node-role.kubernetes.io/control-plane\|node-role.kubernetes.io/control-plane\>",\"operator\":\"Exists\"},{\"effect\":\"NoSchedule\",\"key\":\"<http://node-role.kubernetes.io/master\|node-role.kubernetes.io/master\>",\"operator\":\"Exists\"}],\"volumes\":[{\"name\":\"cattle-credentials\",\"secret\":{\"defaultMode\":320,\"secretName\":\"cattle-credentials-713674f\"}}]}}}}\n"}},"spec":{"template":{"spec":{"$setElementOrder/containers":[{"name":"cluster-register"}],"containers":[{"$setElementOrder/env":[{"name":"CATTLE_FEATURES"},{"name":"CATTLE_IS_RKE"},{"name":"CATTLE_SERVER"},{"name":"CATTLE_CA_CHECKSUM"},{"name":"CATTLE_CLUSTER"},{"name":"CATTLE_K8S_MANAGED"},{"name":"CATTLE_CLUSTER_REGISTRY"},{"name":"CATTLE_SERVER_VERSION"},{"name":"CATTLE_INSTALL_UUID"},{"name":"CATTLE_INGRESS_IP_DOMAIN"},{"name":""}],"env":[{"name":"CATTLE_CA_CHECKSUM","value":""},{"name":"CATTLE_CLUSTER_REGISTRY","value":""},{"name":""}],"name":"cluster-register"}]}}}} to: Resource: "apps/v1, Resource=deployments", GroupVersionKind: "apps/v1, Kind=Deployment" Name: "cattle-cluster-agent", Namespace: "cattle-system" for: "./management-statefile_path_redacted": Deployment.apps "cattle-cluster-agent" is invalid: spec.template.spec.containers[0].env[10].name: Required value Error from server (Invalid): error when applying patch: {"metadata":{"annotations":{"<http://kubectl.kubernetes.io/last-applied-configuration|kubectl.kubernetes.io/last-applied-configuration>":"{\"apiVersion\":\"apps/v1\",\"kind\":\"DaemonSet\",\"metadata\":{\"annotations\":{},\"name\":\"cattle-node-agent\",\"namespace\":\"cattle-system\"},\"spec\":{\"selector\":{\"matchLabels\":{\"app\":\"cattle-agent\"}},\"template\":{\"metadata\":{\"labels\":{\"app\":\"cattle-agent\"}},\"spec\":{\"affinity\":{\"nodeAffinity\":{\"requiredDuringSchedulingIgnoredDuringExecution\":{\"nodeSelectorTerms\":[{\"matchExpressions\":[{\"key\":\"<http://beta.kubernetes.io/os\|beta.kubernetes.io/os\>",\"operator\":\"NotIn\",\"values\":[\"windows\"]}]}]}}},\"containers\":[{\"env\":[{\"name\":\"CATTLE_NODE_NAME\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"spec.nodeName\"}}},{\"name\":\"CATTLE_SERVER\",\"value\":\"<https://rancher01.ac.aixigo.de>\"},{\"name\":\"CATTLE_CA_CHECKSUM\",\"value\":\"\"},{\"name\":\"CATTLE_CLUSTER\",\"value\":\"false\"},{\"name\":\"CATTLE_K8S_MANAGED\",\"value\":\"true\"},{\"name\":\"CATTLE_AGENT_CONNECT\",\"value\":\"true\"},{\"name\":\"CATTLE_SERVER_VERSION\",\"value\":\"v2.6.9\"},{\"name\":\"CATTLE_INSTALL_UUID\",\"value\":\"b55cfe32-1daa-4848-b5f5-a7532f5eacd2\"},{\"name\":\"CATTLE_INGRESS_IP_DOMAIN\",\"value\":\"<http://sslip.io|sslip.io>\"},{\"name\":\"\"}],\"image\":\"rancher/rancher-agent:v2.6.9\",\"imagePullPolicy\":\"IfNotPresent\",\"name\":\"agent\",\"securityContext\":{\"privileged\":true},\"volumeMounts\":[{\"mountPath\":\"/cattle-credentials\",\"name\":\"cattle-credentials\",\"readOnly\":true},{\"mountPath\":\"/etc/kubernetes\",\"name\":\"k8s-ssl\"},{\"mountPath\":\"/var/run\",\"mountPropagation\":\"HostToContainer\",\"name\":\"var-run\"},{\"mountPath\":\"/run\",\"mountPropagation\":\"HostToContainer\",\"name\":\"run\"},{\"mountPath\":\"/etc/docker/certs.d\",\"name\":\"docker-certs\"}]}],\"hostNetwork\":true,\"serviceAccountName\":\"cattle\",\"tolerations\":[{\"operator\":\"Exists\"}],\"volumes\":[{\"hostPath\":{\"path\":\"/etc/kubernetes\",\"type\":\"DirectoryOrCreate\"},\"name\":\"k8s-ssl\"},{\"hostPath\":{\"path\":\"/var/run\",\"type\":\"DirectoryOrCreate\"},\"name\":\"var-run\"},{\"hostPath\":{\"path\":\"/run\",\"type\":\"DirectoryOrCreate\"},\"name\":\"run\"},{\"name\":\"cattle-credentials\",\"secret\":{\"defaultMode\":320,\"secretName\":\"cattle-credentials-713674f\"}},{\"hostPath\":{\"path\":\"/etc/docker/certs.d\",\"type\":\"DirectoryOrCreate\"},\"name\":\"docker-certs\"}]}},\"updateStrategy\":{\"rollingUpdate\":{\"maxUnavailable\":\"50%\"},\"type\":\"RollingUpdate\"}}}\n"}},"spec":{"template":{"spec":{"$setElementOrder/containers":[{"name":"agent"}],"containers":[{"$setElementOrder/env":[{"name":"CATTLE_NODE_NAME"},{"name":"CATTLE_SERVER"},{"name":"CATTLE_CA_CHECKSUM"},{"name":"CATTLE_CLUSTER"},{"name":"CATTLE_K8S_MANAGED"},{"name":"CATTLE_AGENT_CONNECT"},{"name":"CATTLE_SERVER_VERSION"},{"name":"CATTLE_INSTALL_UUID"},{"name":"CATTLE_INGRESS_IP_DOMAIN"},{"name":""}],"env":[{"name":"CATTLE_CA_CHECKSUM","value":""},{"name":""}],"name":"agent"}]}}}} to: Resource: "apps/v1, Resource=daemonsets", GroupVersionKind: "apps/v1, Kind=DaemonSet" Name: "cattle-node-agent", Namespace: "cattle-system" for: "./management-statefile_path_redacted": DaemonSet.apps "cattle-node-agent" is invalid: spec.template.spec.containers[0].env[9].name: Required value
What is it trying to tell me?quaint-house-29916
01/29/2023, 6:43 PMwooden-agency-67975
01/30/2023, 3:19 AMHi All,
I installed Rancher Desktop. Diagnostic says there are no problems. But when I issue a docker command, e.g. docker help, it just hangs forever. My Container Engine is set to 'containerd' and 'kubectl' command is working fine. Anyone knows what is wrong here? Thanks.
white-night-61856
01/30/2023, 6:16 AMcrooked-cat-21365
01/30/2023, 9:24 AMgray-scientist-46533
01/30/2023, 9:34 AMproud-tiger-59540
01/30/2023, 9:52 AMwonderful-shampoo-61843
01/30/2023, 10:16 AMauth_config_secret_name
, how can i create this type of secret in order to deploy the cluster with that private registry ?
https://registry.terraform.io/providers/rancher/rancher2/latest/docs/resources/cluster_v2#auth_config_secret_name
Thank you !elegant-oxygen-73525
01/30/2023, 2:53 PMk3d cluster create epinio -p '80:80@loadbalancer' -p '443:443@loadbalancer'
The deployed traefik service is not accessible via External IP. I am a bit confused, service port forward is working, external IP should be accessible from the local machine. (machine: Mac Intel). Can anyone please helpabundant-psychiatrist-96990
01/30/2023, 5:01 PMrhythmic-angle-26667
01/30/2023, 7:07 PMSome resources are remaining: <http://engineimages.longhorn.io|engineimages.longhorn.io> has 1 resource instances, <http://engines.longhorn.io|engines.longhorn.io> has 2 resource instances, <http://nodes.longhorn.io|nodes.longhorn.io> has 3 resource instances, <http://orphans.longhorn.io|orphans.longhorn.io> has 1 resource instances, <http://replicas.longhorn.io|replicas.longhorn.io> has 7 resource instances, <http://snapshots.longhorn.io|snapshots.longhorn.io> has 2 resource instances, <http://volumes.longhorn.io|volumes.longhorn.io> has 2 resource instances
, and the server log is filled with
Jan 30 19:07:23 k3s-sd-01 k3s[3111022]: W0130 19:07:23.612262 3111022 reflector.go:324] storage/cacher.go:/longhorn.io/volumes: failed to list <http://longhorn.io/v1beta1|longhorn.io/v1beta1>, Kind=Volume: conversion webhook for l>
Jan 30 19:07:23 k3s-sd-01 k3s[3111022]: E0130 19:07:23.612281 3111022 cacher.go:424] cacher (*unstructured.Unstructured): unexpected ListAndWatch error: failed to list <http://longhorn.io/v1beta1|longhorn.io/v1beta1>, Kind=Volume:>
rhythmic-angle-26667
01/30/2023, 7:08 PMkubectl delete
does precisely nothing, even with --force
. I’m a bit stuck; please help!square-rainbow-74440
01/30/2023, 7:18 PMearly-vegetable-32238
01/30/2023, 7:19 PMrhythmic-angle-26667
01/30/2023, 7:24 PMlonghorn-system
namespace; no pods, deployments, or statefulsets.early-vegetable-32238
01/30/2023, 7:27 PM