silly-solstice-24970
11/29/2022, 3:58 PMagreeable-actor-89488
11/29/2022, 4:19 PMagreeable-oil-87482
11/29/2022, 4:31 PMsilly-solstice-24970
11/29/2022, 4:46 PMagreeable-oil-87482
11/29/2022, 4:50 PMsilly-solstice-24970
11/29/2022, 4:52 PMagreeable-oil-87482
11/29/2022, 4:53 PMsilly-solstice-24970
11/29/2022, 4:53 PMagreeable-oil-87482
11/29/2022, 4:54 PMsilly-solstice-24970
11/29/2022, 4:55 PMagreeable-oil-87482
11/29/2022, 4:55 PMsilly-solstice-24970
11/29/2022, 4:55 PMagreeable-oil-87482
11/29/2022, 4:55 PMsilly-solstice-24970
11/29/2022, 4:56 PMagreeable-oil-87482
11/29/2022, 4:57 PMsilly-solstice-24970
11/29/2022, 4:58 PMagreeable-oil-87482
11/29/2022, 4:58 PMsilly-solstice-24970
11/29/2022, 4:59 PMagreeable-oil-87482
11/29/2022, 5:04 PMsilly-solstice-24970
11/29/2022, 5:05 PM└> kg pod -n cattle-system 14:02:39
NAME READY STATUS RESTARTS AGE
cattle-cluster-agent-776d9c5484-6z5tk 1/1 Running 153 (4d5h ago) 88d
cattle-cluster-agent-776d9c5484-whqln 1/1 Running 147 (4d4h ago) 88d
agreeable-oil-87482
11/29/2022, 5:07 PMsilly-solstice-24970
11/29/2022, 5:07 PMreplica 2
in the deploymentapiVersion: apps/v1
kind: Deployment
metadata:
annotations:
<http://deployment.kubernetes.io/revision|deployment.kubernetes.io/revision>: "7"
<http://kubectl.kubernetes.io/last-applied-configuration|kubectl.kubernetes.io/last-applied-configuration>: |
{"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{"<http://management.cattle.io/scale-available|management.cattle.io/scale-available>":"2"},"name":"cattle-cluster-agent","namespace":"cattle-system"},"spec":{"selector":{"matchLabels":{"app":"cattle-cluster-agent"}},"strategy":{"rollingUpdate":{"maxSurge":1,"maxUnavailable":0},"type":"RollingUpdate"},"template":{"metadata":{"labels":{"app":"cattle-cluster-agent"}},"spec":{"affinity":{"nodeAffinity":{"preferredDuringSchedulingIgnoredDuringExecution":[{"preference":{"matchExpressions":[{"key":"<http://node-role.kubernetes.io/controlplane|node-role.kubernetes.io/controlplane>","operator":"In","values":["true"]}]},"weight":100},{"preference":{"matchExpressions":[{"key":"<http://node-role.kubernetes.io/control-plane|node-role.kubernetes.io/control-plane>","operator":"In","values":["true"]}]},"weight":100},{"preference":{"matchExpressions":[{"key":"<http://node-role.kubernetes.io/master|node-role.kubernetes.io/master>","operator":"In","values":["true"]}]},"weight":100},{"preference":{"matchExpressions":[{"key":"<http://cattle.io/cluster-agent|cattle.io/cluster-agent>","operator":"In","values":["true"]}]},"weight":1}],"requiredDuringSchedulingIgnoredDuringExecution":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"<http://beta.kubernetes.io/os|beta.kubernetes.io/os>","operator":"NotIn","values":["windows"]}]}]}},"podAntiAffinity":{"preferredDuringSchedulingIgnoredDuringExecution":[{"podAffinityTerm":{"labelSelector":{"matchExpressions":[{"key":"app","operator":"In","values":["cattle-cluster-agent"]}]},"topologyKey":"<http://kubernetes.io/hostname|kubernetes.io/hostname>"},"weight":100}]}},"containers":[{"env":[{"name":"CATTLE_FEATURES","value":"embedded-cluster-api=false,fleet=false,monitoringv1=false,multi-cluster-management=false,multi-cluster-management-agent=true,provisioningv2=false,rke2=false"},{"name":"CATTLE_IS_RKE","value":"false"},{"name":"CATTLE_SERVER","value":"<https://REDACTED>"},{"name":"CATTLE_CA_CHECKSUM","value":""},{"name":"CATTLE_CLUSTER","value":"true"},{"name":"CATTLE_K8S_MANAGED","value":"true"},{"name":"CATTLE_CLUSTER_REGISTRY","value":""},{"name":"CATTLE_SERVER_VERSION","value":"v2.6.6"},{"name":"CATTLE_INSTALL_UUID","value":"8418bc7f-8261-4caf-bac1-8c5498e6e22a"},{"name":"CATTLE_INGRESS_IP_DOMAIN","value":"<http://sslip.io|sslip.io>"}],"image":"rancher/rancher-agent:v2.6.6","imagePullPolicy":"IfNotPresent","name":"cluster-register","volumeMounts":[{"mountPath":"/cattle-credentials","name":"cattle-credentials","readOnly":true}]}],"serviceAccountName":"cattle","tolerations":[{"effect":"NoSchedule","key":"<http://node-role.kubernetes.io/controlplane|node-role.kubernetes.io/controlplane>","value":"true"},{"effect":"NoSchedule","key":"<http://node-role.kubernetes.io/control-plane|node-role.kubernetes.io/control-plane>","operator":"Exists"},{"effect":"NoSchedule","key":"<http://node-role.kubernetes.io/master|node-role.kubernetes.io/master>","operator":"Exists"}],"volumes":[{"name":"cattle-credentials","secret":{"defaultMode":320,"secretName":"cattle-credentials-7781fce"}}]}}}}
<http://management.cattle.io/scale-available|management.cattle.io/scale-available>: "2"
creationTimestamp: "2021-12-01T14:18:44Z"
generation: 8
name: cattle-cluster-agent
namespace: cattle-system
resourceVersion: "221850422"
uid: 0096bb08-6124-4eea-92f4-ba59964dea7b
spec:
progressDeadlineSeconds: 600
replicas: 2
.....
└> kubectl describe deployments.apps -n cattle-system cattle-cluster-agent 14:08:11
Name: cattle-cluster-agent
Namespace: cattle-system
CreationTimestamp: Wed, 01 Dec 2021 11:18:44 -0300
Labels: <none>
Annotations: <http://deployment.kubernetes.io/revision|deployment.kubernetes.io/revision>: 7
<http://management.cattle.io/scale-available|management.cattle.io/scale-available>: 2
Selector: app=cattle-cluster-agent
Replicas: 2 desired | 2 updated | 2 total | 2 available | 0 unavailable
StrategyType: RollingUpdate
MinReadySeconds: 0
RollingUpdateStrategy: 0 max unavailable, 1 max surge
Pod Template:
Labels: app=cattle-cluster-agent
Service Account: cattle
Containers:
cluster-register:
Image: rancher/rancher-agent:v2.6.6
Port: <none>
Host Port: <none>
Environment:
CATTLE_FEATURES: embedded-cluster-api=false,fleet=false,monitoringv1=false,multi-cluster-management=false,multi-cluster-management-agent=true,provisioningv2=false,rke2=false
CATTLE_IS_RKE: false
CATTLE_SERVER: REDACTED
CATTLE_CA_CHECKSUM:
CATTLE_CLUSTER: true
CATTLE_K8S_MANAGED: true
CATTLE_CLUSTER_REGISTRY:
CATTLE_SERVER_VERSION: v2.6.6
CATTLE_INSTALL_UUID: 8418bc7f-8261-4caf-bac1-8c5498e6e22a
CATTLE_INGRESS_IP_DOMAIN: <http://sslip.io|sslip.io>
Mounts:
/cattle-credentials from cattle-credentials (ro)
Volumes:
cattle-credentials:
Type: Secret (a volume populated by a Secret)
SecretName: REDACTED
Optional: false
Conditions:
Type Status Reason
---- ------ ------
Progressing True NewReplicaSetAvailable
Available True MinimumReplicasAvailable
OldReplicaSets: <none>
NewReplicaSet: cattle-cluster-agent-776d9c5484 (2/2 replicas created)
Events: <none>
agreeable-oil-87482
11/29/2022, 5:43 PMsilly-solstice-24970
11/29/2022, 5:44 PMagreeable-oil-87482
11/29/2022, 5:45 PMsilly-solstice-24970
11/29/2022, 5:46 PMagreeable-oil-87482
11/29/2022, 5:48 PMsilly-solstice-24970
11/29/2022, 5:50 PMagreeable-oil-87482
11/29/2022, 5:50 PMWe have a 12 nodes (bare-metal) cluster using rke2. During the weekend, we lost 1 node and lost the connection with Rancher UI. I was able to find, that from the cluster end, there are 2 hooks into rancher, but I haven’t been able to find hooks on rancher. - Rancher UI runs in a 3 node (VMs) cluster.I don't quite follow this, you have a 12 node bare metal cluster and a 3 node vm cluster. The three node VM cluster runs Rancher and you're saying when one of the bare metal nodes went down you couldn't access Rancher?
silly-solstice-24970
11/29/2022, 5:51 PMagreeable-oil-87482
11/29/2022, 5:51 PMsilly-solstice-24970
11/29/2022, 5:52 PMagreeable-oil-87482
11/29/2022, 5:52 PMDuring the weekend, we lost 1 node and lost the connection with Rancher UI.
silly-solstice-24970
11/29/2022, 5:53 PMagreeable-oil-87482
11/29/2022, 5:54 PMexplore
button in the cluster listsilly-solstice-24970
11/29/2022, 5:54 PMagreeable-oil-87482
11/29/2022, 5:56 PMsilly-solstice-24970
11/29/2022, 5:56 PMagreeable-oil-87482
11/29/2022, 5:57 PMsilly-solstice-24970
11/29/2022, 5:57 PMTerminating
pods, it was rescheduledagreeable-oil-87482
11/29/2022, 5:58 PMsilly-solstice-24970
11/29/2022, 5:58 PMagreeable-oil-87482
11/29/2022, 5:59 PMsilly-solstice-24970
11/29/2022, 6:00 PMcontrolplane,etcd,worker
agreeable-oil-87482
11/29/2022, 6:00 PMsilly-solstice-24970
11/29/2022, 6:00 PMkubectl get nodes | grep "controlplane,etcd,worker" | wc -l
9
agreeable-oil-87482
11/29/2022, 6:04 PMsilly-solstice-24970
11/29/2022, 6:06 PMagreeable-oil-87482
11/29/2022, 6:08 PMsilly-solstice-24970
11/29/2022, 6:08 PMagreeable-oil-87482
11/29/2022, 6:09 PMsilly-solstice-24970
11/29/2022, 6:09 PMagreeable-oil-87482
11/29/2022, 6:09 PMsilly-solstice-24970
11/29/2022, 6:10 PM5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0/5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0-json.log:{"log":"E1125 12:17:24.476289 7819 pod_workers.go:951] \"Error syncing pod, skipping\" err=\"failed to \\\"StartContainer\\\" for \\\"cluster-register\\\" with CrashLoopBackOff: \\\"back-off 5m0s restarting failed container=cluster-register pod=cattle-cluster-agent-776d9c5484-whqln_cattle-system(f08d26d4-d212-4200-a200-7de428e83db7)\\\"\" pod=\"cattle-system/cattle-cluster-agent-776d9c5484-whqln\" podUID=f08d26d4-d212-4200-a200-7de428e83db7\n","stream":"stderr","time":"2022-11-25T12:17:24.476320217Z"}
5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0/5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0-json.log:{"log":"E1125 12:17:37.476090 7819 pod_workers.go:951] \"Error syncing pod, skipping\" err=\"failed to \\\"StartContainer\\\" for \\\"cluster-register\\\" with CrashLoopBackOff: \\\"back-off 5m0s restarting failed container=cluster-register pod=cattle-cluster-agent-776d9c5484-whqln_cattle-system(f08d26d4-d212-4200-a200-7de428e83db7)\\\"\" pod=\"cattle-system/cattle-cluster-agent-776d9c5484-whqln\" podUID=f08d26d4-d212-4200-a200-7de428e83db7\n","stream":"stderr","time":"2022-11-25T12:17:37.476131071Z"}
5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0/5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0-json.log:{"log":"E1125 12:17:52.476107 7819 pod_workers.go:951] \"Error syncing pod, skipping\" err=\"failed to \\\"StartContainer\\\" for \\\"cluster-register\\\" with CrashLoopBackOff: \\\"back-off 5m0s restarting failed container=cluster-register pod=cattle-cluster-agent-776d9c5484-whqln_cattle-system(f08d26d4-d212-4200-a200-7de428e83db7)\\\"\" pod=\"cattle-system/cattle-cluster-agent-776d9c5484-whqln\" podUID=f08d26d4-d212-4200-a200-7de428e83db7\n","stream":"stderr","time":"2022-11-25T12:17:52.476147078Z"}
5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0/5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0-json.log:{"log":"E1125 12:18:03.477807 7819 pod_workers.go:951] \"Error syncing pod, skipping\" err=\"failed to \\\"StartContainer\\\" for \\\"cluster-register\\\" with CrashLoopBackOff: \\\"back-off 5m0s restarting failed container=cluster-register pod=cattle-cluster-agent-776d9c5484-whqln_cattle-system(f08d26d4-d212-4200-a200-7de428e83db7)\\\"\" pod=\"cattle-system/cattle-cluster-agent-776d9c5484-whqln\" podUID=f08d26d4-d212-4200-a200-7de428e83db7\n","stream":"stderr","time":"2022-11-25T12:18:03.477859351Z"}
5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0/5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0-json.log:{"log":"E1125 12:18:16.488822 7819 pod_workers.go:951] \"Error syncing pod, skipping\" err=\"failed to \\\"StartContainer\\\" for \\\"cluster-register\\\" with CrashLoopBackOff: \\\"back-off 5m0s restarting failed container=cluster-register pod=cattle-cluster-agent-776d9c5484-whqln_cattle-system(f08d26d4-d212-4200-a200-7de428e83db7)\\\"\" pod=\"cattle-system/cattle-cluster-agent-776d9c5484-whqln\" podUID=f08d26d4-d212-4200-a200-7de428e83db7\n","stream":"stderr","time":"2022-11-25T12:18:16.488849233Z"}
5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0/5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0-json.log:{"log":"E1125 12:18:31.475653 7819 pod_workers.go:951] \"Error syncing pod, skipping\" err=\"failed to \\\"StartContainer\\\" for \\\"cluster-register\\\" with CrashLoopBackOff: \\\"back-off 5m0s restarting failed container=cluster-register pod=cattle-cluster-agent-776d9c5484-whqln_cattle-system(f08d26d4-d212-4200-a200-7de428e83db7)\\\"\" pod=\"cattle-system/cattle-cluster-agent-776d9c5484-whqln\" podUID=f08d26d4-d212-4200-a200-7de428e83db7\n","stream":"stderr","time":"2022-11-25T12:18:31.475695139Z"}
5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0/5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0-json.log:{"log":"E1125 12:18:42.476356 7819 pod_workers.go:951] \"Error syncing pod, skipping\" err=\"failed to \\\"StartContainer\\\" for \\\"cluster-register\\\" with CrashLoopBackOff: \\\"back-off 5m0s restarting failed container=cluster-register pod=cattle-cluster-agent-776d9c5484-whqln_cattle-system(f08d26d4-d212-4200-a200-7de428e83db7)\\\"\" pod=\"cattle-system/cattle-cluster-agent-776d9c5484-whqln\" podUID=f08d26d4-d212-4200-a200-7de428e83db7\n","stream":"stderr","time":"2022-11-25T12:18:42.476391026Z"}
5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0/5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0-json.log:{"log":"E1125 12:18:53.477920 7819 pod_workers.go:951] \"Error syncing pod, skipping\" err=\"failed to \\\"StartContainer\\\" for \\\"cluster-register\\\" with CrashLoopBackOff: \\\"back-off 5m0s restarting failed container=cluster-register pod=cattle-cluster-agent-776d9c5484-whqln_cattle-system(f08d26d4-d212-4200-a200-7de428e83db7)\\\"\" pod=\"cattle-system/cattle-cluster-agent-776d9c5484-whqln\" podUID=f08d26d4-d212-4200-a200-7de428e83db7\n","stream":"stderr","time":"2022-11-25T12:18:53.47796685Z"}
5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0/5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0-json.log:{"log":"E1125 12:19:06.475478 7819 pod_workers.go:951] \"Error syncing pod, skipping\" err=\"failed to \\\"StartContainer\\\" for \\\"cluster-register\\\" with CrashLoopBackOff: \\\"back-off 5m0s restarting failed container=cluster-register pod=cattle-cluster-agent-776d9c5484-whqln_cattle-system(f08d26d4-d212-4200-a200-7de428e83db7)\\\"\" pod=\"cattle-system/cattle-cluster-agent-776d9c5484-whqln\" podUID=f08d26d4-d212-4200-a200-7de428e83db7\n","stream":"stderr","time":"2022-11-25T12:19:06.475507785Z"}
5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0/5b9405eb75a062d8ab82b9ceb5c0cadc13904ddd9672224bb1e143feae867bc0-json.log:{"log":"E1125 12:19:19.476385 7819 pod_workers.go:951] \"Error syncing pod, skipping\" err=\"failed to \\\"StartContainer\\\" for \\\"cluster-register\\\" with CrashLoopBackOff: \\\"back-off 5m0s restarting failed container=cluster-register pod=cattle-cluster-agent-776d9c5484-whqln_cattle-system(f08d26d4-d212-4200-a200-7de428e83db7)\\\"\" pod=\"cattle-system/cattle-cluster-agent-776d9c5484-whqln\" podUID=f08d26d4-d212-4200-a200-7de428e83db7\n","stream":"stderr","time":"2022-11-25T12:19:19.476424614Z"}
8531818b4ba4c9f483e32a16ed38d6c6cfcec194fd0fcc6870f086323b7f47b3/8531818b4ba4c9f483e32a16ed38d6c6cfcec194fd0fcc6870f086323b7f47b3-json.log:{"log":"{\"caller\":\"main.go:49\",\"event\":\"startUpdate\",\"msg\":\"start of service update\",\"service\":\"cattle-system/cattle-cluster-agent\",\"ts\":\"2022-11-03T15:27:32.633976521Z\"}\n","stream":"stdout","time":"2022-11-03T15:27:32.63399913Z"}
8531818b4ba4c9f483e32a16ed38d6c6cfcec194fd0fcc6870f086323b7f47b3/8531818b4ba4c9f483e32a16ed38d6c6cfcec194fd0fcc6870f086323b7f47b3-json.log:{"log":"{\"caller\":\"service.go:33\",\"event\":\"clearAssignment\",\"msg\":\"not a LoadBalancer\",\"reason\":\"notLoadBalancer\",\"service\":\"cattle-system/cattle-cluster-agent\",\"ts\":\"2022-11-03T15:27:32.633985011Z\"}\n","stream":"stdout","time":"2022-11-03T15:27:32.63401395Z"}
8531818b4ba4c9f483e32a16ed38d6c6cfcec194fd0fcc6870f086323b7f47b3/8531818b4ba4c9f483e32a16ed38d6c6cfcec194fd0fcc6870f086323b7f47b3-json.log:{"log":"{\"caller\":\"main.go:75\",\"event\":\"noChange\",\"msg\":\"service converged, no change\",\"service\":\"cattle-system/cattle-cluster-agent\",\"ts\":\"2022-11-03T15:27:32.63400606Z\"}\n","stream":"stdout","time":"2022-11-03T15:27:32.63403026Z"}
8531818b4ba4c9f483e32a16ed38d6c6cfcec194fd0fcc6870f086323b7f47b3/8531818b4ba4c9f483e32a16ed38d6c6cfcec194fd0fcc6870f086323b7f47b3-json.log:{"log":"{\"caller\":\"main.go:76\",\"event\":\"endUpdate\",\"msg\":\"end of service update\",\"service\":\"cattle-system/cattle-cluster-agent\",\"ts\":\"2022-11-03T15:27:32.63401418Z\"}\n","stream":"stdout","time":"2022-11-03T15:27:32.63404437Z"}
8531818b4ba4c9f483e32a16ed38d6c6cfcec194fd0fcc6870f086323b7f47b3/8531818b4ba4c9f483e32a16ed38d6c6cfcec194fd0fcc6870f086323b7f47b3-json.log:{"log":"{\"caller\":\"main.go:49\",\"event\":\"startUpdate\",\"msg\":\"start of service update\",\"service\":\"cattle-system/cattle-cluster-agent\",\"ts\":\"2022-11-03T15:35:56.639997663Z\"}\n","stream":"stdout","time":"2022-11-03T15:35:56.640017403Z"}
8531818b4ba4c9f483e32a16ed38d6c6cfcec194fd0fcc6870f086323b7f47b3/8531818b4ba4c9f483e32a16ed38d6c6cfcec194fd0fcc6870f086323b7f47b3-json.log:{"log":"{\"caller\":\"service.go:33\",\"event\":\"clearAssignment\",\"msg\":\"not a LoadBalancer\",\"reason\":\"notLoadBalancer\",\"service\":\"cattle-system/cattle-cluster-agent\",\"ts\":\"2022-11-03T15:35:56.640005163Z\"}\n","stream":"stdout","time":"2022-11-03T15:35:56.640028863Z"}
8531818b4ba4c9f483e32a16ed38d6c6cfcec194fd0fcc6870f086323b7f47b3/8531818b4ba4c9f483e32a16ed38d6c6cfcec194fd0fcc6870f086323b7f47b3-json.log:{"log":"{\"caller\":\"main.go:75\",\"event\":\"noChange\",\"msg\":\"service converged, no change\",\"service\":\"cattle-system/cattle-cluster-agent\",\"ts\":\"2022-11-03T15:35:56.640027133Z\"}\n","stream":"stdout","time":"2022-11-03T15:35:56.640052362Z"}
creamy-pencil-82913
11/29/2022, 6:27 PMsilly-solstice-24970
11/29/2022, 6:28 PMcreamy-pencil-82913
11/29/2022, 6:29 PMThe recommended etcd cluster size is 3, 5 or 7
; I personally don’t see the case for having more than 3 servers; the rest should all be agents.silly-solstice-24970
11/29/2022, 6:30 PMcreamy-pencil-82913
11/29/2022, 6:35 PMsilly-solstice-24970
11/29/2022, 6:37 PMrke version v1.3.12
creamy-pencil-82913
11/29/2022, 6:37 PMsilly-solstice-24970
11/29/2022, 6:37 PMcreamy-pencil-82913
11/29/2022, 6:37 PMsilly-solstice-24970
11/29/2022, 6:38 PM{"log":"time=\"2022-11-25T12:21:18Z\" level=error msg=\"error syncing 'rancher-partner-charts': handler helm-clusterrepo-ensure: git -C /var/lib/rancher-data/local-catalogs/v2/ranc
her-partner-charts/8f17acdce9bffd6e05a58a3798840e408c4ea71783381ecd2e9af30baad65974 fetch origin 40d20d4f3eaafabad953bd8d150ebcf7c1ecc3cb error: exit status 128, detail: fatal: una
ble to access '<https://git.rancher.io/partner-charts/>': Could not resolve host: <http://git.rancher.io|git.rancher.io>\\n, requeuing\"\n","stream":"stdout","time":"2022-11-25T12:21:18.469552085Z"}
{"log":"time=\"2022-11-25T12:21:28Z\" level=error msg=\"error syncing 'rancher-partner-charts': handler helm-clusterrepo-ensure: git -C /var/lib/rancher-data/local-catalogs/v2/ranc
her-partner-charts/8f17acdce9bffd6e05a58a3798840e408c4ea71783381ecd2e9af30baad65974 fetch origin 40d20d4f3eaafabad953bd8d150ebcf7c1ecc3cb error: exit status 128, detail: fatal: una
ble to access '<https://git.rancher.io/partner-charts/>': Could not resolve host: <http://git.rancher.io|git.rancher.io>\\n, requeuing\"\n","stream":"stdout","time":"2022-11-25T12:21:28.499309911Z"}