orange-zoo-63756
01/17/2024, 2:24 PMlonghorn-system longhorn-driver-deployer-5f5fc8458d-r4nvg 1/1 Running 4 (82d ago) 266d
longhorn-system longhorn-driver-deployer-689698dfc4-9ddkw 0/1 Init:0/1 0 20h
...
...
longhorn-system longhorn-manager-bhtlr 0/1 Running 0 20h
$ kubectl logs --tail 1 longhorn-manager-bhtlr -n longhorn-system
E0116 20:13:42.818174 1 reflector.go:148] github.com/longhorn/longhorn-manager/k8s/pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1beta2.BackupTarget: failed to list *v1beta2.BackupTarget: conversion webhook for longhorn.io/v1beta1, Kind=BackupTarget failed: Post "<https://longhorn-conversion-webhook.longhorn-system.svc:9501/v1/webhook/conversion?timeout=30s>": no endpoints available for service "longhorn-conversion-webhook"
No endpoints are available for multiple services:
$ kubectl get endpoints -n longhorn-system
NAME ENDPOINTS AGE
pvc-7669d7a5-00fe-497e-94c3-832e397a23d6 <none> 119d
csi-provisioner 10.42.0.39:12345,10.42.1.22:12345,10.42.2.142:12345 266d
csi-resizer 10.42.0.32:12345,10.42.1.29:12345,10.42.2.141:12345 266d
csi-snapshotter 10.42.0.30:12345,10.42.1.21:12345,10.42.2.137:12345 266d
csi-attacher 10.42.0.38:12345,10.42.1.8:12345,10.42.2.153:12345 266d
longhorn-engine-manager 10.42.0.51,10.42.1.30,10.42.2.156 266d
longhorn-replica-manager 10.42.0.50,10.42.1.31,10.42.2.157 266d
longhorn-frontend 10.42.0.68:8000,10.42.1.165:8000 421d
longhorn-recovery-backend 266d
longhorn-backend 421d
longhorn-admission-webhook 266d
longhorn-conversion-webhook 266d
$ kubectl get svc -n longhorn-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
csi-attacher ClusterIP 10.43.222.2 <none> 12345/TCP 266d
csi-provisioner ClusterIP 10.43.104.166 <none> 12345/TCP 266d
csi-resizer ClusterIP 10.43.105.123 <none> 12345/TCP 266d
csi-snapshotter ClusterIP 10.43.151.99 <none> 12345/TCP 266d
pvc-7669d7a5-00fe-497e-94c3-832e397a23d6 ClusterIP 10.43.31.22 <none> 2049/TCP 119d
longhorn-backend ClusterIP 10.43.147.202 <none> 9500/TCP 421d
longhorn-frontend ClusterIP 10.43.215.46 <none> 80/TCP 421d
longhorn-conversion-webhook ClusterIP 10.43.165.91 <none> 9501/TCP 266d
longhorn-admission-webhook ClusterIP 10.43.16.143 <none> 9502/TCP 266d
longhorn-recovery-backend ClusterIP 10.43.58.173 <none> 9503/TCP 266d
longhorn-engine-manager ClusterIP None <none> <none> 266d
longhorn-replica-manager ClusterIP None <none> <none> 266d
Could someone provide a fix/workaround for this issue?