I've successfully upgraded longhorn 1.4.0 to 1.5.3...
# longhorn-storage
o
I've successfully upgraded longhorn 1.4.0 to 1.5.3 (kubectl, not helm) on a few devices but ran into this issue on 2 different devices: longhorn-driver-deployer is stuck in initializing phase since it is waiting on longhorn manager to start.
Copy code
longhorn-system    longhorn-driver-deployer-5f5fc8458d-r4nvg             1/1     Running     4 (82d ago)        266d
longhorn-system    longhorn-driver-deployer-689698dfc4-9ddkw             0/1     Init:0/1    0                  20h

...
...
longhorn-system    longhorn-manager-bhtlr                                0/1     Running     0                  20h
Copy code
$ kubectl logs --tail 1 longhorn-manager-bhtlr -n longhorn-system

E0116 20:13:42.818174       1 reflector.go:148] github.com/longhorn/longhorn-manager/k8s/pkg/client/informers/externalversions/factory.go:117: Failed to watch *v1beta2.BackupTarget: failed to list *v1beta2.BackupTarget: conversion webhook for longhorn.io/v1beta1, Kind=BackupTarget failed: Post "<https://longhorn-conversion-webhook.longhorn-system.svc:9501/v1/webhook/conversion?timeout=30s>": no endpoints available for service "longhorn-conversion-webhook"
No endpoints are available for multiple services:
Copy code
$ kubectl get endpoints -n longhorn-system

NAME                                       ENDPOINTS                                             AGE
pvc-7669d7a5-00fe-497e-94c3-832e397a23d6   <none>                                                119d
csi-provisioner                            10.42.0.39:12345,10.42.1.22:12345,10.42.2.142:12345   266d
csi-resizer                                10.42.0.32:12345,10.42.1.29:12345,10.42.2.141:12345   266d
csi-snapshotter                            10.42.0.30:12345,10.42.1.21:12345,10.42.2.137:12345   266d
csi-attacher                               10.42.0.38:12345,10.42.1.8:12345,10.42.2.153:12345    266d
longhorn-engine-manager                    10.42.0.51,10.42.1.30,10.42.2.156                     266d
longhorn-replica-manager                   10.42.0.50,10.42.1.31,10.42.2.157                     266d
longhorn-frontend                          10.42.0.68:8000,10.42.1.165:8000                      421d
longhorn-recovery-backend                                                                        266d
longhorn-backend                                                                                 421d
longhorn-admission-webhook                                                                       266d
longhorn-conversion-webhook                                                                      266d
Copy code
$ kubectl get svc -n longhorn-system

NAME                                       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)     AGE
csi-attacher                               ClusterIP   10.43.222.2     <none>        12345/TCP   266d
csi-provisioner                            ClusterIP   10.43.104.166   <none>        12345/TCP   266d
csi-resizer                                ClusterIP   10.43.105.123   <none>        12345/TCP   266d
csi-snapshotter                            ClusterIP   10.43.151.99    <none>        12345/TCP   266d
pvc-7669d7a5-00fe-497e-94c3-832e397a23d6   ClusterIP   10.43.31.22     <none>        2049/TCP    119d
longhorn-backend                           ClusterIP   10.43.147.202   <none>        9500/TCP    421d
longhorn-frontend                          ClusterIP   10.43.215.46    <none>        80/TCP      421d
longhorn-conversion-webhook                ClusterIP   10.43.165.91    <none>        9501/TCP    266d
longhorn-admission-webhook                 ClusterIP   10.43.16.143    <none>        9502/TCP    266d
longhorn-recovery-backend                  ClusterIP   10.43.58.173    <none>        9503/TCP    266d
longhorn-engine-manager                    ClusterIP   None            <none>        <none>      266d
longhorn-replica-manager                   ClusterIP   None            <none>        <none>      266d
Could someone provide a fix/workaround for this issue?
129 Views