https://rancher.com/ logo
Title
m

many-area-51777

08/31/2022, 12:19 PM
Hey, just upgrade to 2.6.8 and rancher is not coming up anymore. First I had to extend the liveness settings as mentioned in https://github.com/rancher/rancher/issues/38177 but even though the logs show that something is happening, at the end the pod dies and tries to restart.
Started seeing these after 2.6.8 upgrade:
nauthorized
W0903 22:11:23.709526 32 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.24.0-rancher1/tools/cache/reflector.go:168: failed to list *summary.SummarizedObject: Unauthorized
E0903 22:11:23.709553 32 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.24.0-rancher1/tools/cache/reflector.go:168: Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Unauthorized
W0903 22:11:23.726334 32 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.24.0-rancher1/tools/cache/reflector.go:168: failed to list *summary.SummarizedObject: Unauthorized
E0903 22:11:23.726360 32 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.24.0-rancher1/tools/cache/reflector.go:168: Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Unauthorized
2022/09/03 22:11:24 [ERROR] failed to call leader func: failed to add management data: Unauthorized
2022/09/03 22:11:24 [ERROR] failed to call leader func: Unauthorized
W0903 22:11:24.358226 32 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.24.0-rancher1/tools/cache/reflector.go:168: failed to list *summary.SummarizedObject: Unauthorized
E0903 22:11:24.358258 32 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.24.0-rancher1/tools/cache/reflector.go:168: Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Unauthorized
I0903 22:11:24.474577 32 leaderelection.go:283] failed to renew lease kube-system/cattle-controllers: timed out waiting for the condition
E0903 22:11:24.474647 32 leaderelection.go:306] Failed to release lock: resource name may not be empty
2022/09/03 22:11:24 [FATAL] leaderelection lost for cattle-controllers
Any known solution for that? I can't start rancher pod anymore.
Started seeing these after 2.6.8 upgrade:
nauthorized
W0903 22:11:23.709526 32 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.24.0-rancher1/tools/cache/reflector.go:168: failed to list *summary.SummarizedObject: Unauthorized
E0903 22:11:23.709553 32 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.24.0-rancher1/tools/cache/reflector.go:168: Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Unauthorized
W0903 22:11:23.726334 32 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.24.0-rancher1/tools/cache/reflector.go:168: failed to list *summary.SummarizedObject: Unauthorized
E0903 22:11:23.726360 32 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.24.0-rancher1/tools/cache/reflector.go:168: Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Unauthorized
2022/09/03 22:11:24 [ERROR] failed to call leader func: failed to add management data: Unauthorized
2022/09/03 22:11:24 [ERROR] failed to call leader func: Unauthorized
W0903 22:11:24.358226 32 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.24.0-rancher1/tools/cache/reflector.go:168: failed to list *summary.SummarizedObject: Unauthorized
E0903 22:11:24.358258 32 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.24.0-rancher1/tools/cache/reflector.go:168: Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Unauthorized
I0903 22:11:24.474577 32 leaderelection.go:283] failed to renew lease kube-system/cattle-controllers: timed out waiting for the condition
E0903 22:11:24.474647 32 leaderelection.go:306] Failed to release lock: resource name may not be empty
2022/09/03 22:11:24 [FATAL] leaderelection lost for cattle-controllers
Any known solution for that? I can't start rancher pod anymore.