the complete logs are -sh-4.2$ kubectl -n cattle-system logs -l app=rancher W0902 050727.435772 34 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/rke.cattle.io/v1/rkebootstraptemplates?resourceVersion=906146237&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") E0902 050727.435816 34 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/rke.cattle.io/v1/rkebootstraptemplates?resourceVersion=906146237&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") W0902 050727.470105 34 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/cluster.x-k8s.io/v1beta1/clusters?resourceVersion=906146238&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") E0902 050727.470153 34 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/cluster.x-k8s.io/v1beta1/clusters?resourceVersion=906146238&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") W0902 050727.868705 34 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/node.k8s.io/v1beta1/runtimeclasses?resourceVersion=906146214&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") E0902 050727.868769 34 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/node.k8s.io/v1beta1/runtimeclasses?resourceVersion=906146214&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") W0902 050728.145523 34 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/cert-manager.io/v1/issuers?resourceVersion=906146237&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") E0902 050728.145574 34 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/cert-manager.io/v1/issuers?resourceVersion=906146237&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") W0902 050728.613687 34 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/events.k8s.io/v1/events?resourceVersion=910043819&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") E0902 050728.613737 34 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/events.k8s.io/v1/events?resourceVersion=910043819&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") W0902 050727.131451 34 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/management.cattle.io/v3/projectalerts?resourceVersion=906146237&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") E0902 050727.131483 34 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/management.cattle.io/v3/projectalerts?resourceVersion=906146237&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") W0902 050727.656405 34 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/admissionregistration.k8s.io/v1/validatingwebhookconfigurations?resourceVersion=906146214&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") E0902 050727.656441 34 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/admissionregistration.k8s.io/v1/validatingwebhookconfigurations?resourceVersion=906146214&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") W0902 050727.756945 34 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/certmanager.k8s.io/v1alpha1/issuers?resourceVersion=906146214&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") E0902 050727.757022 34 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/certmanager.k8s.io/v1alpha1/issuers?resourceVersion=906146214&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") W0902 050728.474378 34 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/apps/v1/replicasets?resourceVersion=906146727&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") E0902 050728.474485 34 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/apps/v1/replicasets?resourceVersion=906146727&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") W0902 050728.627930 34 reflector.go:325] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/management.cattle.io/v3/multiclusterapps?resourceVersion=906146237&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") E0902 050728.627966 34 reflector.go:139] pkg/mod/github.com/rancher/client-go@v1.23.3-rancher2/tools/cache/reflector.go168 Failed to watch *summary.SummarizedObject: failed to list *summary.SummarizedObject: Get "https://10.43.0.1:443/apis/management.cattle.io/v3/multiclusterapps?resourceVersion=906146237&timeout=15m0s": x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "kube-ca") 2022/09/02 050708 [INFO] Waiting for server to become available: the server has asked for the client to provide credentials 2022/09/02 050710 [INFO] Waiting for server to become available: the server has asked for the client to provide credentials 2022/09/02 050712 [INFO] Waiting for server to become available: the server has asked for the client to provide credentials 2022/09/02 050714 [INFO] Waiting for server to become available: the server has asked for the client to provide credentials 2022/09/02 050716 [INFO] Waiting for server to become available: the server has asked for the client to provide credentials 2022/09/02 050718 [INFO] Waiting for server to become available: the server has asked for the client to provide credentials 2022/09/02 050720 [INFO] Waiting for server to become available: the server has asked for the client to provide credentials 2022/09/02 050722 [INFO] Waiting for server to become available: the server has asked for the client to provide credentials 2022/09/02 050724 [INFO] Waiting for server to become available: the server has asked for the client to provide credentials 2022/09/02 050726 [INFO] Waiting for server to become available: the server has asked for the client to provide credentials -sh-4.2$

@fast-piano-59234 actually you were right - it was the rkestate file. though I still would have had to upgrade cert-manager. Thanks!!