Hello, Since I’ve upgraded to Rancher 2.11.x (test...
# general
f
Hello, Since I’ve upgraded to Rancher 2.11.x (tested on 2.11.1 and 2.11.2), I can’t delete namespace anymore. They are stuck in Terminating and when I try to get all resources in the namespace I got following errors.
Copy code
$ kubectl get-all -n <stuck-ns>
W0527 11:15:24.709087 2243547 client.go:102] Could not fetch complete list of API resources, results will be incomplete: unable to retrieve the complete list of server APIs: ext.cattle.io/v1: the server is currently unable to handle the request
h
if you edit the namespace what does it show under finalizers?
Copy code
spec:
  finalizers:
whatever that is, is the reason
f
The finalizer is
kubernetes
h
if you delete that line does it finally delete the namespace ?
f
No, I have to remove the
kubernetes
finalizer to the namespace resource, then apply it to
/finalize
endpoint to remove the namespace.
The issue is raised on every namespace deletion.
h
oh - I see; sorry - I need tea this morning
f
The root cause seems to be linked to the absence of
<http://ext.cattle.io/v1|ext.cattle.io/v1>
resources API.
Copy code
$ kubectl get apiservice <http://v1.ext.cattle.io|v1.ext.cattle.io>
NAME               SERVICE                                  AVAILABLE                  AGE
<http://v1.ext.cattle.io|v1.ext.cattle.io>   cattle-system/imperative-api-extension   False (MissingEndpoints)   2d4h
Copy code
$ kubectl api-resources | grep <http://ext.cattle.io|ext.cattle.io>
<empty>
The
imperative-api-extension
service selects an undefined label (
app=rancher
) instead of (
app=rancher-manager
)
Copy code
$ kubectl describe -n cattle-system service/imperative-api-extension
Name:                     imperative-api-extension
Namespace:                cattle-system
Labels:                   <none>
Annotations:              <none>
Selector:                 app=rancher
Type:                     ClusterIP
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       10.96.98.186
IPs:                      10.96.98.186
Port:                     <unset>  6666/TCP
TargetPort:               6666/TCP
Endpoints:
Session Affinity:         None
Internal Traffic Policy:  Cluster
Events:                   <none>
The endpoints are empty
Copy code
$ kubectl get endpoints -n cattle-system imperative-api-extension
NAME                       ENDPOINTS   AGE
imperative-api-extension   <none>      2d4h
When updated the label selector, the change fixed the issue.
c
You might comment on that issue, noting the impact your seeing wrt namespace deletion
f
Comment done, thank for advice