https://rancher.com/ logo
Join Slack
Powered by
Rancher / Rancher-Monitoring question. We have a c...
# general
f
Rancher / Rancher-Monitoring question. We have a cluster we've upgraded through like 1.25 through 1.32 now. Rancher-Monitoring wound up not getting upgraded along the way and it's currently on 103.1.1+up45.31.1 and I'm attempting to upgrade to 106.1.2+up69.8.2-rancher.7 and I'm running into the following error:
Copy code
Error: UPGRADE FAILED: unable to build kubernetes objects from current release manifest: resource mapping not found for name: "rancher-monitoring-prometheus-node-exporter" namespace: "cattle-monitoring-system" from "": no matches for kind "PodSecurityPolicy" in version "policy/v1beta1"
Somewhere along the way of upgrading k3s, I want to say that the last remaining PodSecurityPolicy may have been deleted manually or by the process of upgrading k3s. I think helm is trying to find them to remove them and is of course unsuccessful. Anyone run into this with helm and know how to resolve this? More details will be in-thread
Full helm upgrade output
Untitled
After attempting the upgrade through the Rancher UI, I noticed that the CRDs are upgraded, but the app is at the older version
Copy code
% helm --kube-context=... -n cattle-monitoring-system list --all
NAME                  	NAMESPACE               	REVISION	UPDATED                                	STATUS  	CHART                                            	APP VERSION
rancher-monitoring    	cattle-monitoring-system	2       	2024-08-23 14:46:51.478744389 +0000 UTC	deployed	rancher-monitoring-103.1.1+up45.31.1             	v0.65.1
rancher-monitoring-crd	cattle-monitoring-system	4       	2025-06-26 00:06:44.869156938 +0000 UTC	deployed	rancher-monitoring-crd-106.1.2+up69.8.2-rancher.7
Yeah, I think the old release created a PSP...
Copy code
helm --kube-context=r/macc-kdev -n cattle-monitoring-system get manifest rancher-monitoring
in the output...
Copy code
---
# Source: rancher-monitoring/charts/prometheus-node-exporter/templates/psp.yaml
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: rancher-monitoring-prometheus-node-exporter
  namespace: cattle-monitoring-system
  labels:
    <http://helm.sh/chart|helm.sh/chart>: prometheus-node-exporter-4.20.0
    <http://app.kubernetes.io/managed-by|app.kubernetes.io/managed-by>: Helm
    <http://app.kubernetes.io/component|app.kubernetes.io/component>: metrics
    <http://app.kubernetes.io/part-of|app.kubernetes.io/part-of>: prometheus-node-exporter
    <http://app.kubernetes.io/name|app.kubernetes.io/name>: prometheus-node-exporter
    <http://app.kubernetes.io/instance|app.kubernetes.io/instance>: rancher-monitoring
    <http://app.kubernetes.io/version|app.kubernetes.io/version>: "1.6.0"
    jobLabel: node-exporter
    release: rancher-monitoring
spec:
  ...
I got help in the Kubernetes workspace under #helm-uses. There's a plugin called helm-mapkubeapis fixed up the existing release "manifests"; When I looked at 
helm get manifest rancher-monitoring --revision N
, it looks like the PodSecurityPolicy was removed from the history so it was able to continue the upgrade and skip deleting a resource from an API that no longer exists
Copy code
# add rancher repo
helm repo add rancher <https://charts.rancher.io/>
helm repo update

# add mapkubeapis plugin
helm plugin install <https://github.com/helm/helm-mapkubeapis>

# Upgrade snippets
helm -n cattle-monitoring-system list
helm -n cattle-monitoring-system mapkubeapis rancher-monitoring
helm -n cattle-monitoring-system upgrade rancher-monitoring-crd rancher/rancher-monitoring-crd --reset-then-reuse-values --wait
helm -n cattle-monitoring-system upgrade rancher-monitoring rancher/rancher-monitoring --reset-then-reuse-values --wait
Open in Slack
PreviousNext
Powered by