Worked out a solution to this by importing the following YAML in Rancher:
apiVersion:
helm.cattle.io/v1
kind: HelmChartConfig
metadata:
name: rke2-ingress-nginx
namespace: kube-system
spec:
valuesContent: |-
controller:
config:
enable-underscores-in-headers: true