acceptable-xylophone-16224
08/23/2022, 9:32 PMkubectl exec
using the Rancher generated Kubeconfig. The command results in a blank error message: Error from server:
. Upon further investigation, it seems that the exec
command is unable to post to the Rancher proxy for the API server, and it receives a 403 forbidden
response:
POST https://<rancher-url>/k8s/clusters/local/api/v1/namespaces/<namespace>/pods/<pod>/exec?command=sh&container=<container>&stdin=true&stdout=true&tty=true 403 Forbidden in 238 milliseconds
All of our requests are routed through nginx, and we've confirmed that the nginx ingress receives the same 403 error. However, turning on trace logs for the Rancher pods, it appears that the message was never processed in the pod. We've also verified that our RBAC rules are sufficient for executing an exec
command as all kubectl auth can-i
prompts return "yes." Please let me know if you have any ideas on troubleshooting this issue. Thank you!tall-school-18125
08/24/2022, 12:23 PMacceptable-xylophone-16224
08/24/2022, 1:13 PMkubectl exec
has two API calls through Rancher. The first request is a GET
request to list all the containers in a pod, and the second is a POST
request to run a command on the found container. The GET
request returns a 200
status, while the POST
request returns a 403
. This means the token is working for some API requests.tall-school-18125
08/24/2022, 1:24 PMacceptable-xylophone-16224
08/24/2022, 1:49 PMkubectl exec
command works from the Rancher shell in the cluster explorer. I’m only running into this issue on my local machine using the generated kubeconfig for that same cluster.