https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# random
a
This message was deleted.
d
My rke2 cluster has 3 control planes and 3 workers. For a few weeks now I have been noticing that the control planes are 100% CPU consuming. The kube-apiserver is the pod that is consuming the most. Does anyone have any idea what the offender could be?
kubernetes version: v1.24.16+rke2r1
Rancher version: v2.7.6
c
you’d need to figure out what’s putting so much load on the apiserver. The more client requests, the more CPU time it will take.
d
What is the best way to discover the origin of requests? Logs don't help much.
Copy code
0911 22:29:00.578953       1 webhook.go:154] Failed to make webhook authenticator request: the server rejected our request for an unknown reason
E0911 22:29:00.579020       1 authentication.go:70] "Unable to authenticate the request" err="[invalid bearer token, Token has been invalidated, the server rejected our request for an unknown reason]"
E0911 22:29:00.631198       1 webhook.go:154] Failed to make webhook authenticator request: the server rejected our request for an unknown reason
E0911 22:29:00.631239       1 authentication.go:70] "Unable to authenticate the request" err="[invalid bearer token, square/go-jose: error in cryptographic primitive, the server rejected our request for an unknown reason]"
E0911 22:29:00.631416       1 webhook.go:154] Failed to make webhook authenticator request: the server rejected our request for an unknown reason
E0911 22:29:00.631479       1 authentication.go:70] "Unable to authenticate the request" err="[invalid bearer token, square/go-jose: error in cryptographic primitive, the server rejected our request for an unknown reason]"
E0911 22:29:00.711165       1 webhook.go:154] Failed to make webhook authenticator request: the server rejected our request for an unknown reason
E0911 22:29:00.711204       1 authentication.go:70] "Unable to authenticate the request" err="[invalid bearer token, Token has been invalidated, the server rejected our request for an unknown reason]"
E0911 22:29:01.430705       1 webhook.go:154] Failed to make webhook authenticator request: the server rejected our request for an unknown reason
E0911 22:29:01.430759       1 authentication.go:70] "Unable to authenticate the request" err="[invalid bearer token, Token has been invalidated, the server rejected our request for an unknown reason]"
E0911 22:29:01.577810       1 webhook.go:154] Failed to make webhook authenticator request: the server rejected our request for an unknown reason
E0911 22:29:01.577884       1 authentication.go:70] "Unable to authenticate the request" err="[invalid bearer token, square/go-jose: error in cryptographic primitive, the server rejected our request for an unknown reason]"
E0911 22:29:01.642372       1 webhook.go:154] Failed to make webhook authenticator request: the server rejected our request for an unknown reason
Copy code
A piece of my log
c
turning on audit logging would probably be the best way to collect that data
👀 1
but that is non-trivial
Open in Slack
PreviousNext
Powered by