Hi guys, We are facing couple of serious issues in rancher. 1.Issue with prometheus • A month back we had 50x more load than usual. Since then we are having issue with prometheus. • We couldn't see the metrics and some of the prometheus pods are crashing and there are going to crash loop backoff state. • We tried a few solutions like increasing the resources, deleting the

/wal

folder but with no success • This is causing some issue in autoscaling i.e horizontal pod auto scaler 2. New users cannot login to rancher • We are not able to add new users using their github account or even manually to rancher • We see this issue - https://forums.rancher.com/t/x509-certificate-has-expired-or-is-not-yet-valid/20518 • We already tried the solution from this post and it did not work We are new to rancher. The guy who has setup the entire thing is not in the team no more. So we are trying trial and error with zero knowledge. It would be helpful if you guys help us resolving this. Thanks..

can you share how your rancher is setup and how are you running prometheus , With Prometheus if the service was down then it would show blank data for that time period