Hello all, Does anyone work with Rancher + Prometheus alerts + (possibly Slack or PagerDuty)? I need to configure 1) timing of scraping and 2) timing of sending alerts to slack when it is fired We have rancher 2.6.7 and Prometheus (from Apps charts) I have configured to send slack alerts, and I am getting alerts, but cannot find interval for scraping. Like, its sending fire alerts whenever it thinks it should be sending… 😏 I have under Monitoring > Advanced > PrometheusRules > resource kubernetes-apps and inside Group Name: kubernetes-apps (runbook from Github repo). Now I have changed timing of it in PromQL as well as duration which is given in seconds (above PromQL). I just dont know whether it is the location to configure scraping and sending alerts or does it have to be set through PodMonitoring in Monitoring (though podmonitoring should be just for pods) Please help…

Is your goal to send your alerts to Slack or PagerDuty?

Slack

and the scraping isn't pulling in anything to push to slack is it?

Thank you for getting back to me! This is an alert inside rancher -> Monitoring -> Advanced -> PrometheusRules -> rancher-monitoring-kubernetes-apps. Here I have set to 60 seconds to fire the alert in case pod is not ready. It is firing. Then I am getting slack alert after 6 minutes. 1. Now, I want to get alerts to Slack in specific time regarding this issue, lets say every 15 minutes, how do I do that? 2. Second thing is, what is the interval for scraping this pod again? Where is that configuration?

Off the top of my head I don't know. Let me check the docs and see if I can find it.

Thanks

I wasn't able to find this in our documentation. I am sure someone has done it before. I will look and see if anyone has come across this issue.

Thank you very much for your help, my task currently pending and if anything emerges please do let me know, eventually this is something I need to figure out…