Does anyone work with Rancher + Prometheus alerts + (possibly Slack or PagerDuty)?
I need to configure 1) timing of scraping and 2) timing of sending alerts to slack when it is fired
We have rancher 2.6.7 and Prometheus (from Apps charts)
I have configured to send slack alerts, and I am getting alerts, but cannot find interval for scraping. Like, its sending fire alerts whenever it thinks it should be sending… 😏
I have under Monitoring > Advanced > PrometheusRules > resource kubernetes-apps and inside Group Name: kubernetes-apps (runbook from Github repo). Now I have changed timing of it in PromQL as well as duration which is given in seconds (above PromQL).
I just dont know whether it is the location to configure scraping and sending alerts or does it have to be set through PodMonitoring in Monitoring (though podmonitoring should be just for pods)