https://rancher.com/ logo
#general
Title
# general
a

adamant-kite-43734

09/12/2022, 12:14 PM
This message was deleted.
b

bitter-ability-10964

09/14/2022, 6:57 AM
Yes, of course. You only have to: 1. Deploy Prometheus chart in your clusters 2. Configure Grafana in one of your clusters (central monitoring cluster) to use the prometheus of other clusters as datasource. For this, you must expose the Prometheus in other clusters using Ingress and protect them, e.g. by IP address whitelists.
s

salmon-carpenter-62625

09/14/2022, 7:40 AM
ok but the rancher monitoring charts are something that can be installed by manual way or should it be done by rancher api? Also what about the federation mode, I suppose that when we expose prometeush and have no a master one then then there is a risk of losing metrics if cluster collapse, So it's like to have a monitoring on the same cluster, so we will now it's down but not know why till it's back.
b

bitter-ability-10964

09/14/2022, 7:45 AM
You can deploy manually via Rancher UI or via Terraform. We chose to deploy via rancher2 Terraform provider, which at the end uses the API
If you use Longhorn or any other high-available storage layer, then you won't loose metrics.
s

salmon-carpenter-62625

09/14/2022, 7:56 AM
I mean here about the situation where your cluster is not avaiable, ok you have the data on the storage but when you login to grafana there is a risk that prometeus that use that storage is on high load and grafana will not be able to get the data and display it so till fix of your cluster you will not see your metrics. That's why my plan was to use prometeus in federation mode so the main one will scrap the metrics from anothers till they work and by that that I should see them
till collapse
also I read about architecture of monitoring v2 and noted that rancher2 use pushprox for some cluster components, so not sure if I should play now with federation mode or maybe configure pushprox to send metrics to central prometheus.
b

bitter-ability-10964

09/14/2022, 7:59 AM
To be honest, I don't know what you mean with federation mode
s

salmon-carpenter-62625

09/14/2022, 8:01 AM
b

bitter-ability-10964

09/14/2022, 8:02 AM
Ah... As always I would start with a simple deployment and then if there is a good reason to extend it
Don't impose too much complexity right from the start
s

salmon-carpenter-62625

09/14/2022, 9:27 AM
huh, yea but I have limited decision here so try to figure out what's possible.
Btw I see that can install monitoring like 100.1.0+up19.0.3 from UI and I wonder how this stand to what I see on https://github.com/rancher/system-charts/tree/release-v2.6/charts/rancher-monitoring
More over from teraform site https://registry.terraform.io/providers/rancher/rancher2/latest/docs/resources/cluster I can read:
*Note* Cluster monitoring version 0.2.0 or above, can't be enabled until cluster is fully deployed as kubeVersion requirement has been introduced to helm chart
So it require to have additional post-install step. Also I cannot find any single manual instruction how to install monitoring charts by non web ui way.
I know I can just use the helm and deploy the chart taken from repository but again there might be some settings that I'm not aware and set by rancher logic
b

bitter-ability-10964

09/14/2022, 9:38 AM
You don't deploy the internal (old) monitoring by Terraform. You should deploy the chart using the Terraform chart resource
s

salmon-carpenter-62625

09/14/2022, 9:44 AM
b

bitter-ability-10964

09/14/2022, 9:45 AM
Yes
s

salmon-carpenter-62625

09/14/2022, 9:47 AM
ok, do you know if this resource do something extra or just act as a helm in backround so take the values and deploy chart?
I wonder how it differ from helm_release resoucres
b

bitter-ability-10964

09/14/2022, 9:51 AM
In my understanding, it acts like a Helm client. What resource is helm_release?
I also found annoying issue of rancher2_app_v2: https://github.com/rancher/terraform-provider-rancher2/issues/576
b

bitter-ability-10964

09/14/2022, 9:54 AM
Yes, you could also use the Helm provider. But because we also do some other tasks with Rancher, we decided to use the Rancher2 terraform resource
And yes, this is an issue. But one can cope with it. Every provider has some issues...
s

salmon-carpenter-62625

09/14/2022, 9:58 AM
ok wil play a bit on both to see what match more my needs. I used monitoring v1 in the past and it works well more or less but here and now the descition was change to use dedicated cluster just for monitoring purpose and wonder how to setup all of that.
18 Views