Anybody could give me a sense of sizing of a GKE cluster to host the rancher manager in HA mode to manage 10 small k8s clusters, about 10 human users of the UI/API, and run a prometheus/grafana stack for about total of 500 containers/apps/computers and other sensors. My goal is to avoid spawning too big of a cluster and be as cheap as possible but still have HA/prod level.
03/20/2023, 6:32 PM
so the rancher cluster itself wont need more than 3 nodes and around 4gi mem per node (8 to be super safe)
the prometheus stuff will take A LOT more memory
id recommend having a pool just for rancher, and a second pool dedicated just for monitoring
03/20/2023, 6:33 PM
I had all that on older computers on-premise with not enough IO/spindles to handle the needs of prometheus and the cluster itself. That was terrible 🙂
03/20/2023, 6:34 PM
both rancher and prom are really i/o intensive depending on how you set them up
rancher loves to write little tiny logs for EVERYTHING haha
03/21/2023, 12:45 AM
I run rancher-monitoring without any modifications and it uses on average 7G of RAM, spiking up to 15 or 20. That's with about 100 node cluster, with a few hundred containers running.