https://rancher.com/ logo
Join Slack
Powered by
Hello guys, I am running k3s in my homelab and scr...
# k3s
w
Hello guys, I am running k3s in my homelab and scrapping its metrics from an external prometheus container which can access this node over tailscale. I am able to access other sources but not controller manager and scheduler
Copy code
kube-controller-manager-arg:
  - "bind-address=0.0.0.0"
kube-scheduler-arg:
  - "bind-address=0.0.0.0"
tls-san:
  - "localhost"
  - "127.0.0.1"
  - "100.71.124.99"
  - "dexy"
etcd-expose-metrics: true
etcd-arg:
  - "listen-metrics-urls=<http://127.0.0.1:2381>,<http://100.71.124.99:2381>"   # for metrics
node-ip: 100.71.124.99
node-external-ip: 100.71.124.99
advertise-address: 100.71.124.99
flannel-iface: tailscale0
I want to scrape metrics only over the tailscale network and not over any other network but loopback. But whatever bind addresses I try to provide for them, I am unable to access 
kube-controller-manager
and 
kube-scheduler
metrics.
When I try to curl this node in my tailnet from another node I am not able to get any metrics
Copy code
]$ curl --cacert ./ca.crt   -H "Authorization: Bearer $(cat ./kube-sa-token)"   <https://100.71.124.99:10259/metrics>
curl: (60) SSL: no alternative certificate subject name matches target host name '100.71.124.99'
More details here: <https://curl.se/docs/sslcerts.html>

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.
even if I try to run it on the same node I am not able to get the metrics but If I run the same command on loopback, it works
Copy code
]$ curl --cacert ./ca.crt   -H "Authorization: Bearer $(cat ./kube-sa-token)"   <https://127.0.0.1:10259/metrics>

works
Also other endpoints which appear to be up in prometheus image attached are accessible using this curl command with tailnet IP
It would be really helpful if someone can guide me related to this.
c
You don't need to scrape them all separately. K3s runs all the components together in one process, so all the component metrics are available on all metrics ports. You already have all the metrics you need.
w
ohh is that the case.
does that mean that the combined metrics from any worker node is exported with cadvisor on port 10250?
Then I wont need to bind these to 0.0.0.0 anyway
3 Views
Open in Slack
PreviousNext
Powered by