https://rancher.com/ logo
Join Slack
Powered by
Moin all! I'm trying to set a proxy in my harvest...
# harvester
m
Moin all! I'm trying to set a proxy in my harvester single node PoC through the UI. For some reason it won't save the changes though. Is there anything specific I need to do here? I add both http and https proxy like this 
<http://172.10.11.12:8080>
and the noproxy I add the usual from the docs 
localhost,127.0.0.1,0.0.0.0,10.0.0.0/8,...
The harvester CIDR's are default so they should be covered by the 
10.0.0.0/8
Saving doesn't throw an error but when I check the config again it's empty 🤔
r
Hi Paul, do you have any screenshots or perhaps a support bundle generated that we can take a closer look at the issue?
m
This is what I want to add
after save I see this
(the proxy IP is a dummy here, not sure if that matters)
I generated a support bundle, that surely won't doxx my self, right? 😄 I can share it too
t
Just curious what you are using the proxy for?
m
As of now, I do not want to use the proxy. At least I do not expect it to be used. On the harvester side, it's single node PoC. The initial setup and bringup of a VM in harvester works as expected. Now when I want to join the harvester through the virtualisation management in my rancher cluster, the rancher-cluster-agent pod in harvester starts to talk to my general proxy for some reason. I do not know where it knows my proxy IP from. I did not configure it during the initial setup of harvester and I couldn't find it in any environment, not harvester or in the pod or elsewhere (as of my limited knowledge inside harvester). My harvester node is in a different network than my main rancher cluster.
Before trying to join harvester into rancher, I see no traffic towards the proxy on tcpdump on harvester
My idea was then to just add the proxy and correct noproxy entries, but those aren't saved 😕
t
That is odd. Does Rancher have proxy set? That’s where it’s probably getting the proxy settings.
m
The rancher global settings doesn't have a proxy config, but it's old RKE1. Maybe it's in the docker.json, let me check.
I also checked the yml which is served by rancher for harvester to join and that doesn't include any proxy settings.
hmm docker.json doesn't have any proxies set.
r
Are you using the Rancher dashboard to access the imported Harvester cluster and configure the proxy settings?
May I know what version is this?
m
Rancher is running 2.9.1 which might be an issue because it's kinda old. I'm trying to configure the proxy directly through the harvester UI
at the moment, there is no connection between the rancher and harvester which I assume is related to the agent trying to use the proxy.
I have a separate harvester/rancher PoC where rancher is running on harvester directly and I'm also not able to update the proxy there. although I only tried it through harvester UI directly. I will try through rancher->harvester
t
What version of harvest are you running?
m
1.50 stable release
(in both setups)
t
Just curious if Harvester should use a proxy to get to your Rancher?
m
Technically it shouldn't. I can netcat between the harvester and rancher nodes on the ports from the docs.
but you might be on to something. I think we didn't set a default route for the harvester node network 🤔
ah nevermind, we have a default route
t
Somethings odd, you don’t have a proxy setting in Rancher, you don’t have a proxy setting in harvester. So why would all of a sudden it start wanting to talk to a proxy.
m
yea that's the confusion. I assume it has something do to with what rancher "informs" harvester about. but I have no idea how to check that. Sadly because of how our infra looks like, I do not have access to any logs of the proxy. No easy way to tell what's being sent there 😞
I naivly though I could loop over the resources with kubectl and grep for proxy/the proxy IP. That's what I'm doing right now. Not sure if that makes sense though.
is there a way to set the same proxy through kubectl which I would set through the webinterface?
What's also confusing is that /etc/systemd/system/rancher-system-agent.env shows both proxy values as empty and noproxy with the defaults as per docs 🤔
t
Maybe look at the Yaml that Rancher is producing in the browser to see what is getting created in Harvester. Aka the client config url.
r
is there a way to set the same proxy through kubectl which I would set through the webinterface?
Yes, that’s what I was going to suggest you do.
Try
Copy code
kubectl edit settings.harvesterhci http-proxy
and put the following in the 
value
field
Copy code
'{"httpProxy":"<your-http-proxy>","httpsProxy":"<your-http-proxy>","noProxy":"<your-no-proxy>"}'
m
I checked a bit deeper into tcpdump and I see that the proxy is getting requests for the harvester internal 10.53.0.1!
if I set noproxy, do I also need to seed the proxy values?
r
what do you mean
m
I mean, the settings doesn't show anything. And I would like to set just the no_proxy values and leave the proxy blank (since I do not want to use the proxy anyways). This hopefully makes sure that whatever causes the proxy to be used, at least skips requests to 10.0.0.0/8
but I'm not sure if it is required to set the proxy if I set no_proxy 🙂
r
sorry i’m confused if there’s no proxy set, why do you need a noproxy setting?
m
that's the main problem I have. I do not set the proxy, but the proxy is being used.
r
btw, i can reproduce the original issue in a clean install v1.5.0 cluster
m
and I'm trying to find ways around it, because I couldn't find a way to fix it 🙂
r
so it’s a bug
m
Good to know, thanks for confirming that on your end.
r
you’re willing to help, i invite you to create a GH issue for it
or i can create it if you have concerns
m
I'll do that. I should check https://github.com/harvester/harvester/issues for issues beforehand. That's the right place, no?
btw, setting the proxy with kubectl worked and shows the correct values in the UI as well.
kind of off-topic, my first github issue. How do I add the correct labels? 🙈
r
nice! thank you Paul
we’ll help to triage the issue. no worries.
👍 1
m
On the original problem in my infrastructure: I actually found out where the rouge proxy comes from. Somehow it is defined in fleetcontroller deployments which must come from rancher somehow. I’ll investigate that closer.
Open in Slack
PreviousNext
Powered by