https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# general
a
This message was deleted.
c
Basic network troubleshooting is where i’d start. Can you resolve the address? can you ping it? Can you perform a basic curl of that address without the extra parameters?
b
yeah, I don't think it is network issue since the node did download the script over the network from the rancher server
also I get an 401 http error running the curl command without the X-Cattle_Id header
if I run curl in verbose mode, I can see the traffic, both client and server agrees on the https cert ... then it hangs
and I can see traffic going both ways when I run tcpdump
c
are you sure that curl command is actually where it’s hanging then, and not actually some command after that?
b
I put "echo" everywhere to trace the script execution and that curl command is where it hangs
now that you ask, let me double check
so, as mentioned, I have the script like this
Copy code
echo "3"

            echo "######## ATTEMPT $i of $RETRYCOUNT"

            echo "calling ===> curl $noproxy --connect-timeout 60 --max-time 60 --write-out \"%{http_code}\n\" ${CURL_CAFLAG} ${CURL_LOG} -H \"Authorization: Bearer ${CATTLE_TOKEN}\" -H \"X-Cattle-Id: ${CATTLE_ID}\" -H \"X-Cattle-Role-Etcd: ${CATTLE_ROLE_ETCD}\" -H \"X-Cattle-Role-Control-Plane: ${CATTLE_ROLE_CONTROLPLANE}\" -H \"X-Cattle-Role-Worker: ${CATTLE_ROLE_WORKER}\" -H \"X-Cattle-Node-Name: ${CATTLE_NODE_NAME}\" -H \"X-Cattle-Address: ${CATTLE_ADDRESS}\" -H \"X-Cattle-Internal-Address: ${CATTLE_INTERNAL_ADDRESS}\" -H \"X-Cattle-Labels: ${CATTLE_LABELS}\" -H \"X-Cattle-Taints: ${CATTLE_TAINTS}\" \"${CATTLE_SERVER}\"/v3/connect/agent -o ${CATTLE_AGENT_VAR_DIR}/rancher2_connection_info.json"

            RESPONSE=$(curl $noproxy --connect-timeout 60 --max-time 60 --write-out "%{http_code}\n" ${CURL_CAFLAG} ${CURL_LOG} -H "Authorization: Bearer ${CATTLE_TOKEN}" -H "X-Cattle-Id: ${CATTLE_ID}" -H "X-Cattle-Role-Etcd: ${CATTLE_ROLE_ETCD}" -H "X-Cattle-Role-Control-Plane: ${CATTLE_ROLE_CONTROLPLANE}" -H "X-Cattle-Role-Worker: ${CATTLE_ROLE_WORKER}" -H "X-Cattle-Node-Name: ${CATTLE_NODE_NAME}" -H "X-Cattle-Address: ${CATTLE_ADDRESS}" -H "X-Cattle-Internal-Address: ${CATTLE_INTERNAL_ADDRESS}" -H "X-Cattle-Labels: ${CATTLE_LABELS}" -H "X-Cattle-Taints: ${CATTLE_TAINTS}" "${CATTLE_SERVER}"/v3/connect/agent -o ${CATTLE_AGENT_VAR_DIR}/rancher2_connection_info.json)

            echo "call done with response $RESPONSE"
and I get the output
Copy code
3
######## ATTEMPT 1 of 4500
calling ===> curl  --connect-timeout 60 --max-time 60 --write-out "%{http_code}\n"  -sS -H "Authorization: Bearer ..." -H "X-Cattle-Id: ..." -H "X-Cattle-Role-Etcd: true" -H "X-Cattle-Role-Control-Plane: true" -H "X-Cattle-Role-Worker: true" -H "X-Cattle-Node-Name: " -H "X-Cattle-Address: " -H "X-Cattle-Internal-Address: " -H "X-Cattle-Labels: <http://cattle.io/os=linux|cattle.io/os=linux>" -H "X-Cattle-Taints: " "https://..."/v3/connect/agent -o /var/lib/rancher/agent/rancher2_connection_info.json
and I manually call the script like
Copy code
./system-agent-install.sh --server https://... --label '<http://cattle.io/os=linux|cattle.io/os=linux>' --token .... --etcd --controlplane --worker
im pretty sure the curl command hangs --> the rancher server is not sending a response back
h
DNS? If its curl command, can you resolve the name?
b
yes can resolve the name
h
nc -v server 443 ?
b
if I do curl to the rancher server I get a json back
network is fine
requests with authorization bearer and x-cattle-id headers hangs (at least to the /v3/connect/agent endpoint)
c
maybe check logs on the rancher pod then?
b
is this rancher pod running on the rke2 machine or is it running in the hosts running rancher?
c
the one that you’re making the request against
b
ah, ok I dont have access to that host, I will ask for help to the rancher sys admins tomorrow
33 Views
Open in Slack
PreviousNext
Powered by