< fast piano 59234> < full painter 23916> I am pinging you t rancher-users #general

Title

curved-lifeguard-39360

08/18/2022, 4:22 PM

@fast-piano-59234 @full-painter-23916 I am pinging you two because you have had good answers for me in the past. I am experiencing the same issue as outlined in this issue on github. https://github.com/rancher/rancher/issues/21324. However that issue has been open since 2019 with no resolution. Basically what happens is that I run this command:

rancher kubectl rollout status -w deployment acmepa -n acmepa

It immediately replys with

deployment "acmepa" successfully rolled out

but then hangs there for 5 minutes or so.

This does not happen on my MAC or in Ubuntu 18. But does on Ubuntu 20.

tall-school-18125

08/18/2022, 5:49 PM

what kubernetes version is it?

and what Rancher version?

curved-lifeguard-39360

08/18/2022, 5:52 PM

rancher 2.6.1, k8s EKS 1.20

I just built a brand new deployment machine using ubuntu 18. And encountered the same problem.

Ran:

rancher kubectl rollout status -w deployment acmepa -n acmepa

and got this response:

deployment "acmepa" successfully rolled out
E0818 17:54:51.211673    8228 reflector.go:383] <http://k8s.io/client-go/tools/watch/informerwatcher.go:146|k8s.io/client-go/tools/watch/informerwatcher.go:146>: Failed to watch *unstructured.Unstructured: Get <https://rch2.panosoft.com/k8s/clusters/c-mjkrt/apis/apps/v1/namespaces/acmepa/deployments?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dacmepa&resourceVersion=134547883&timeoutSeconds=461&watch=true>: unexpected EOF

similar to others that posted in that tread I linked to earlier.

tall-school-18125

08/18/2022, 6:02 PM

In the comments on that issue, they said it worked when http2 was disabled in the ingress - https://github.com/rancher/rancher/issues/21324#issuecomment-952457417 does that have any effect?

curved-lifeguard-39360

08/18/2022, 6:18 PM

I will check that. Not sure if I am using nginx lb or not.

hmmm that fix assumes that I have a configmap called

nginx-configuration

which I do not.

This was the command that I used to install and create the ingress:

helm repo add ingress-nginx <https://kubernetes.github.io/ingress-nginx>
helm repo update
helm upgrade --install \
  ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --set controller.service.type=LoadBalancer \
  --version 3.12.0 \
  --create-namespace

The result was and classic ELB.

weird thing is that was created 300 days ago. But this problem only recently started happening.

tall-school-18125

08/19/2022, 5:49 PM

Since it only recently started happening, did anything else change recently?

A bug with proxied kubectl commands hanging was fixed in 2.6.6 https://github.com/rancher/rancher/issues/37707, so I would be interested to know if this bug is also reproducible in 2.6.6.

curved-lifeguard-39360

08/26/2022, 4:58 PM

is 2.6.6 considered stable yet?

So I disabled http/2 on the server load balancer and that resolved the issue.

tall-school-18125

08/31/2022, 10:37 PM

2.6.7 is the latest stable version

4 Views

#general Join Slack