https://rancher.com/ logo
Title
c

curved-lifeguard-39360

08/18/2022, 4:22 PM
@fast-piano-59234 @full-painter-23916 I am pinging you two because you have had good answers for me in the past. I am experiencing the same issue as outlined in this issue on github. https://github.com/rancher/rancher/issues/21324. However that issue has been open since 2019 with no resolution. Basically what happens is that I run this command:
rancher kubectl rollout status -w deployment acmepa -n acmepa
It immediately replys with
deployment "acmepa" successfully rolled out
but then hangs there for 5 minutes or so.
This does not happen on my MAC or in Ubuntu 18. But does on Ubuntu 20.
t

tall-school-18125

08/18/2022, 5:49 PM
what kubernetes version is it?
and what Rancher version?
c

curved-lifeguard-39360

08/18/2022, 5:52 PM
rancher 2.6.1, k8s EKS 1.20
I just built a brand new deployment machine using ubuntu 18. And encountered the same problem.
Ran:
rancher kubectl rollout status -w deployment acmepa -n acmepa
and got this response:
deployment "acmepa" successfully rolled out
E0818 17:54:51.211673    8228 reflector.go:383] <http://k8s.io/client-go/tools/watch/informerwatcher.go:146|k8s.io/client-go/tools/watch/informerwatcher.go:146>: Failed to watch *unstructured.Unstructured: Get <https://rch2.panosoft.com/k8s/clusters/c-mjkrt/apis/apps/v1/namespaces/acmepa/deployments?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dacmepa&resourceVersion=134547883&timeoutSeconds=461&watch=true>: unexpected EOF
similar to others that posted in that tread I linked to earlier.
t

tall-school-18125

08/18/2022, 6:02 PM
In the comments on that issue, they said it worked when http2 was disabled in the ingress - https://github.com/rancher/rancher/issues/21324#issuecomment-952457417 does that have any effect?
c

curved-lifeguard-39360

08/18/2022, 6:18 PM
I will check that. Not sure if I am using nginx lb or not.
hmmm that fix assumes that I have a configmap called
nginx-configuration
which I do not.
This was the command that I used to install and create the ingress:
helm repo add ingress-nginx <https://kubernetes.github.io/ingress-nginx>
helm repo update
helm upgrade --install \
  ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --set controller.service.type=LoadBalancer \
  --version 3.12.0 \
  --create-namespace
The result was and classic ELB.
weird thing is that was created 300 days ago. But this problem only recently started happening.
t

tall-school-18125

08/19/2022, 5:49 PM
Since it only recently started happening, did anything else change recently?
A bug with proxied kubectl commands hanging was fixed in 2.6.6 https://github.com/rancher/rancher/issues/37707, so I would be interested to know if this bug is also reproducible in 2.6.6.
c

curved-lifeguard-39360

08/26/2022, 4:58 PM
is 2.6.6 considered stable yet?
So I disabled http/2 on the server load balancer and that resolved the issue.
t

tall-school-18125

08/31/2022, 10:37 PM
2.6.7 is the latest stable version