https://rancher.com/ logo
Title
e

enough-furniture-17909

07/14/2022, 5:26 PM
hi, we upgraded rancher from 2.5.12 to 2.6.6 today and are noticing a significant decrease in the new UI's performance when viewing one of our larger clusters ( v1.20 on 40 nodes, with 1400 deployments across 7 namespaces) for example, it takes 20seconds just to view the list of nodes and chrome's task manger is showing mem usage of 4,700,00K and 100 cpu. Does anyone have any suggestions on any settings we can tweak to improve the ui's responsiveness?
t

tall-school-18125

07/15/2022, 11:39 PM
This is a known issue and we are working on it, with some fixes scheduled for the v2.7.0 release. Can you please share more details about how many resources are in the cluster? It will help us reproduce the issue. • How many users? • How many roles? • How many rolebindings? • How many namespaces? • How many projects?
e

enough-furniture-17909

07/18/2022, 12:20 PM
@tall-school-18125, thanks for the response. We reverted to 2.5.12 for now. Is there a rough timeframe on the 2.7 release? Here's some stats from that cluster: • How many users? 348 • How many roles? 43 • How many rolebindings? 354 • How many namespaces? 29 • How many projects? 14 • total resources: 3779 • deployments: 1443 • daemonsets: 41 • statefulsets: 30 • pods: 3617 • avg # of events listed on old, 2.5 rancher cluster dashboard: ~3000 • cluster was originally created using rancher 2.4.4 back in 2019 and is currently running kubernetes 1.20.11 Thanks, Mark
t

tall-school-18125

07/18/2022, 4:31 PM
Thank you, this helps. I can't give exact dates for the 2.7 release because it could change, but it is currently scheduled for some time in October.
e

enough-furniture-17909

07/18/2022, 4:41 PM
@tall-school-18125 thanks! I don't suppose a new rancher 2.5.x that supports kubernetes 1.21 is out or coming out before then? I ask because part of the reason for the upgrade for us is aws eks end of life for 1.20 is Nov 1st and we'd like all our clusters to be running same version (1.21) before then .
t

tall-school-18125

07/18/2022, 4:50 PM
I asked internally - Kubernetes 1.21 support won't be in 2.5.x
e

enough-furniture-17909

07/18/2022, 4:53 PM
gotcha. thanks again!
t

tall-school-18125

07/19/2022, 9:12 PM
If you have some time, can you please try to get these different endpoints and let us know if v3 is faster? It would help us figure out if /v3 is faster if there are a lot of user sessions at once. • [rancherServerUrl]/v1/nodes vs [rancherServerUrl]/v3/nodes • [rancherServerUrl]/v1/management.cattle.io.users vs [rancherServerUrl]/v3/users
e

enough-furniture-17909

07/19/2022, 9:32 PM
@tall-school-18125 - stats via chrome dev tools against rancher 2.5.12. nodes v1: 048ms, 3.0kB nodes v3: 271ms, 26.7kB users v1: 301ms, 58.0kB users v3: 282ms, 49.0 kB
t

tall-school-18125

07/19/2022, 9:36 PM
ok, thank you.
w

wonderful-tent-44389

07/19/2022, 9:47 PM
Oh for v3 that works, but for v1 that url is for the mgmt cluster. Can you time getting the nodes for the downstream cluster. Something like:
/k8s/clusters/c-XYZ/v1/nodes
t

tall-school-18125

07/19/2022, 9:51 PM
for v1 that url is for the mgmt cluster
@wonderful-tent-44389 That was intentional - My hypothesis is that the Steve API for the mgmt cluster is going to be slower than the Steve API for downstream clusters because it is shared across all logged in users and there is a new watcher for each user session.
So I think the slowness would be noticeable for management cluster resources where the UI for cluster explorer has to fetch all the resources for all clusters from the Steve API of the management cluster, then filter them down by cluster ID
w

wonderful-tent-44389

07/19/2022, 9:59 PM
Ahh gotcha.
e

enough-furniture-17909

12/15/2022, 3:10 PM
@tall-school-18125 i see that 2.7 is out now. Were any of the UI performance issues addressed in this major release? I did not see any comments in the release notes related to it.