sticky-summer-13450
08/27/2022, 6:07 PMorange-cpu-47176
08/29/2022, 10:51 AMbrainy-whale-97450
08/29/2022, 6:38 PMstale-painting-80203
08/30/2022, 12:42 AMI have a Ubuntu 20.04 server VM running on a Harvester node, which I am using to process large files. when I runIt starts incrementing buff/cache memory, until it uses up 100% of free memory and causes the machine to reboot. If I continually clear the page cash usingsha256sum my-file
then I am able to prevent reboot.echo 1 > /proc/sys/vm/drop_caches
Is there anyway to change the VM configuration to fix this issues?
shy-restaurant-94821
08/30/2022, 2:10 PMAug 28 20:20:28 localhost kernel: mce: [Hardware Error]: CPU 0: Machine Check: 0 Bank 5: cc18cf0000010093
Aug 28 20:20:28 localhost kernel: mce: [Hardware Error]: TSC 0 ADDR 107d4a0c0 MISC 21400c0c86
Aug 28 20:20:28 localhost kernel: mce: [Hardware Error]: PROCESSOR 0:206d7 TIME 1661718020 SOCKET 0 APIC 0 microcode 71a
I wasn’t able to install 1.0.3 from a flashdrive or ISO. I had to install 1.0.2 and then update.
I’ve also noticed VM performance is terrible. I’ve tried deploying Rancher clusters to my Harvester box, but they never finish deploying and keep running into random errors. Sometimes a node will successfully go online, but it won’t stay online and will go back into provisioning.big-judge-33880
09/01/2022, 10:36 AMshy-restaurant-94821
09/01/2022, 5:06 PMshy-restaurant-94821
09/02/2022, 3:27 PMsticky-summer-13450
09/03/2022, 9:46 AMErrApplied
state, as seen in the logs of the pod running the upgrade job - as found using the command in that trouble-shooting page.
Unfortunately that troubleshooting section does not say what to do to remediate any issues at this stage.
I'm thinking, kill the pod and see what happens...?
harvester: v1.0.3
harvesterChart: 1.0.3
os: Harvester v1.0.3
kubernetes: v1.22.12+rke2r1
rancher: v2.6.4-harvester3
monitoringChart: 100.1.0+up19.0.3
kubevirt: 0.49.0-2
rancherDependencies:
fleet:
chart: 100.0.3+up0.3.9
app: 0.3.9
fleet-crd:
chart: 100.0.3+up0.3.9
app: 0.3.9
rancher-webhook:
chart: 1.0.4+up0.2.5
app: 0.2.5
<http://managedchart.management.cattle.io/harvester|managedchart.management.cattle.io/harvester> patched
<http://managedchart.management.cattle.io/harvester-crd|managedchart.management.cattle.io/harvester-crd> patched
<http://managedchart.management.cattle.io/rancher-monitoring|managedchart.management.cattle.io/rancher-monitoring> patched
<http://managedchart.management.cattle.io/rancher-monitoring-crd|managedchart.management.cattle.io/rancher-monitoring-crd> patched
Upgrading Rancher
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0^M100 129 100 129 0 0 43000 0 --:--:-- --:--:-- --:--:-- 43000
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
^M 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0^M100 55.6M 100 55.6M 0 0 1326M 0 --:--:-- --:--:-- --:--:-- 1326M
time="2022-09-02T18:34:48Z" level=info msg="Extract mapping / => /tmp/upgrade/rancher"
time="2022-09-02T18:34:48Z" level=info msg="Checking local image archives in /tmp/upgrade/images for <http://index.docker.io/rancher/system-agent-installer-rancher:v2.6.4-harvester3|index.docker.io/rancher/system-agent-installer-rancher:v2.6.4-harvester3>"
time="2022-09-02T18:34:49Z" level=info msg="Extracting file run.sh to /tmp/upgrade/rancher/run.sh"
time="2022-09-02T18:34:49Z" level=info msg="Extracting file rancher-2.6.4-harvester3.tgz to /tmp/upgrade/rancher/rancher-2.6.4-harvester3.tgz"
time="2022-09-02T18:34:49Z" level=info msg="Extracting file helm to /tmp/upgrade/rancher/helm"
Rancher values:
bootstrapPassword: admin
features: multi-cluster-management=false,multi-cluster-management-agent=false
hostPort: 8443
ingress:
enabled: false
noDefaultAdmin: false
rancherImage: rancher/rancher
rancherImagePullPolicy: IfNotPresent
rancherImageTag: v2.6.4-harvester3
replicas: -2
systemDefaultRegistry: ""
tls: external
useBundledSystemChart: true
Skip update Rancher. The version is already v2.6.4-harvester3
Upgrading Harvester Cluster Repo
deployment.apps/harvester-cluster-repo patched
Waiting for deployment "harvester-cluster-repo" rollout to finish: 0 out of 1 new replicas have been updated...
Waiting for deployment "harvester-cluster-repo" rollout to finish: 1 old replicas are pending termination...
Waiting for deployment "harvester-cluster-repo" rollout to finish: 1 old replicas are pending termination...
deployment "harvester-cluster-repo" successfully rolled out
<http://clusterrepo.catalog.cattle.io/harvester-charts|clusterrepo.catalog.cattle.io/harvester-charts> patched
Upgrading Harvester
<http://managedchart.management.cattle.io/harvester-crd|managedchart.management.cattle.io/harvester-crd> patched
<http://managedchart.management.cattle.io/harvester|managedchart.management.cattle.io/harvester> configured
<http://managedchart.management.cattle.io/harvester|managedchart.management.cattle.io/harvester> patched
<http://managedchart.management.cattle.io/harvester-crd|managedchart.management.cattle.io/harvester-crd> patched
Waiting for ManagedChart fleet-local/harvester from generation 8
Target version: 1.0.3, Target state: modified
Current version: 1.0.3, Current state: null, Current generation: 8
Sleep for 5 seconds to retry
Current version: 1.0.3, Current state: ErrApplied, Current generation: 10
Sleep for 5 seconds to retry
Current version: 1.0.3, Current state: ErrApplied, Current generation: 10
Sleep for 5 seconds to retry
Current version: 1.0.3, Current state: ErrApplied, Current generation: 10
Sleep for 5 seconds to retry
sparse-ghost-91718
09/05/2022, 6:01 PMsticky-painter-30848
09/05/2022, 6:12 PMcrooked-scooter-58172
09/06/2022, 1:44 PMdazzling-france-77424
09/06/2022, 5:11 PMsticky-summer-13450
09/06/2022, 5:27 PMdazzling-france-77424
09/06/2022, 5:50 PMstale-orange-90901
09/06/2022, 6:50 PMsteep-teacher-68650
09/07/2022, 11:09 AMsticky-painter-30848
09/14/2022, 9:46 AMstocky-orange-9200
09/15/2022, 9:37 PMstocky-orange-9200
09/15/2022, 9:38 PMstocky-orange-9200
09/15/2022, 9:38 PMcuddly-chef-59548
09/20/2022, 2:43 PMcuddly-chef-59548
09/20/2022, 5:54 PMcrooked-scooter-58172
09/21/2022, 5:50 PMbig-judge-33880
09/22/2022, 3:52 PMbrainy-camera-57071
09/26/2022, 9:28 AMdamp-tent-77102
09/26/2022, 10:42 PMquick-sandwich-76600
09/27/2022, 11:45 PMgorgeous-television-46599
09/28/2022, 6:43 AMclean-river-69704
09/29/2022, 2:59 PM