This message was deleted Rancher Users #harvester

Join Slack

This message was deleted.

# harvester

adamant-kite-43734

09/03/2023, 5:52 PM

This message was deleted.

prehistoric-balloon-31801

09/04/2023, 9:47 AM

I'll check rke2-server log:

Copy code

journalctl -u rke2-server

aloof-agency-97600

09/04/2023, 7:49 PM

Copy code

harvester:~ # journalctl -u rke2-server
Jun 23 02:12:53 harvester rke2[14011]: time="2023-06-23T02:12:53Z" level=warning msg="Proxy error: read failed: read tcp 127.0.0.1:9345->127.0.0.1:38482: read: connection reset by peer"
Jun 23 02:41:13 harvester rke2[14011]: time="2023-06-23T02:41:13Z" level=warning msg="Proxy error: read failed: read tcp 127.0.0.1:9345->127.0.0.1:40498: read: connection reset by peer"
-- Boot 90f98dd372d34fadbd98167a10a6eb37 --
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="certificate CN=harvester signed by CN=rke2-server-ca@1682837514: notBefore=2023-04-30 06:51:54 +0000 UTC notAfter=2024-06-22 09:1>
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="certificate CN=system:node:harvester,O=system:nodes signed by CN=rke2-client-ca@1682837514: notBefore=2023-04-30 06:51:54 +0000 U>
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Module overlay was already loaded"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Module br_netfilter was already loaded"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_max' to 196608"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_established' to 86400"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_tcp_timeout_close_wait' to 3600"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Set sysctl 'net/ipv4/conf/all/forwarding' to 1"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Runtime image <http://index.docker.io/rancher/rke2-runtime:v1.24.11-rke2r1|index.docker.io/rancher/rke2-runtime:v1.24.11-rke2r1> bin and charts directories already exist; skipping extract"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Updated manifest /var/lib/rancher/rke2/server/manifests/harvester-cloud-provider.yaml to set cluster configuration values"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="No cluster configuration value changes necessary for manifest /var/lib/rancher/rke2/server/manifests/rancher/managed-chart-config>
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Updated manifest /var/lib/rancher/rke2/server/manifests/rancher-vsphere-cpi.yaml to set cluster configuration values"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-multus.yaml to set cluster configuration values"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="No cluster configuration value changes necessary for manifest /var/lib/rancher/rke2/server/manifests/rancher/addons.yaml"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Updated manifest /var/lib/rancher/rke2/server/manifests/rancher-vsphere-csi.yaml to set cluster configuration values"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-cilium.yaml to set cluster configuration values"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Updated manifest /var/lib/rancher/rke2/server/manifests/harvester-csi-driver.yaml to set cluster configuration values"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-calico-crd.yaml to set cluster configuration values"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-calico.yaml to set cluster configuration values"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-metrics-server.yaml to set cluster configuration values"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="No cluster configuration value changes necessary for manifest /var/lib/rancher/rke2/server/manifests/rancher/cluster-agent.yaml"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="No cluster configuration value changes necessary for manifest /var/lib/rancher/rke2/server/manifests/rancher/rke2-etcd-snapshot-e>
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-canal.yaml to set cluster configuration values"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-coredns.yaml to set cluster configuration values"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Updated manifest /var/lib/rancher/rke2/server/manifests/rke2-ingress-nginx.yaml to set cluster configuration values"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Logging containerd to /var/lib/rancher/rke2/agent/containerd/containerd.log"
Jun 23 09:10:14 harvester rke2[1845]: time="2023-06-23T09:10:14Z" level=info msg="Running containerd -c /var/lib/rancher/rke2/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/>
Jun 23 09:10:15 harvester rke2[1845]: time="2023-06-23T09:10:15Z" level=info msg="Waiting for containerd startup: rpc error: code = Unknown desc = server is not initialized yet"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="containerd is now running"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Pulling images from /var/lib/rancher/rke2/agent/images/cloud-controller-manager-image.txt"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Imported images from /var/lib/rancher/rke2/agent/images/cloud-controller-manager-image.txt in 1.845247ms"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Pulling images from /var/lib/rancher/rke2/agent/images/etcd-image.txt"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Imported images from /var/lib/rancher/rke2/agent/images/etcd-image.txt in 314.345µs"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Pulling images from /var/lib/rancher/rke2/agent/images/kube-apiserver-image.txt"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Imported images from /var/lib/rancher/rke2/agent/images/kube-apiserver-image.txt in 196.509µs"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Pulling images from /var/lib/rancher/rke2/agent/images/kube-controller-manager-image.txt"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Imported images from /var/lib/rancher/rke2/agent/images/kube-controller-manager-image.txt in 127.584µs"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Pulling images from /var/lib/rancher/rke2/agent/images/kube-proxy-image.txt"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Imported images from /var/lib/rancher/rke2/agent/images/kube-proxy-image.txt in 399.852µs"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Pulling images from /var/lib/rancher/rke2/agent/images/kube-scheduler-image.txt"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Imported images from /var/lib/rancher/rke2/agent/images/kube-scheduler-image.txt in 153.407µs"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Connecting to proxy" url="<wss://127.0.0.1:9345/v1-rke2/connect>"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Running kubelet --address=0.0.0.0 --alsologtostderr=false --anonymous-auth=false --authentication-token-webhook=true --authorizat>
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Handling backend connection request [harvester]"
Jun 23 09:10:16 harvester rke2[1845]: time="2023-06-23T09:10:16Z" level=info msg="Waiting to retrieve kube-proxy configuration; server is not ready: <https://127.0.0.1:9345/v1-rke2/readyz>: 500 Internal Server Err>
Jun 23 09:10:16 harvester rke2[1950]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See <https://kubernetes.io/docs/tasks>>
Jun 23 09:10:16 harvester rke2[1950]: Flag --file-check-frequency has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See <https://kubernetes.io/docs/ta>>
Jun 23 09:10:16 harvester rke2[1950]: Flag --sync-frequency has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See <https://kubernetes.io/docs/tasks/ad>>

aloof-agency-97600

09/04/2023, 7:54 PM

At this point, probably I'll wait for the release of v1.2.0 and reinstall the whole cluster. My experience in the last two version (1.1.1 and 1.1.2) is that Harvester seems to be prone to stuck at "broken state" after reboot/shutdown. I'm uncertain if this is nature of k8s or Harvester though.

👀 1

5 Views

Open in Slack

Previous Next