https://rancher.com/ logo
Join the conversationJoin Slack
Channels
academy
amazon
arm
azure
cabpr
chinese
ci-cd
danish
deutsch
developer
elemental
epinio
espanol
events
extensions
fleet
français
gcp
general
harvester
harvester-dev
hobbyfarm
hypper
japanese
k3d
k3os
k3s
k3s-contributor
kim
kubernetes
kubewarden
lima
logging
longhorn-dev
longhorn-storage
masterclass
mesos
mexico
nederlands
neuvector-security
office-hours
one-point-x
onlinemeetup
onlinetraining
opni
os
ozt
phillydotnet
portugues
rancher-desktop
rancher-extensions
rancher-setup
rancher-wrangler
random
rfed_ara
rio
rke
rke2
russian
s3gw
service-mesh
storage
submariner
supermicro-sixsq
swarm
terraform-controller
terraform-provider-rancher2
terraform-provider-rke
theranchcast
training-0110
training-0124
training-0131
training-0207
training-0214
training-1220
ukranian
v16-v21-migration
vsphere
windows
Powered by Linen
rke2
  • s

    stale-painting-80203

    05/17/2022, 2:33 PM
    I am trying to install and run rke2 in a SLES 15 SP3 VM and facing issues. I am using the guide here https://docs.rke2.io/install/quickstart/. I have disabled the firewall and enabled IPv4/IPv6 port forwarding in Wicked. Seems the server is running, but I do see errors in the logs. Also when I installed the agent in the same VM I got an error:
    msg="failed to get CA certs: Get \"<https://127.0.0.1:6444/cacerts>\": read tcp 127.0.0.1:48636->127.0.0.1:6444: read: connection reset by peer
    Curl gives following error:
    curl -v <https://127.0.0.1:6444/cacerts>
    *  Trying 127.0.0.1:6444...
    * TCP_NODELAY set
    * Connected to 127.0.0.1 (127.0.0.1) port 6444 (#0)
    * ALPN, offering h2
    * ALPN, offering http/1.1
    * TLSv1.3 (OUT), TLS handshake, Client hello (1):
    * OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 127.0.0.1:6444 
    * Closing connection 0
    curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to 127.0.0.1:6444
    Anyone know what this might be caused by? curl -k https://127.0.0.1:6443 gives a response.
    rke2-server.service - Rancher Kubernetes Engine v2 (server)
       Loaded: loaded (/etc/systemd/system/rke2-server.service; enabled; vendor preset: disabled)
       Active: active (running) since Mon 2022-05-16 20:46:15 PDT; 4min 55s ago
        Docs: <https://github.com/rancher/rke2#readme>
      Process: 48957 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service (code=exited>
      Process: 48960 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
      Process: 48961 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
      Main PID: 48962 (rke2)
       Tasks: 205
       CGroup: /system.slice/rke2-server.service
           ├─ 2652 /var/lib/rancher/rke2/data/v1.22.9-rke2r2-88ecb1441384/bin/containerd-shim-runc-v2 -namespace k8s.>
           ├─ 2673 /pause
           ├─ 2707 /var/lib/rancher/rke2/data/v1.22.9-rke2r2-88ecb1441384/bin/containerd-shim-runc-v2 -namespace k8s.>
           ├─ 2711 /var/lib/rancher/rke2/data/v1.22.9-rke2r2-88ecb1441384/bin/containerd-shim-runc-v2 -namespace k8s.>
           ├─ 2737 /var/lib/rancher/rke2/data/v1.22.9-rke2r2-88ecb1441384/bin/containerd-shim-runc-v2 -namespace k8s.>
           ├─ 2758 /pause
           ├─ 2778 /pause
           ├─ 2813 /var/lib/rancher/rke2/data/v1.22.9-rke2r2-88ecb1441384/bin/containerd-shim-runc-v2 -namespace k8s.>
           ├─ 2836 /var/lib/rancher/rke2/data/v1.22.9-rke2r2-88ecb1441384/bin/containerd-shim-runc-v2 -namespace k8s.>
           ├─ 2841 /pause
           ├─ 2866 /pause
           ├─ 2882 kube-proxy --cluster-cidr=10.42.0.0/16 --conntrack-max-per-core=0 --conntrack-tcp-timeout-close-wa>
           ├─ 3069 etcd --config-file=/var/lib/rancher/rke2/server/db/etcd/config
           ├─ 3761 /var/lib/rancher/rke2/data/v1.22.9-rke2r2-88ecb1441384/bin/containerd-shim-runc-v2 -namespace k8s.>
           ├─ 3782 /pause
           ├─ 3807 /var/lib/rancher/rke2/data/v1.22.9-rke2r2-88ecb1441384/bin/containerd-shim-runc-v2 -namespace k8s.>
           ├─ 3835 /pause
           ├─ 3863 /var/lib/rancher/rke2/data/v1.22.9-rke2r2-88ecb1441384/bin/containerd-shim-runc-v2 -namespace k8s.>
           ├─ 3890 /var/lib/rancher/rke2/data/v1.22.9-rke2r2-88ecb1441384/bin/containerd-shim-runc-v2 -namespace k8s.>
           ├─ 3920 /var/lib/rancher/rke2/data/v1.22.9-rke2r2-88ecb1441384/bin/containerd-shim-runc-v2 -namespace k8s.>
           ├─ 3940 /pause
           ├─ 3941 /pause
           ├─ 4073 /coredns -conf /etc/coredns/Corefile
           ├─38373 kube-apiserver --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --allow-privileged>
           ├─42523 /cluster-proportional-autoscaler --namespace=kube-system --configmap=rke2-coredns-rke2-coredns-aut>
           ├─43471 /var/lib/rancher/rke2/data/v1.22.9-rke2r2-88ecb1441384/bin/containerd-shim-runc-v2 -namespace k8s.>
           ├─43493 /pause
           ├─43738 /opt/bin/flanneld --ip-masq --kube-subnet-mgr
           ├─45408 /usr/sbin/runsvdir -P /etc/service/enabled
           ├─45734 runsv felix
           ├─45735 runsv monitor-addresses
           ├─45736 runsv allocate-tunnel-addrs
           ├─45737 runsv node-status-reporter
           ├─45738 runsv cni
           ├─45739 calico-node -monitor-addresses
           ├─45740 calico-node -felix
           ├─45741 calico-node -allocate-tunnel-addrs
           ├─45742 calico-node -status-reporter
           ├─45743 calico-node -monitor-token
           ├─46226 /var/lib/rancher/rke2/data/v1.22.9-rke2r2-88ecb1441384/bin/containerd-shim-runc-v2 -namespace k8s.>
           ├─46261 /pause
           ├─48962 /opt/rke2/bin/rke2 server
           ├─48995 containerd -c /var/lib/rancher/rke2/agent/etc/containerd/config.toml -a /run/k3s/containerd/contai>
           ├─49051 kubelet --volume-plugin-dir=/var/lib/kubelet/volumeplugins --file-check-frequency=5s --sync-freque>
           └─49452 kube-scheduler --permit-port-sharing=true --authentication-kubeconfig=/var/lib/rancher/rke2/server>
    
    May 16 20:50:57 suse-vm rke2[48962]: E0516 20:50:57.578889  48962 memcache.go:101] couldn't get resource list for metr>
    May 16 20:51:05 suse-vm rke2[48962]: I0516 20:51:04.992233  48962 trace.go:205] Trace[411526421]: "Reflector ListAndWa>
    May 16 20:51:05 suse-vm rke2[48962]: Trace[411526421]: ---"Objects listed" 61358ms (20:51:03.401)
    May 16 20:51:05 suse-vm rke2[48962]: Trace[411526421]: [1m2.51858886s] [1m2.51858886s] END
    May 16 20:51:08 suse-vm rke2[48962]: time="2022-05-16T20:51:08-07:00" level=info msg="Event(v1.ObjectReference{Kind:\"A>
    May 16 20:51:09 suse-vm rke2[48962]: E0516 20:51:09.360628  48962 memcache.go:196] couldn't get resource list for metr>
    May 16 20:51:09 suse-vm rke2[48962]: E0516 20:51:09.931690  48962 memcache.go:101] couldn't get resource list for metr>
    May 16 20:51:10 suse-vm rke2[48962]: time="2022-05-16T20:51:10-07:00" level=info msg="Starting /v1, Kind=Secret control>
    May 16 20:51:10 suse-vm rke2[48962]: time="2022-05-16T20:51:10-07:00" level=info msg="Starting /v1, Kind=Node controlle>
    May 16 20:51:10 suse-vm rke2[48962]: I0516 20:51:10.690858  48962 leaderelection.go:248] attempting to acquire leader
    c
    n
    • 3
    • 7
  • s

    straight-fireman-71417

    05/17/2022, 5:49 PM
    Is there anyway to perform an RKE2 snapshot and upload it to 2 buckets?
    n
    c
    • 3
    • 8
  • m

    magnificent-vr-88571

    05/18/2022, 10:08 AM
    I am using rke2 v1.21.5 and facing issue in using private registry.
    rke2 --version
    rke2 version v1.21.5+rke2r2 (9e4acdc6018ae74c36523c99af25ab861f3884da)
    go version go1.16.6b7
    I have created registries.yaml with aws ECR as private registry
    mirrors:
      <http://2341234234.dkr.ecr.yyyyy.amazonaws.com|2341234234.dkr.ecr.yyyyy.amazonaws.com>:
        endpoint:
          - "<https://2341234234.dkr.ecr.yyyyy.amazonaws.com>"
    configs:
      "<http://2341234234.dkr.ecr.yyyyy.amazonaws.com|2341234234.dkr.ecr.yyyyy.amazonaws.com>:
        auth:
          username: xxxxx
          password: xxxxx
        tls:
          insecure_skip_verify: true
    While I create a pod to pull registry with following image name in pod.yaml, it fails with image pull error.
    image: <http://2341234234.dkr.ecr.yyyyy.amazonaws.com/image-name:v1|2341234234.dkr.ecr.yyyyy.amazonaws.com/image-name:v1>
    Looking forward some insights
    c
    • 2
    • 32
  • v

    victorious-analyst-3332

    05/18/2022, 2:03 PM
    Howdy all 👋 Is it expected that the
    kube-controller-manager
    metrics on port 10252 are not exposed in RKE2 deployments. We are currently testing
    v1.22.9+rke2r2
    deployed via Rancher
    v2.6.5
    and are seeing the following differences from RKE deployments. Thanks a lot. RKE cluster:
    # netstat -tulpn | grep kube-contro
    tcp6       0      0 :::10252                :::*                    LISTEN      17396/kube-controll
    tcp6       0      0 :::10257                :::*                    LISTEN      17396/kube-controll
    RKE2 cluster:
    # netstat -tulpn | grep kube-contro
    tcp        0      0 127.0.0.1:10257         0.0.0.0:*               LISTEN      39961/kube-controll
    c
    r
    • 3
    • 9
  • g

    gifted-cricket-25537

    05/19/2022, 10:54 AM
    Hi all! I'm probably blind, but I couldn't find the solution to this problem yet. I'm trying to start my cluster with:
    cni: "calico"
    disable-kube-proxy: true
    and since there's no
    kube-proxy
    installed the
    tigera-operator
    can't reach the API
    2022/05/19 07:50:02 [ERROR] Get "<https://10.43.0.1:443/api?timeout=32s>": dial tcp 10.43.0.1:443: i/o timeout
    I know that I could override it using the
    kubernetes-services-endpoint
    CM, although I can't inject that during the installation. Any clues?
    v
    • 2
    • 23
  • f

    faint-airport-83518

    05/19/2022, 11:17 PM
    with rke2 we don't need leader election anymore? https://github.com/rancher/rke2/issues/349. I tried pulling the logic out from https://github.com/rancherfederal/rke2-azure-tf/blob/main/modules/custom_data/files/rke2-init.sh#L51-L71 and am getting an error.
    g
    • 2
    • 12
  • c

    curved-caravan-26314

    05/20/2022, 12:22 AM
    Hello, I'm reading through the rke2 ha documentation. it doesn't talk about the need to add kubectl again for the other server nodes. Is it safe to still install kubectl on the other server nodes?
    c
    • 2
    • 1
  • a

    alert-oxygen-95787

    05/20/2022, 10:40 AM
    Hi, I'm a k3s user - just for local testing, my company uses EKS in production. Do Rancher recommend using k3s or rke2 for local testing? Is work on k3s eventually going to stop in preference to rke2? TIA
    👀 1
    s
    c
    r
    • 4
    • 4
  • c

    curved-caravan-26314

    05/20/2022, 12:05 PM
    Morning, I am looking to go from using ubuntu to Suse Leap 15.3. For the hardware I have, the legacy install of ubuntu is what I use. What is the equivalent to install suse leap?
    c
    • 2
    • 1
  • s

    shy-zebra-53074

    05/20/2022, 10:39 PM
    Happy Friday! I’m performing an analysis against RKE2 using the DISA STIGs and I came across this one:
    Rule Title: The Kubernetes Controller Manager must use TLS 1.2, at a minimum, to protect the confidentiality of sensitive data during electronic dissemination.
    
    Discussion: The Kubernetes Controller Manager will prohibit the use of SSL and unauthorized versions of TLS protocols to properly secure communication.
    
    The use of unsupported protocol exposes vulnerabilities to the Kubernetes by rogue traffic interceptions, man-in-the-middle attacks, and impersonation of users or services from the container platform runtime, registry, and key store. To enable the minimum version of TLS to be used by the Kubernetes Controller Manager, the setting "tls-min-version" must be set.
    
    Check Text: Change to the /etc/kubernetes/manifests/ directory on the Kubernetes Master Node. Run the command:
    
    grep -i tls-min-version * 
    
    If the setting "tls-min-version" is not configured in the Kubernetes Controller Manager manifest file or it is set to "VersionTLS10" or "VersionTLS11", this is a finding.
    
    Fix Text: Edit the Kubernetes Controller Manager manifest file in the /etc/kubernetes/manifests directory on the Kubernetes Master Node. Set the value of "--tls-min-version" to "VersionTLS12" or higher.
    However I don’t see the
    tls-min-version
    flag set for the
    kube-controller-manager
    service:
    kube-controller-manager --flex-volume-plugin-dir=/var/lib/kubelet/volumeplugins --terminated-pod-gc-threshold=1000 --permit-port-sharing=true --cloud-provider=aws --cloud-config= --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/rke2/server/cred/controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/rke2/server/cred/controller.kubeconfig --bind-address=127.0.0.1 --cluster-cidr=10.0.0.0/12 --cluster-signing-kube-apiserver-client-cert-file=/var/lib/rancher/rke2/server/tls/client-ca.crt --cluster-signing-kube-apiserver-client-key-file=/var/lib/rancher/rke2/server/tls/client-ca.key --cluster-signing-kubelet-client-cert-file=/var/lib/rancher/rke2/server/tls/client-ca.crt --cluster-signing-kubelet-client-key-file=/var/lib/rancher/rke2/server/tls/client-ca.key --cluster-signing-kubelet-serving-cert-file=/var/lib/rancher/rke2/server/tls/server-ca.crt --cluster-signing-kubelet-serving-key-file=/var/lib/rancher/rke2/server/tls/server-ca.key --cluster-signing-legacy-unknown-cert-file=/var/lib/rancher/rke2/server/tls/client-ca.crt --cluster-signing-legacy-unknown-key-file=/var/lib/rancher/rke2/server/tls/client-ca.key --feature-gates=JobTrackingWithFinalizers=true --kubeconfig=/var/lib/rancher/rke2/server/cred/controller.kubeconfig --profiling=false --root-ca-file=/var/lib/rancher/rke2/server/tls/server-ca.crt --secure-port=10257 --service-account-private-key-file=/var/lib/rancher/rke2/server/tls/service.key --use-service-account-credentials=true
    c
    j
    • 3
    • 26
  • t

    thankful-plastic-94519

    05/22/2022, 1:08 AM
    I've searched and found a general "no" to this question, but I wonder more specifically about the reasoning / why. I have a busy cluster with high churn rate, my current issues stem from IO latency in the etcd space / control plane. Because rke2 only supports colocated etcd & control plane components, this seems like a pretty severe shortcoming. Is there any advise towards addressing this type of issue with rke2?
    c
    • 2
    • 2
  • m

    magnificent-vr-88571

    05/23/2022, 12:49 PM
    @creamy-pencil-82913 @gray-lawyer-73831 In reference to https://rancher-users.slack.com/archives/C01PHNP149L/p1652868532770339 Currently facing below error in pod events after setting up private registry with endpoint.
    Events:
      Type     Reason     Age                     From               Message
      ----     ------     ----                    ----               -------
      Normal   Scheduled  21s                     default-scheduler  Successfully assigned default/ecr-pod to sv11
      Normal   BackOff    8s                      kubelet            Back-off pulling image "private-registry/utils-img:latest"
      Warning  Failed     8s                      kubelet            Error: ImagePullBackOff
      Normal   Pulling    <invalid> (x2 over 8s)  kubelet            Pulling image "private-registry/utils-img:latest"
      Warning  Failed     <invalid> (x2 over 8s)  kubelet            Failed to pull image "private-registry/utils-img:latest": rpc error: code = NotFound desc = failed to pull and unpack image "private-registry/utils-img:latest": failed to unpack image on snapshotter overlayfs: unexpected media type text/html for sha256:6066282cb389d5bee17ec9f08335850ece266457a557511d3f5e78eabc34df21: not found  
    Warning  Failed     <invalid> (x2 over 8s)  kubelet            Error: ErrImagePull
    I am trying to add rewrite as mentioned https://github.com/k3s-io/k3s/issues/5502#issuecomment-1109107238 to overcome this issue. After adding rewrite
    /etc/rancher/rke2/registries.yaml
    its not reflected in the environment, would like to know rewrite in
    /var/lib/rancher/rke2/agent/etc/containerd/config.toml
    to overcome this issue. correct me if am wrong.
    g
    c
    • 3
    • 16
  • a

    ambitious-butcher-83703

    05/23/2022, 11:39 PM
    Hello — Using RKE2 in a multinode air-gapped setup. If I have some custom containers I want to able to run in the cluster, do I need to have the tar files in /var/lib/rancher/rke2/agent/images/ in each of the nodes? or just on the master. Will the pods only be able to pull the image if it exists on the node its trying to run on? Thanks in advance!
    a
    f
    • 3
    • 7
  • b

    billions-kite-9416

    05/25/2022, 10:34 AM
    Hi everyone, I am having some trouble creating a custom rke2 cluster from Rancher UI and the error message is not really helpful. The rancher-system-agent binary gets downloaded to the nodes but there is an error saying
    run.sh
    is not found
    May 25 13:18:53  rancher-system-agent[4497]: time="2022-05-25T13:18:53+03:00" level=info msg="Pulling image <http://index.docker.io/rancher/system-a|index.docker.io/rancher/system-a>
    gent-installer-rke2:v1.23.6-rke2r2"
    May 25 13:18:55  rancher-system-agent[4497]: time="2022-05-25T13:18:55+03:00" level=info msg="[Applyinator] Running command: sh [-c run.sh]"
    May 25 13:18:55  rancher-system-agent[4497]: time="2022-05-25T13:18:55+03:00" level=info msg="[48304be41fe2284adc2b5937c9d5c9c7b09009b2cae77
    6320036286c2f0c3191_0:stderr]: sh: 1: run.sh: not found"
    May 25 13:18:55  rancher-system-agent[4497]: time="2022-05-25T13:18:55+03:00" level=info msg="[Applyinator] Command sh [-c run.sh] finished
    with err: <nil> and exit code: 127"
    May 25 13:18:55  rancher-system-agent[4497]: time="2022-05-25T13:18:55+03:00" level=error msg="error executing instruction 0: <nil>"
    May 25 13:18:55  rancher-system-agent[4497]: time="2022-05-25T13:18:55+03:00" level=error msg="error encountered during parsing of last run
    time: parsing time \"\" as \"Mon Jan _2 15:04:05 MST 2006\": cannot parse \"\" as \"Mon\""
    May 25 13:18:55  rancher-system-agent[4497]: time="2022-05-25T13:18:55+03:00" level=info msg="[Applyinator] No image provided, creating empt
    y working directory /var/lib/rancher/agent/work/20220525-131853/48304be41fe2284adc2b5937c9d5c9c7b09009b2cae776320036286c2f0c3191_0"
    May 25 13:18:55  rancher-system-agent[4497]: time="2022-05-25T13:18:55+03:00" level=info msg="[Applyinator] Running command: sh [-c rke2 etc
    d-snapshot list --etcd-s3=false 2>/dev/null]"
    May 25 13:18:55  rancher-system-agent[4497]: time="2022-05-25T13:18:55+03:00" level=info msg="[Applyinator] Command sh [-c rke2 etcd-snapsho
    t list --etcd-s3=false 2>/dev/null] finished with err: <nil> and exit code: 127"
    May 25 13:18:56  rancher-system-agent[4497]: time="2022-05-25T13:18:56+03:00" level=error msg="error loading x509 client cert/key for probe
    kube-apiserver (/var/lib/rancher/rke2/server/tls/client-kube-apiserver.crt//var/lib/rancher/rke2/server/tls/client-kube-apiserver.key): open /var/lib/
    rancher/rke2/server/tls/client-kube-apiserver.crt: no such file or directory"
    👀 1
    g
    f
    • 3
    • 7
  • f

    future-monitor-61871

    05/25/2022, 7:39 PM
    Is there a fully populated config.yaml out there somewhere that I can look at? Or is there an option to produce a blank one from the rke2 executable? I've dug on github but haven't found such an artifact.
    c
    • 2
    • 3
  • h

    hundreds-evening-84071

    05/26/2022, 12:50 PM
    When creating RKE2 cluster on Linux, I have /var/lib/rancher separate partition So, wondering what directory is used when a windows worker node is added for RKE2? I also want to create a separate partition there.
    • 1
    • 1
  • s

    shy-zebra-53074

    05/26/2022, 4:35 PM
    Hey all! I’ve been able to set custom configs via:
    kube-apiserver-arg
    kube-controller-manager-arg
    and
    kube-scheduler-arg
    However, I would like to ensure that a certain config is NOT set that is currently being set by default. For example I would like to unset the
    --hostname-override
    flag for the
    kubelet
    service and make sure that flag is not passed when starting the kubelet service
    c
    • 2
    • 11
  • b

    bored-rain-98291

    05/27/2022, 6:18 PM
    Greetings: I am configuring a cluster using RKE2. I understand that nodes are for the control plane and that agents are for running application workloads. Is there any best-practice advice on mixing/matching on the same node? also what would the memory requirements be? I have only 3 nodes which have blocks of memory that i can adjust. Currently they have 4gb and this is for a dev cluster. Interested in the optimal setup for this small size? 1 node/2 agents, 2 nodes - 1 agent? thanks for any insight!
    c
    • 2
    • 5
  • g

    great-photographer-94826

    05/30/2022, 8:24 AM
    Hi #rke2! I want to create a high-availability RKE2 cluster where it is necessary to distribute nodes at two sites (e.g. EU-A and EU-B geographically distributed sites). It is recommended to have an odd number of masters in a cluster. But it is necessary for each sites to be failure tolerance. Is it good idea when I place 3 master nodes each sites (EU-A 3 master nodes, EU-B 3 master nodes)? Then fault tolerance is ensured both within and between sites. Am I wrong? I don't want to build multi-cluster RKE2 architecture because of Longhorn.
    f
    • 2
    • 3
  • b

    billowy-animal-38808

    05/31/2022, 2:39 PM
    Hello everyone. Does Rke2 allow the creation of LoadBalancer services by default? I have a rke2 Cluster and I created a LoadBalancer service but when listing the services it is in a
    pending
    state. Can someone explain to me what to do to enable this option?
    f
    f
    +3
    • 6
    • 43
  • s

    sparse-businessperson-74827

    06/01/2022, 1:28 PM
    Hi, I am having problem with one of the nodes joining rke2 cluster running Rancher 2.6.2 The server has died and was rebuilt. The references to old server were removed from UI. Now when I try to add the node(same IP and name as before) I am getting this:
    Jun  1 15:20:33 node-1 rke2[35837]: time="2022-06-01T15:20:33+02:00" level=error msg="Failed to connect to proxy" error="unexpected EOF"
    Jun  1 15:20:33 node-1 rke2[35837]: time="2022-06-01T15:20:33+02:00" level=error msg="Remotedialer proxy error" error="unexpected EOF"
    • 1
    • 1
  • b

    bored-rain-98291

    06/02/2022, 1:14 PM
    Greetings friends. I am trying to install RKE2 on 3 centos8 servers. When i configure the first node with the config.yaml in /etc/rancher/config.yaml it causes the node to crash and not start clean. Ive tried to get the system to recognize the other server nodes but simply cant get it to work. I can get it to work with a single node. Any help would be greatly appreciated.
    s
    • 2
    • 36
  • n

    narrow-noon-75604

    06/02/2022, 3:47 PM
    Hi, We are trying to deploy the RKE2 on a centos7 cluster...containing 1 server node and 3 agent nodes...The complete installation was smooth and able to get all the nodes in "Ready" state. But the issue is that the pods under "calico-system" namespace are going into "CrashLoopBackOff" with "Authorization error" as below,
    $ kubectl logs -f pod/calico-node-r2wj8 -n calico-system
    Error from server (InternalError): Internal error occurred: Authorization error (user=kube-apiserver, verb=get, resource=nodes, subresource=proxy)
    Created a new clusterrolebinding with "system:kubelet-api-admin" role but that did not help. Please help us with the steps to fix this issue and let me know if you need any more details
    c
    • 2
    • 2
  • s

    salmon-dress-41836

    06/07/2022, 5:38 AM
    There are server flags provided to modify some of the items in the generated static pod manifests but is there any way to modify anything outside of that? E.g. liveness probes
    c
    • 2
    • 7
  • n

    narrow-noon-75604

    06/07/2022, 2:28 PM
    Hi, I want to deploy rke2 with dual stack but could not able to find any references for config.yaml. can anyone please share me any reference links of config.yaml for dual stack configuration using calico network. This is the current configuration I am using,
    write-kubeconfig-mode: "0644"
    tls-san:
      - "<http://master.167.254.204.58.nip.io|master.167.254.204.58.nip.io>"
    node-label:
      - "nodetype=master"
    cluster-cidr: "10.42.0.0/16"
    service-cidr: "10.43.0.0/16"
    cluster-dns: "10.43.0.10"
    cluster-domain: "<http://master.xxx.xxx.xxx.xxx.nip.io|master.xxx.xxx.xxx.xxx.nip.io>"
    cni:
      - calico
    disable:
      - rke2-canal
      - rke2-kube-proxy
    c
    b
    • 3
    • 15
  • c

    curved-caravan-26314

    06/07/2022, 4:30 PM
    Is OEL 7.9 compatible with rke2
    n
    • 2
    • 1
  • h

    hundreds-hairdresser-46043

    06/08/2022, 9:22 AM
    Hi People. So firewallD has issues with the CNI plugins as per this: https://docs.rke2.io/known_issues/#firewalld-conflicts-with-default-networking my question would be then - what options do I have to enable a firewall on the servers themselves ? so far we just left the firewalls off - but at some point we would need to enable them - suggestions?
    r
    • 2
    • 6
  • h

    hundreds-evening-84071

    06/08/2022, 5:32 PM
    hey guys, running rancher 2.6.5 stable release... created RKE2 cluster and also added a windows worker node... however, I see by default, it uses
    c:\var\lib\rancher
    as directory... is there an option - or a way I can specify something other than c-drive ? perhaps like
    d:\var\lib\rancher
    ? reason I want to do this is to keep users-stuff off the c-drive to prevent c-drive from filling up and crashing the OS.
    c
    • 2
    • 10
  • n

    narrow-noon-75604

    06/08/2022, 7:18 PM
    Hi, I am trying to install RKE2 v1.23.6 on a RHEL8 cluster (1server & 4agents) using the Airgap tarball method. The installation is successful, the nodes are added properly and into Ready state.
    kubectl get nodes
    NAME           STATUS                     ROLES                       AGE    VERSION
    rke2-master    Ready,SchedulingDisabled   control-plane,etcd,master   112m   v1.23.6+rke2r2
    rke2-worker1   Ready                      <none>                      108m   v1.23.6+rke2r2
    rke2-worker2   Ready                      <none>                      108m   v1.23.6+rke2r2
    rke2-worker3   Ready                      <none>                      108m   v1.23.6+rke2r2
    All the pods under "kube-system" & "calico-system" namespace are up & Running properly.
    kubectl get po -n calico-system
    NAME                                       READY   STATUS    RESTARTS   AGE
    calico-kube-controllers-69cb5d9c7b-xllwx   1/1     Running   0          113m
    calico-node-86pcd                          1/1     Running   0          110m
    calico-node-dxtjk                          1/1     Running   0          110m
    calico-node-lb8n7                          1/1     Running   0          113m
    calico-node-ld8mf                          1/1     Running   0          110m
    calico-typha-86c8b747c4-rqfhk              1/1     Running   0          113m
    calico-typha-86c8b747c4-xs7zs              1/1     Running   0          110m
    kubectl get po -n kube-system
    NAME                                                    READY   STATUS      RESTARTS   AGE
    cloud-controller-manager-rke2-master                    1/1     Running     0          114m
    etcd-rke2-master                                        1/1     Running     0          114m
    helm-install-rke2-calico-crd-j44qb                      0/1     Completed   0          114m
    helm-install-rke2-calico-hv4xg                          0/1     Completed   1          114m
    helm-install-rke2-coredns-btsgx                         0/1     Completed   0          114m
    helm-install-rke2-ingress-nginx-c99sw                   0/1     Completed   0          114m
    helm-install-rke2-metrics-server-j5gjq                  0/1     Completed   0          114m
    kube-apiserver-rke2-master                              1/1     Running     0          113m
    kube-controller-manager-rke2-master                     1/1     Running     0          114m
    kube-proxy-rke2-master                                  1/1     Running     0          114m
    kube-proxy-rke2-worker1                                 1/1     Running     0          110m
    kube-proxy-rke2-worker2                                 1/1     Running     0          110m
    kube-proxy-rke2-worker3                                 1/1     Running     0          110m
    kube-scheduler-rke2-master                              1/1     Running     0          114m
    rke2-coredns-rke2-coredns-69c8f974c-gvwqg               1/1     Running     0          7m56s
    rke2-coredns-rke2-coredns-69c8f974c-qmq8m               1/1     Running     0          7m56s
    rke2-coredns-rke2-coredns-autoscaler-65c9bb465d-4g4sw   1/1     Running     0          114m
    rke2-ingress-nginx-controller-2bhbc                     1/1     Running     0          113m
    rke2-ingress-nginx-controller-gnpk5                     1/1     Running     0          110m
    rke2-ingress-nginx-controller-znh8z                     1/1     Running     0          110m
    rke2-ingress-nginx-controller-ztpjf                     1/1     Running     0          110m
    rke2-metrics-server-6564db4569-m5kcc                    1/1     Running     0          113m
    I have deployed kafka but the pods are going to "CrashLoopBackOff" with a dns error,
    java.net.UnknownHostException: zookeeper.msgbus.svc: Temporary failure in name resolution
    The logs of coredns pods are throwing a lot of errors as,
    [ERROR] plugin/errors: 2 4686730224998678655.5699833978585684595. HINFO: read udp xx.xx.xx.xx:51070->yy.yy.yy.yy:53: read: no route to host
    [ERROR] plugin/errors: 2 4686730224998678655.5699833978585684595. HINFO: read udp xx.xx.xx.xx:42332->yy.yy.yy.yy:53: read: no route to host
    [ERROR] plugin/errors: 2 4686730224998678655.5699833978585684595. HINFO: read udp xx.xx.xx.xx:57092->yy.yy.yy.yy:53: read: no route to host
    I have opened port 53 to allow both tcp and udp traffic and am not about the cause of this issue. Any suggestions would be appreciated and let me know if you need any more debug logs
    c
    h
    • 3
    • 14
  • b

    brief-mouse-13981

    06/09/2022, 5:59 AM
    Hi all. I loved rke2 for a every long time and know i am planning to implement it on a customers site. But the customer thinks somehow that the vanilla Kubernetes has more of a stable development team (because of the fact that it is vanilla). Can anybody help me with reasonable arguments so i can convince the customer to choose for rke2 ?
    c
    r
    c
    • 4
    • 6
Powered by Linen
Title
b

brief-mouse-13981

06/09/2022, 5:59 AM
Hi all. I loved rke2 for a every long time and know i am planning to implement it on a customers site. But the customer thinks somehow that the vanilla Kubernetes has more of a stable development team (because of the fact that it is vanilla). Can anybody help me with reasonable arguments so i can convince the customer to choose for rke2 ?
c

creamy-pencil-82913

06/09/2022, 6:32 AM
If you reach out to SUSE sales they should be able to help you with that. They have some good material. Unless you're planning on going without support, I guess :)
b

brief-mouse-13981

06/09/2022, 6:42 AM
jup, support isn´t something the customer is willing to pay for. I will try to scrape some material from the sales website from suse. Thanks
r

rapid-helmet-86074

06/09/2022, 1:51 PM
Over a decade ago, I installed a bunch of Linux VMs for various projects that I was maintaining but was told I'd be ramped down. At that time the three Linux distros with support were RHEL, SUSE, & Ubuntu. RHEL & SUSE required you install the support-bearing distro immediately or you'd never have support, but Ubuntu allowed same install and people could just call up for a year of support later if desired. That's why I chose Ubuntu for that customer letting them know that they had that option if they wanted/needed it later after I'd moved on. I didn't ask their sales folks if an unsupported cluster could have support turned on, but I did ask if I wanted to start out getting support the first year for both prod & dev clusters and then drop dev support after the first year and it was up and running and the sales guy said that was pretty common. I also appreciated with Rancher versus, say OpenShift, that the free version of the software can be installed without a license cost (and thus without vendor support) at development teams' shops or even more often in small clusters to prototype a feature (I currently have four small segmented cluster for different Rancher installs for some testing, $0 license fees). Another argument for a "Kubernetes distribution" such as RKE2 with Rancher, over vanilla Kubernetes is that you get a lot of bundled useful support tools that you'd otherwise have to install piece by piece. If the distribution mismatches the desired toolset, then it's not a help, but it can be worth it to find such things first. Next argument, and it's a balance, is that a distribution like Rancher & RKE2 will have integration testing for the various bundled parts. This will lead to sometimes those parts not being the very latest versions (even usually), but it also usually means less problems with them that have to be solved on the user's cluster than when you have vanilla weaving everything together. So it's a bit of a pro and a con. Final thing that RKE2 specifically gives you is a more locked down default profile than you'll get from vanilla Kubernetes. At the same time the vanilla Kubernetes docs told me I had to disable SELinux I could, and eventually did, install RKE2 with SELinux enforcing. So it's saving some work on lockdowns and security compliance checklists for you as well.
b

brief-mouse-13981

06/09/2022, 2:23 PM
@rapid-helmet-86074 thank you for your answer. I think i will just install RKE2 with these arguments. That should suffice.
c

careful-table-97595

06/23/2022, 1:48 AM
@brief-mouse-13981 And if you want something really nice and near maintenance-free, you can couple rke2 with flatcar OS (https://flatcar-linux.org/) to get an immutable OS as CoreOS is for Openshift, then add the flatcar upgrade operator (https://github.com/flatcar-linux/flatcar-linux-update-operator) to automatically maintain your OS up-to-date from a kubernetes-aware mecanism, and also the rancher system-upgrade-controller to upgrade rke2 as a workload (https://github.com/rancher/system-upgrade-controller, also mentioned here (https://docs.rke2.io/upgrade/automated_upgrade/) Nothing looking like a kubernetes cluster for developers only! 🙂
That said, I'm still working on making automated rke2 upgrades work. The rest, however, is a breeze!
View count: 25