https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# k3s
a
This message was deleted.
h
what is the label for /var/lib/rancher directory?
r
Copy code
dev-k8re-01:/var/lib # ls -lZ | grep rancher
drwxr-xr-x. 1 root           root           unconfined_u:object_r:var_lib_t:s0                6 Aug 25 12:14 rancher
h
that is what I have (running rke2 not k3s) for /var/lib/rancher directory and ask long as the label is correct (before installation) at that directories under that should get correct label was this label set before you installed k3s?
r
I believe so. I can verify in a moment. There’s very little that is configured on this system before k3s is installed. This system is set up in the same way as a bunch of other systems we have and it seems that most of the time it works, however occasionally (such as this one) it breaks, and the only difference that I can see is that 
storage
has the 
container_file_t
context on the systems that run fine, and 
k3s_data_t
on the systems that fail.
h
what version is your k3s deployment?
r
1.32.7+k3s1 running on openSUSE Leap Micro 6.1 I’d check on 1.33 but we had a broken dependency with that version and I haven’t had time to check if it’s been resolved
Okay so I went back through my installation script, and I’m actually manually running 
chcon system_u:object_r:container_file_t:s0 /var/lib/rancher/k3s/data/storage/
immediately after k3s is installed because I guess I ran into this issue before and forgot. I disabled that and let the default stay: 
system_u:object_r:k3s_data_t:s0
After rebooting, the local provisioner helper pods start failing again due to the OP selinux issue.
So it seems on LEAP Micro 6.1, k3s+selinux is broken OOTB. I’m going to attempt to reproduce on completely fresh system without any other modifications.
Alright, small script that reproduces the issue on a fresh LEAP MicroOS 6.1 system:
Copy code
#!/usr/bin/env bash

mount /var

curl -fsSL -o /usr/bin/yq <https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64>
chmod +x /usr/bin/yq

export INSTALL_K3S_BIN_DIR=/usr/bin
export INSTALL_K3s_SKIP_START=true

curl -fsSL <https://get.k3s.io> |
  sed 's#k3s >/.*#k3s; then#' | \
  sh -s - server \
    --disable=traefik \
    --disable=servicelb \
    --disable=local-storage \
    --node-name=pvctest \
    --selinux

kdir=/var/lib/rancher/k3s
k3sm=${kdir}/server/manifests
lsim=${k3sm}/local-storage-im.yaml

mkdir -p ${k3sm}

curl -fsSL -o ${lsim} <https://raw.githubusercontent.com/k3s-io/k3s/refs/heads/master/manifests/local-storage.yaml>
yq -i '. |= with(select(.kind == "StorageClass"); .volumeBindingMode = "Immediate" )' ${lsim}
sed -i "s/%{SYSTEM_DEFAULT_REGISTRY}%//" ${lsim}
sed -i "s#%{DEFAULT_LOCAL_STORAGE_PATH}%#${kdir}/data/storage#" ${lsim}

cat <<EOF > ${k3sm}/pvctest.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    <http://volume.kubernetes.io/selected-node|volume.kubernetes.io/selected-node>: pvctest
  name: pvctest
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  storageClassName: local-path
  volumeMode: Filesystem
EOF
After setting up the system copy paste to 
script.sh
, make it executable, then run 
transactional-update run ./script.sh
and reboot. After the system comes back up you can see that the local-provisioner helper pod is failing to create the volume.
a
Copy code
# Rule 3: Allow the container runtime to manage its storage directory.
# This grants the necessary permissions for the local-path-provisioner
# to create, modify, and delete directories and files within its storage path.

allow container_runtime_t var_lib_t:dir { add_name create getattr setattr search write remove_name };
allow container_runtime_t var_lib_t:file { create setattr unlink write };
Saw one of my clusters running this custom policy to fix local path provisioner errors 🙂
36 Views
Open in Slack
PreviousNext
Powered by