https://rancher.com/ logo
Join Slack
Powered by
This message was deleted.
# k3d
a
This message was deleted.
r
So I poked around and found some failures in audit log. Test vm has a normal user named test and with test I ran 
sudo wc /var/log/audit/audit.log ; k3d ; sudo cp /var/log/audit/audit.log ./k3d-fail-audit.log
then as root I ran 
wc /var/log/audit/audit.log ; k3d ; cp /var/log/audit/audit.log ./k3d-root-audit.log
then used the wc values to cut out all the early bits and get to comparisons. I found that the failing system call states that an execve is failing but I'm not quite clear why. I put the text of the audit log success & failures in the conversation but didn't want to clutter the main window with it.
failed (k3d-fail-audit.log) entries:
Copy code
node=localhost.localdomain type=USER_START msg=audit(1694484868.882:467): pid=1909 uid=0 auid=1000 ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_open grantors=pam_keyinit,pam_limits,pam_systemd,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'UID="root" AUID="test"
node=localhost.localdomain type=USER_END msg=audit(1694484868.941:468): pid=1909 uid=0 auid=1000 ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:session_close grantors=pam_keyinit,pam_limits,pam_systemd,pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'UID="root" AUID="test"
node=localhost.localdomain type=CRED_DISP msg=audit(1694484868.941:469): pid=1909 uid=0 auid=1000 ses=1 subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 msg='op=PAM:setcred grantors=pam_unix acct="root" exe="/usr/bin/sudo" hostname=? addr=? terminal=/dev/pts/0 res=success'UID="root" AUID="test"
node=localhost.localdomain type=FANOTIFY msg=audit(1694484868.942:470): resp=2
node=localhost.localdomain type=SYSCALL msg=audit(1694484868.942:470): arch=c000003e syscall=59 success=no exit=-1 a0=563e980e5cf0 a1=563e980e6650 a2=563e980b9330 a3=8 items=1 ppid=1872 pid=1914 auid=1000 uid=1000 gid=1000 euid=1000 suid=1000 fsuid=1000 egid=1000 sgid=1000 fsgid=1000 tty=pts0 ses=1 comm="bash" exe="/usr/bin/bash" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)ARCH=x86_64 SYSCALL=execve AUID="test" UID="test" GID="test" EUID="test" SUID="test" FSUID="test" EGID="test" SGID="test" FSGID="test"
node=localhost.localdomain type=CWD msg=audit(1694484868.942:470): cwd="/home/test"
node=localhost.localdomain type=PATH msg=audit(1694484868.942:470): item=0 name="/usr/local/bin/k3d" inode=404914 dev=fd:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=unconfined_u:object_r:bin_t:s0 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0OUID="root" OGID="root"
node=localhost.localdomain type=PROCTITLE msg=audit(1694484868.942:470): proctitle="-bash"
node=localhost.localdomain type=SYSCALL msg=audit(1694484868.944:471): arch=c000003e syscall=59 success=yes exit=0 a0=563e980e6550 a1=563e980e6880 a2=563e980b9330 a3=8 items=2 ppid=1872 pid=1915 auid=1000 uid=1000 gid=1000 euid=0 suid=0 fsuid=0 egid=1000 sgid=1000 fsgid=1000 tty=pts0 ses=1 comm="sudo" exe="/usr/bin/sudo" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="privileged"ARCH=x86_64 SYSCALL=execve AUID="test" UID="test" GID="test" EUID="root" SUID="root" FSUID="root" EGID="test" SGID="test" FSGID="test"
node=localhost.localdomain type=EXECVE msg=audit(1694484868.944:471): argc=4 a0="sudo" a1="cp" a2="/var/log/audit/audit.log" a3="./k3d-fail-audit.log"
successful (k3d-root-audit.log) entries:
Copy code
node=localhost.localdomain type=PROCTITLE msg=audit(1694484989.242:557): proctitle=726D002D69002E2F6B33642D726F6F742D61756469742E6C6F67
node=localhost.localdomain type=SYSCALL msg=audit(1694485014.171:558): arch=c000003e syscall=59 success=yes exit=0 a0=55b102048fb0 a1=55b102048f80 a2=55b10201c3e0 a3=8 items=2 ppid=1933 pid=1987 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="sudo" exe="/usr/bin/sudo" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="privileged"ARCH=x86_64 SYSCALL=execve AUID="test" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"
node=localhost.localdomain type=EXECVE msg=audit(1694485014.171:558): argc=4 a0="sudo" a1="cp" a2="/var/log/audit/audit.log" a3="./k3d-root-audit.log"
So I think the actual relevant bits are: fail:
Copy code
node=localhost.localdomain type=SYSCALL msg=audit(1694484868.942:470): arch=c000003e syscall=59 success=no exit=-1 a0=563e980e5cf0 a1=563e980e6650 a2=563e980b9330 a3=8 items=1 ppid=1872 pid=1914 auid=1000 uid=1000 gid=1000 euid=1000 suid=1000 fsuid=1000 egid=1000 sgid=1000 fsgid=1000 tty=pts0 ses=1 comm="bash" exe="/usr/bin/bash" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key=(null)ARCH=x86_64 SYSCALL=execve AUID="test" UID="test" GID="test" EUID="test" SUID="test" FSUID="test" EGID="test" SGID="test" FSGID="test"
node=localhost.localdomain type=CWD msg=audit(1694484868.942:470): cwd="/home/test"
node=localhost.localdomain type=PATH msg=audit(1694484868.942:470): item=0 name="/usr/local/bin/k3d" inode=404914 dev=fd:00 mode=0100755 ouid=0 ogid=0 rdev=00:00 obj=unconfined_u:object_r:bin_t:s0 nametype=NORMAL cap_fp=0 cap_fi=0 cap_fe=0 cap_fver=0 cap_frootid=0OUID="root" OGID="root"
node=localhost.localdomain type=PROCTITLE msg=audit(1694484868.942:470): proctitle="-bash"
vs success:
Copy code
node=localhost.localdomain type=SYSCALL msg=audit(1694485014.171:558): arch=c000003e syscall=59 success=yes exit=0 a0=55b102048fb0 a1=55b102048f80 a2=55b10201c3e0 a3=8 items=2 ppid=1933 pid=1987 auid=1000 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="sudo" exe="/usr/bin/sudo" subj=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 key="privileged"ARCH=x86_64 SYSCALL=execve AUID="test" UID="root" GID="root" EUID="root" SUID="root" FSUID="root" EGID="root" SGID="root" FSGID="root"
Any assistance would be appreciated.
w
Sorry, I have no idea 🤷‍♂️
r
Thanks anyway. It's a bit of a low priority task so I only get back to it periodically. I'll try and make sure to update post when/if I find a solution.
I haven't tracked down the fix, but I'm nearly certain the issue is fapolicyd and it's just rejecting any executable it doesn't trust. So completely unrelated to k3d and all part of Rocky 8 with DISA STIG security policy appliled.
w
Well, I'm glad that it isn't k3d's fault, but this is still pretty restrictive 😬
r
Yep, and all that gets logged is in the audit log stating that system call execv failed, which was noticeable without the log message. The thing I eventually tried that pointed me there is I verified I could run /usr/bin/base64 and then copied it to /usr/local/bin and found that I was getting the same failure now for /usr/local/bin/base64, which immediately led me away from k3d having anything to do with it and it being just an 'unknown executable block'-sort of policy somewhere.
2 Views
Open in Slack
PreviousNext
Powered by