This message was deleted.
# harvester
a
This message was deleted.
s
We can recreate. We are using hardened Rocky 9.5 with SELinux enabled. Ran across this article linked from container-selinux GitHub Readme - https://danwalsh.livejournal.com/81143.html
t
just use docker? lol
how are you installing podman? cloud-init?
s
We did try a quick attempt installing rootless Docker... we are verifying that was done properly now.
Podman is installed by default in the image.
t
podman in docker?
s
OS image - sorry
Podman is preinstalled in Rocky/RHEL
t
Oh… do you have
container-selinux.noarch
installed?
not on my rocky qcow
s
Yes
Copy code
rpm -q container-selinux
container-selinux-2.232.1-1.el9.noarch
t
just curious if you are getting anything from
journalctl -xeu qemu-guest-agent
?
s
No, we haven't been able to catch anything in the qemu-guest-agent logs. We've looked. But we can see the process is getting restarted.
t
it is odd. the guest agent only reports back out the OS stuff.
it is not harvester. the guest agent is a rocky package.
s
Yeah, so weird. We've tried both RHEL and Rocky 9.5 with and without SELinux enabled. No errors in the qemu-guest-agent logs, but yet it restarts...
We're going to try RHEL 8.10 next
t
that is so weird.
yum udpate
?
what version of harvester?
b
There were some affected versions of qemu-guest-agent that had a bug for this behavior. It's possible it was that and not anything with podman.
w
Do you know right off what version of qemu-guest-agent this was?
b
Let me look it up, but there could have been a regression.
s
Harvester version is 1.4.0 @thousands-advantage-10804
s
Thanks @bland-article-62755 for the information! We'll look more into this.
Backing down to RHEL/Rocky 8.10 worked for us! We tried changing versions of the qemu-guest-agent on 9.5 without success. Looking at Harvester 1.4.x docs technically they state compatibility with 9.4... Hopefully this helps someone else who may find this thread.
t
are you stigging rhel?
s
Yes
t
AH. that maybe causing it. can you share the stig process you are following?
s
Yes-ish - at high-level, OpenSCAP remediation of all DISA CAT 1s, several CAT 2s, and some CAT 3s. FIPS enabled.
Is there anything specific you're looking for?
t
not really. seeing if we can recreate it. I can start with fips. 😄
1
can you deploy 9.5 without the stig?
s
Good ? - we did not try, but I will see if we can.
t
I am running the DISA stig ansible now. FIPS didn’t cause the bouncing
s
Well the interesting part was we only saw the bouncing of the qemu-guest-agent under "load" or certain conditions - specifically when pulling/loading images with Podman or Docker. Perhaps it was any stressful resource load, but we could reliably repeat it that way. Otherwise the VM would run....
t
have you tried double the cpu and memory?
s
Yep
t
ah.. I wonder if it is the seliunx hardeneing.
s
That is what we were settling on
Based on the call out on container-selinux README (even though it's old) we thought maybe that was the issue. 1. https://github.com/containers/container-selinux?tab=readme-ov-file
SELinux, Podman, and Libvirt
Information regarding SELinux blocking Podman container from talking to Libvirt
t
I think the link from dan walsh is not releated.
s
Ok
t
you should be able to manually check the
/etc/systemd/system
unit file for
qemu-guest-agent
for the exec statement. The
ls -aslZ
the file to see what context. Also run it manually to see if it throws an error. My stig’d version does not throw the error.
s
Spinning up STIGed Rocky 9.5 to test
I just checked with colleague, and he said he was seeing on non-STIGed Rocky 9.5 as well (sorry). So, not STIG related at all.
t
huh. I can’t recreate it. The next logical step would be to check the files’ context and then possibly recreate unclass.
s
I'll see if I can come up with simplest use case to reproduce. Thanks!
So, I was able to reproduce it (still on our STIGed image) by just launching the VM and running
podman pull pytorch/pytorch:2.6.0-cuda11.8-cudnn9-devel
It eventually disconnected and qemu agent restarted. IP disappears in Harvester quickly and then comes back.
I have to head out for the weekend, but thanks for all the help.
t
ok. so it works until you run podman? interesting.