Anyone else frequently experiencing ImagePullBackOff errors rancher-users #k3d

Title

melodic-market-42092

10/18/2022, 10:56 AM

Anyone else frequently experiencing ImagePullBackOff errors even though the exact images referred to has been imported into the cluster with

k3d image import ..

? We are seeing this a lot in our CI, and it's difficult to debug

wide-garage-9465

10/18/2022, 10:58 AM

I think the

image import

functionality has a bug of failing silently which has to be fixed. It's not on every run?

melodic-market-42092

10/18/2022, 10:59 AM

No, it's not on every run, but quite often

wide-garage-9465

10/18/2022, 11:03 AM

Are you importing multiple images in a single command? Are you importing from tarball or from image store?

melodic-market-42092

10/18/2022, 11:04 AM

I'm importing 6 images with a single call to

k3d image import ..

. I'm importing docker images that's on the local system. They've just been built with docker build

wide-garage-9465

10/18/2022, 11:06 AM

Alright, thanks for the input 👍

melodic-market-42092

10/18/2022, 11:07 AM

Anything I can do to catch the silent error somehow?

Or to help debug it further?

I found a few existing issues on the repo for this. I'll study those first. Sorry for not checking

Is there anything more than adding

--trace

to the image import command I can do to get more information about what's happening? I have an example of the trace logs now, where the image import seems to go ok, but the pods fail with ImagePullBackoff errors, but I don't think the logs are revealing anything.

flat-glass-90449

10/20/2022, 1:18 PM

Yes, it happened to me also when importing specifically an image with the tag 'latest', I changed the tag to 'test' and it worked, probably anything but 'latest' will work

melodic-market-42092

10/20/2022, 1:19 PM

I walked into that issue previously, causing me to change all tags to "local" instead of "latest". So it's not that issue right now 🙂

flat-glass-90449

10/20/2022, 1:21 PM

I would try and view the logs on the k3d docker worker containers, the ones that act like the hosts

Would also try to view current events using kubectl:

kubectl get events --sort-by='.lastTimestamp'

melodic-market-42092

10/20/2022, 1:31 PM

Thanks! I'll add those two also, so that I may have more info the next time this happens

3 Views

#k3d Join Slack