This message was deleted.

You could enable debug on both nodes and see if you get any additional info on the error. Mentioning the version you're running would also be helpful.

Hi! Thanks for your answer. I enabled debug on both nodes and have a bit more info but not much. I was using 1.29.4 and tried updating to 1.29.15 and now to 1.30.11 to see if there was any difference. As I started from scratch, I did not yet run into the issue again, so we'll see! From what I gathered from the logs, the agent could find the master's peer for Spegel correctly, managed to get the HEAD and GET on the manifest as well as one blob of the image, but subsequent blob fetching failed and the IP mentioned in the logs was no longer that of the master peer but 127.0.0.1 which seems strange? But that could also be normal, I don't really know Spegel's/Containerd's internals

containerd always talks to spegel at the loopback address. spegel in turn will go out to other nodes to fetch content. it acts like a proxy.

I see! But I don't understand why the first blob was ok and the others were not 🤔 If I manually launched a pod on the master node, it was working normally.

hard to know for sure

Thanks for your time anyways! I will report back if I still get strange issues now that I have upgraded k3s