Channels
academy
amazon
arm
azure
cabpr
chinese
ci-cd
danish
deutsch
developer
elemental
epinio
espanol
events
extensions
fleet
français
gcp
general
harvester
harvester-dev
hobbyfarm
hypper
japanese
k3d
k3os
k3s
k3s-contributor
kim
kubernetes
kubewarden
lima
logging
longhorn-dev
longhorn-storage
masterclass
mesos
mexico
nederlands
neuvector-security
office-hours
one-point-x
onlinemeetup
onlinetraining
opni
os
ozt
phillydotnet
portugues
rancher-desktop
rancher-extensions
rancher-setup
rancher-wrangler
random
rfed_ara
rio
rke
rke2
russian
s3gw
service-mesh
storage
submariner
supermicro-sixsq
swarm
terraform-controller
terraform-provider-rancher2
terraform-provider-rke
theranchcast
training-0110
training-0124
training-0131
training-0207
training-0214
training-1220
ukranian
v16-v21-migration
vsphere
windows
Title
s
sticky-megabyte-50644
08/04/2022, 5:44 PMđź‘‹ Hello, team!
I'm facing issues with RKE2 on air-gapped environment. "Failed to create sandbox", because its unable to pull rancher/pause:3.6.
I'm pretty sure this image was part of the original RKE2 deployment
c
creamy-pencil-82913
08/04/2022, 6:08 PMthat image is included in the airgap tarballs. Did you download them and place them in the agent/images directory on your nodes, or import them into your private registry mirror?
s
sticky-megabyte-50644
08/04/2022, 6:09 PMno - I used the official TF module which sets up an AWS launch configuration, then I closed off the airgap.
I'll try putting the tarball (uncompressed if I understand correctly?) under /var/lib/rancher/rke2/agent/images
c
creamy-pencil-82913
08/04/2022, 6:10 PMI don’t think that would be sufficient. There’s more to an airgap install than just installing things and then cutting it off from the internet. See https://docs.rke2.io/install/airgap/
s
sticky-megabyte-50644
08/04/2022, 6:29 PMty
can I ask a last question, is that airgap tarball loaded into containerd on agent bootup time? if so, whenever the worker node gets populated with other images, when the image cleanup takes place, what excludes system images (like pause) from the cleanup?
c
creamy-pencil-82913
08/04/2022, 6:42 PMpause should be protected from GC by the kubelet itself, but the kubelet didn’t do a good job of that prior to 1.24
s
sticky-megabyte-50644
08/04/2022, 6:43 PMso how do we mitigate? upgrade to 1.24?
c
creamy-pencil-82913
08/04/2022, 6:43 PMthe best way to make sure that doesn’t happen is to make sure you don’t run out of disk space and tune your GH thresholds, or use a private registry mirror so that if GC does prune your images they can be pulled again.
s
sticky-megabyte-50644
08/04/2022, 6:43 PMok thats unerstandable, thank you!!!