adamant-kite-43734
03/26/2024, 3:23 PMproud-jewelry-46860
03/26/2024, 4:54 PMbillowy-traffic-45695
03/26/2024, 5:16 PMbillowy-traffic-45695
03/26/2024, 5:20 PMbillowy-traffic-45695
03/26/2024, 5:22 PMproud-jewelry-46860
03/26/2024, 5:46 PMk8s.log
seems fine. background.log
not having .kube/config
is fine; the only issue is the failure to run update-ca-certificates
. lima.log
doesn't tell us anything interesting in that line (just that it's stopped).
It might be useful to enable debug logs then trying again, but this time focus on lima.log
around where it's updating the certs (that is, examine the bits right around it running limactl.ventura --debug shell --workdir=. 0 sudo update-ca-certificates
).billowy-traffic-45695
03/26/2024, 5:47 PMbillowy-traffic-45695
03/26/2024, 5:54 PMbillowy-traffic-45695
03/26/2024, 5:55 PMbillowy-traffic-45695
03/26/2024, 5:56 PMtime=“2024-03-26T17:49:54Z” level=debug msg=“executing ssh (may take a long)): [/usr/bin/ssh -F /dev/null -o IdentityFile=\\“/Users/paulrandall/.rdlima/_config/user\\” -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\\“^<mailto:aes128-gcm@openssh.com|aes128-gcm@openssh.com>,<mailto:aes256-gcm@openssh.com|aes256-gcm@openssh.com>\\” -o User=paulrandall -o ControlMaster=auto -o ControlPath=\\“/Users/paulrandall/.rdlima/0/ssh.sock\\” -o ControlPersist=yes -q -p 49287 127.0.0.1 -- cd . || exit 1 ; exec \\“$SHELL\\” --login -c ‘sudo update-ca-certificates’]“\n
+
‘WARNING: Skipping duplicate certificate in file ca-cert-Certum_Trusted_Network_CA.pem\n’ +
‘WARNING: Skipping duplicate certificate in file ca-cert-emSign_ECC_Root_CA_-_G3.pem\n’ +
‘WARNING: Skipping duplicate certificate in file ca-cert-rd-37.pem\n’ +
‘WARNING: Skipping duplicate certificate in file ca-cert-GlobalSign_Root_E46.pem\n’ +billowy-traffic-45695
03/26/2024, 5:58 PMbillowy-traffic-45695
03/26/2024, 5:58 PMbillowy-traffic-45695
03/26/2024, 6:00 PMbillowy-traffic-45695
03/26/2024, 6:04 PMproud-jewelry-46860
03/26/2024, 6:30 PMThat's probably the actual problem? That script actually runsrun-parts: /etc/ca-certificates/update.d/certhash: exit status 132
/usr/bin/c_rehash /etc/ssl/certs
, so there's probably one cert that it didn't like.billowy-traffic-45695
03/26/2024, 6:31 PMproud-jewelry-46860
03/26/2024, 6:32 PMLIMA_HOME
environment variable to /Users/paulrandall/.rdlima/
• Run /Applications/Rancher <http://Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura|Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura> start 0
• (This starts the same VM Rancher Desktop starts, but without all the things we do to make it works as a container thing)
• Run /Applications/Rancher <http://Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura|Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura> shell 0
to get into the VM
• Run /usr/bin/c_rehash -v /etc/ssl/certs
to see if that tell you which cert is breaking things.billowy-traffic-45695
03/26/2024, 6:32 PMproud-jewelry-46860
03/26/2024, 6:33 PMproud-jewelry-46860
03/26/2024, 6:34 PMnerdctl
or `docker`; but before you do that, use the Rancher Desktop UI to make a snapshot so you have more chances to roll back?billowy-traffic-45695
03/26/2024, 6:36 PMbillowy-traffic-45695
03/26/2024, 6:38 PMproud-jewelry-46860
03/26/2024, 6:39 PM<http://Terminal.app|Terminal.app>
(as in, the application)
• enter export LIMA_HOME=/Users/paulrandall/.rdlima/
to set the variable for that session (it goes away once you close the window)
• Run the things I had above.proud-jewelry-46860
03/26/2024, 6:42 PMrdctl snapshot create --help
for instructions) after you quit Rancher Desktop.billowy-traffic-45695
03/26/2024, 6:45 PMbillowy-traffic-45695
03/26/2024, 6:49 PMproud-jewelry-46860
03/26/2024, 6:49 PMexport …
line is set up to let limactl
later know what your setup looks like.proud-jewelry-46860
03/26/2024, 6:49 PMrdctl snapshot create abcdef
or something to create a snapshotbillowy-traffic-45695
03/26/2024, 6:51 PMproud-jewelry-46860
03/26/2024, 6:52 PMbillowy-traffic-45695
03/26/2024, 6:53 PMLIMA_HOME
environment variable to /Users/paulrandall/.rdlima/
• Run /Applications/Rancher <http://Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura|Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura> start 0
• (This starts the same VM Rancher Desktop starts, but without all the things we do to make it works as a container thing)
• Run /Applications/Rancher <http://Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura|Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura> shell 0
to get into the VM
• Run /usr/bin/c_rehash -v /etc/ssl/certs
to see if that tell you which cert is breaking things.
Also do I run them while RD is running or when it’s quit?proud-jewelry-46860
03/26/2024, 6:53 PMexport
line is what is used to _Set the LIMA_HOME
environment variable_.billowy-traffic-45695
03/26/2024, 6:53 PMproud-jewelry-46860
03/26/2024, 6:54 PMbillowy-traffic-45695
03/26/2024, 6:54 PMproud-jewelry-46860
03/26/2024, 6:54 PMbillowy-traffic-45695
03/26/2024, 6:55 PMbillowy-traffic-45695
03/26/2024, 6:56 PMbillowy-traffic-45695
03/26/2024, 6:57 PMproud-jewelry-46860
03/26/2024, 6:59 PMexport LIMA_HOME=/Users/paulrandall/.rdlima/
• /Applications/Rancher <http://Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura|Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura> start 0
• (wait for things to happen)
• /Applications/Rancher <http://Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura|Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura> shell 0
• /usr/bin/c_rehash -v /etc/ssl/certs
billowy-traffic-45695
03/26/2024, 6:59 PMproud-jewelry-46860
03/26/2024, 7:00 PMbillowy-traffic-45695
03/26/2024, 7:01 PMproud-jewelry-46860
03/26/2024, 7:02 PMexport LIMA_HOME=/Users/paulrandall/.rdlima/
• export PATH="$PATH:/Applications/Rancher <http://Desktop.app/Contents/Resources/resources/darwin/lima/bin|Desktop.app/Contents/Resources/resources/darwin/lima/bin>"
• "/Applications/Rancher <http://Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura|Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura>" start 0
• (wait for things to happen)
• "/Applications/Rancher <http://Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura|Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura>" shell 0
• /usr/bin/c_rehash -v /etc/ssl/certs
(Added the second line with export PATH
)billowy-traffic-45695
03/26/2024, 7:04 PMbillowy-traffic-45695
03/26/2024, 7:13 PMbillowy-traffic-45695
03/26/2024, 7:21 PMproud-jewelry-46860
03/26/2024, 7:24 PMproud-jewelry-46860
03/26/2024, 7:26 PM/Users/paulrandall/.rdlima/0/diffdisk
(it's a qemu disk image, or a raw disk if you're on VZ)billowy-traffic-45695
03/26/2024, 7:26 PMbillowy-traffic-45695
03/26/2024, 7:26 PMproud-jewelry-46860
03/26/2024, 7:26 PMbillowy-traffic-45695
03/26/2024, 7:27 PMlimactl shell 0
to open the shell.
paulrandall@Mac-Mini-Server ~ %
“/Applications/Rancher Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl.ventura” shell 0
lima-rancher-desktop:/Users/paulrandall$ /usr/bin/c_rehash -v /etc/ssl/certs
ERROR: Access denied ‘/etc/ssl/certs’
lima-rancher-desktop:/Users/paulrandall$billowy-traffic-45695
03/26/2024, 7:28 PMproud-jewelry-46860
03/26/2024, 7:30 PMproud-jewelry-46860
03/26/2024, 7:30 PM/etc/ssl/
is owned by rootproud-jewelry-46860
03/26/2024, 7:31 PMbillowy-traffic-45695
03/26/2024, 7:31 PMproud-jewelry-46860
03/26/2024, 7:33 PMbillowy-traffic-45695
03/26/2024, 7:34 PMproud-jewelry-46860
03/26/2024, 7:35 PMbillowy-traffic-45695
03/26/2024, 7:35 PMproud-jewelry-46860
03/26/2024, 7:36 PMbillowy-traffic-45695
03/26/2024, 7:37 PMproud-jewelry-46860
03/26/2024, 7:42 PMc_rehash
on that and try to see if you can narrow things down to a single cert that breaks things. Given the previous comments, though, I don't expect good results.billowy-traffic-45695
03/26/2024, 7:43 PMbillowy-traffic-45695
03/26/2024, 7:44 PMproud-jewelry-46860
03/26/2024, 7:45 PMrd-
certs in /usr/local/share/ca-certificates/
, and all the dangling symlinks that creates in /etc/ssl/certs/
, and see if that magically helps (since people have claimed that a factory reset was useful…)proud-jewelry-46860
03/26/2024, 7:45 PMc_rehash
again)proud-jewelry-46860
03/26/2024, 7:46 PMbillowy-traffic-45695
03/26/2024, 7:50 PMbillowy-traffic-45695
03/26/2024, 7:51 PMproud-jewelry-46860
03/26/2024, 7:52 PMproud-jewelry-46860
03/26/2024, 7:52 PMproud-jewelry-46860
03/26/2024, 7:53 PMdiffdisk
you might be able to copy it in; Docker Desktop has no idea how to deal with the lima files you have (because they have a completely different backend).proud-jewelry-46860
03/26/2024, 7:53 PMbillowy-traffic-45695
03/26/2024, 7:54 PMbillowy-traffic-45695
03/26/2024, 7:56 PMproud-jewelry-46860
03/26/2024, 7:58 PMproud-jewelry-46860
03/26/2024, 7:59 PMproud-jewelry-46860
03/26/2024, 7:59 PMbillowy-traffic-45695
03/26/2024, 8:01 PMbillowy-traffic-45695
03/26/2024, 8:01 PMproud-jewelry-46860
03/26/2024, 8:07 PMdb
) and not mapped to a place on the host, so it's inside the diffdiskbillowy-traffic-45695
03/26/2024, 8:08 PMproud-jewelry-46860
03/26/2024, 8:15 PMlimactl shell
), right? You can just copy files off of it…proud-jewelry-46860
03/26/2024, 8:16 PMlimactl start
started the VM, and you got a ssh session with limactl shell
, so you can poke around already.billowy-traffic-45695
03/26/2024, 8:17 PMbillowy-traffic-45695
03/26/2024, 8:18 PMbillowy-traffic-45695
03/26/2024, 8:19 PMproud-jewelry-46860
03/26/2024, 8:19 PMscp -F /Users/paulrandall/.rdlima/0/ssh.config lima-0:/path/to/file /path/on/mac-os
to copy files outproud-jewelry-46860
03/26/2024, 8:20 PMproud-jewelry-46860
03/26/2024, 8:21 PMbillowy-traffic-45695
03/26/2024, 8:22 PMproud-jewelry-46860
03/26/2024, 8:22 PMbillowy-traffic-45695
03/26/2024, 8:23 PMproud-jewelry-46860
03/26/2024, 8:25 PMbillowy-traffic-45695
03/26/2024, 8:28 PMproud-jewelry-46860
03/26/2024, 8:31 PMproud-jewelry-46860
03/26/2024, 8:32 PM/var/lib/docker/volumes
in the VM.billowy-traffic-45695
03/26/2024, 8:33 PMproud-jewelry-46860
03/26/2024, 8:33 PMsu
or sudo
.billowy-traffic-45695
03/26/2024, 8:34 PMbillowy-traffic-45695
03/26/2024, 8:34 PMbillowy-traffic-45695
03/26/2024, 8:35 PMproud-jewelry-46860
03/26/2024, 8:36 PMsudo su
?billowy-traffic-45695
03/26/2024, 8:37 PMbillowy-traffic-45695
03/26/2024, 8:38 PMbillowy-traffic-45695
03/26/2024, 8:39 PMbillowy-traffic-45695
03/26/2024, 8:41 PMbillowy-traffic-45695
03/26/2024, 8:45 PMproud-jewelry-46860
03/26/2024, 8:46 PMbillowy-traffic-45695
03/26/2024, 8:48 PMproud-jewelry-46860
03/26/2024, 8:50 PMtar czf /tmp/db.tgz .
) and copy that?proud-jewelry-46860
03/26/2024, 8:51 PM/Users/paulrandall
if I remember correctly, so you might be able to just create the archive there and check that it shows up)billowy-traffic-45695
03/26/2024, 8:53 PMbillowy-traffic-45695
03/26/2024, 8:54 PMproud-jewelry-46860
03/26/2024, 8:55 PMcd /var/lib/docker/volumes/db ; tar czf /Users/paulrandall/wordpress-db.tgz .
billowy-traffic-45695
03/26/2024, 8:56 PMbillowy-traffic-45695
03/26/2024, 8:57 PMbillowy-traffic-45695
03/26/2024, 9:00 PMproud-jewelry-46860
03/26/2024, 9:04 PMbillowy-traffic-45695
03/26/2024, 9:05 PMbillowy-traffic-45695
03/26/2024, 9:06 PMproud-jewelry-46860
03/26/2024, 9:10 PMproud-jewelry-46860
03/26/2024, 9:11 PMbillowy-traffic-45695
03/26/2024, 9:13 PMbillowy-traffic-45695
03/26/2024, 9:14 PMbillowy-traffic-45695
03/26/2024, 9:15 PMbillowy-traffic-45695
03/27/2024, 8:20 AMproud-jewelry-46860
03/27/2024, 4:36 PMbillowy-traffic-45695
04/01/2024, 4:39 PMproud-jewelry-46860
04/01/2024, 7:03 PMrdctl shutdown
from cron
, I guess? We might need to dig into the behaviour when the host shuts down, though, yes.billowy-traffic-45695
04/02/2024, 1:29 PMproud-jewelry-46860
04/02/2024, 4:56 PM