Has anyone noticed this from their lima shell rancher-users #rancher-desktop

Title

quick-keyboard-83126

03/02/2023, 5:42 PM

Has anyone noticed this from their lima shell?

`lima-rancher-desktop:/etc# tail -f /var/log/messages`:

Mar  2 17:41:24 alpine <http://daemon.info|daemon.info> init: process '/sbin/getty -L 0 ttyS0 vt100' (pid 5836) exited. Scheduling for restart.
Mar  2 17:41:24 alpine <http://daemon.info|daemon.info> supervise-daemon[6021]: Child command line: /usr/local/bin/rancher-desktop-guestagent -kubernetes=false -iptables=false -debug
Mar  2 17:41:24 alpine daemon.warn supervise-daemon[3443]: /usr/local/bin/rancher-desktop-guestagent, pid 6021, exited with return code 0
Mar  2 17:41:25 alpine <http://daemon.info|daemon.info> init: starting pid 6047, tty '/dev/ttyS0': '/sbin/getty -L 0 ttyS0 vt100'
Mar  2 17:41:25 alpine auth.err getty[6047]: tcgetattr: I/O error^M
Mar  2 17:41:29 alpine <http://daemon.info|daemon.info> supervise-daemon[6084]: Child command line: /usr/local/bin/rancher-desktop-guestagent -kubernetes=false -iptables=false -debug
Mar  2 17:41:29 alpine daemon.warn supervise-daemon[3443]: /usr/local/bin/rancher-desktop-guestagent, pid 6084, exited with return code 0
Mar  2 17:41:35 alpine <http://daemon.info|daemon.info> supervise-daemon[6144]: Child command line: /usr/local/bin/rancher-desktop-guestagent -kubernetes=false -iptables=false -debug
Mar  2 17:41:35 alpine <http://daemon.info|daemon.info> init: process '/sbin/getty -L 0 ttyS0 vt100' (pid 6047) exited. Scheduling for restart.
Mar  2 17:41:35 alpine daemon.warn supervise-daemon[3443]: /usr/local/bin/rancher-desktop-guestagent, pid 6144, exited with return code 0
Mar  2 17:41:36 alpine <http://daemon.info|daemon.info> init: starting pid 6152, tty '/dev/ttyS0': '/sbin/getty -L 0 ttyS0 vt100'
Mar  2 17:41:36 alpine auth.err getty[6152]: tcgetattr: I/O error^M
Mar  2 17:41:40 alpine <http://daemon.info|daemon.info> supervise-daemon[6156]: Child command line: /usr/local/bin/rancher-desktop-guestagent -kubernetes=false -iptables=false -debug
Mar  2 17:41:40 alpine daemon.warn supervise-daemon[3443]: /usr/local/bin/rancher-desktop-guestagent, pid 6156, exited with return code 0
Mar  2 17:41:45 alpine <http://daemon.info|daemon.info> supervise-daemon[6165]: Child command line: /usr/local/bin/rancher-desktop-guestagent -kubernetes=false -iptables=false -debug
Mar  2 17:41:45 alpine daemon.warn supervise-daemon[3443]: /usr/local/bin/rancher-desktop-guestagent, pid 6165, exited with return code 0
Mar  2 17:41:46 alpine <http://daemon.info|daemon.info> init: process '/sbin/getty -L 0 ttyS0 vt100' (pid 6152) exited. Scheduling for restart.
Mar  2 17:41:47 alpine <http://daemon.info|daemon.info> init: starting pid 6172, tty '/dev/ttyS0': '/sbin/getty -L 0 ttyS0 vt100'
Mar  2 17:41:47 alpine auth.err getty[6172]: tcgetattr: I/O error^M

calm-sugar-3169

03/02/2023, 6:10 PM

is this on 1.7?

quick-keyboard-83126

03/02/2023, 6:10 PM

1.7.0

👀 1

calm-sugar-3169

03/02/2023, 6:21 PM

Maybe it’s related to this? https://gitlab.alpinelinux.org/alpine/aports/-/issues/9324

quick-keyboard-83126

03/02/2023, 6:22 PM

I tried commenting out those and kill -HUPign and it didn't seem to help much

i can't tell if i lose my world when i restart rancher desktop

calm-sugar-3169

03/02/2023, 6:22 PM

ha, I was going to suggest that 🤔

quick-keyboard-83126

03/02/2023, 6:23 PM

lima-rancher-desktop:/home/jsoref.linux# grep tty /etc/inittab
# Set up a couple of getty's
#tty1::respawn:/sbin/getty 38400 tty1
#tty2::respawn:/sbin/getty 38400 tty2
#tty3::respawn:/sbin/getty 38400 tty3
#tty4::respawn:/sbin/getty 38400 tty4
#tty5::respawn:/sbin/getty 38400 tty5
#tty6::respawn:/sbin/getty 38400 tty6
# Put a getty on the serial port
#ttyS0::respawn:/sbin/getty -L ttyS0 115200 vt100
ttyAMA0::respawn:/sbin/getty -L 0 ttyAMA0 vt100
#ttyS0::respawn:/sbin/getty -L 0 ttyS0 vt100

(I know one is uncommented, but you can see that the output specifies something that doesn't match the uncommented entry)

calm-sugar-3169

03/02/2023, 6:27 PM

Also, how do you think I can get into that state? I can’t see those messages on my end 😐

quick-keyboard-83126

03/02/2023, 6:27 PM

tail -f /var/log/messages

calm-sugar-3169

03/02/2023, 6:28 PM

not there for me 🤷

quick-keyboard-83126

03/02/2023, 6:28 PM

hmm

i have some proxy lines that i tried to put in /etc/conf.d/docker

lemme remove them and see if that changes things

👍 1

otherwise, i don't think i have anything exciting

calm-sugar-3169

03/02/2023, 6:29 PM

I wonder if it’s worth removing it and see if you see the logs again

quick-keyboard-83126

03/02/2023, 6:29 PM

i added

strace

and

tcpdump

to do some debugging...

none of these feel like good candidates 🙂 -- i'm on an m1 if that matters

calm-sugar-3169

03/02/2023, 6:30 PM

oh, that might be the candidate I think

I’m on

x86_64

intel

quick-keyboard-83126

03/02/2023, 6:30 PM

Linux lima-rancher-desktop 5.15.78-0-virt #1-Alpine SMP Fri, 11 Nov 2022 10:19:45 +0000 aarch64 Linux

calm-sugar-3169

03/02/2023, 6:31 PM

Let me get someone with an M1 machine to see if they can see those errs maybe?!

quick-keyboard-83126

03/02/2023, 6:31 PM

so, restarted w/ those commented out -- still happens

calm-sugar-3169

03/02/2023, 6:33 PM

just curious if anything interesting in the kernel logs?

dmesg

quick-keyboard-83126

03/02/2023, 6:33 PM

dmesg

👀 1

calm-sugar-3169

03/02/2023, 6:36 PM

@fast-garage-66093 are you on a

M1

machine? can you check if you see the logs mentioned above in

/var/log/messages

I can’t see them on my machine, I’m wondering if it’s

arch

specific issue that we might be seeing here.

fast-garage-66093

03/02/2023, 6:45 PM

I get these 3 lines every 10 seconds, but that's it:

Mar  2 18:44:28 alpine <http://daemon.info|daemon.info> init: process '/sbin/getty -L 0 ttyS0 vt100' (pid 5939) exited. Scheduling for restart.
Mar  2 18:44:29 alpine <http://daemon.info|daemon.info> init: starting pid 5953, tty '/dev/ttyS0': '/sbin/getty -L 0 ttyS0 vt100'
Mar  2 18:44:29 alpine auth.err getty[5953]: tcgetattr: I/O error^M

quick-keyboard-83126

03/02/2023, 6:45 PM

@calm-sugar-3169 so, sounds like it's an aarch thing 🙂

calm-sugar-3169

03/02/2023, 6:45 PM

yeah, those logs are the issue

fast-garage-66093

03/02/2023, 6:46 PM

I thought this was about the guest agent, wasn't it?

I don't get any guest-agent logs in there

calm-sugar-3169

03/02/2023, 6:47 PM

it’s more of an alpine issue than guest agent I think

quick-keyboard-83126

03/02/2023, 6:47 PM

I'm only really complaining about the getty / tcgetattr i/o errors

calm-sugar-3169

03/02/2023, 6:47 PM

the only error I see in Josh’s logs are

Mar  2 18:44:29 alpine auth.err getty[5953]: tcgetattr: I/O error^M

everything else is

info

warn

quick-keyboard-83126

03/02/2023, 6:48 PM

@calm-sugar-3169 that it's constantly restarting the gettys is a problem

they should be sleeping 100% of the time

fast-garage-66093

03/02/2023, 6:48 PM

So why is the guest-agent constantly stopping for him?

Mar  2 17:41:40 alpine <http://daemon.info|daemon.info> supervise-daemon[6156]: Child command line: /usr/local/bin/rancher-desktop-guestagent -kubernetes=false -iptables=false -debug
Mar  2 17:41:40 alpine daemon.warn supervise-daemon[3443]: /usr/local/bin/rancher-desktop-guestagent, pid 6156, exited with return code 0

calm-sugar-3169

03/02/2023, 6:52 PM

@quick-keyboard-83126 are you constantly seeing also

Mar  2 17:41:40 alpine <http://daemon.info|daemon.info> supervise-daemon[6156]: Child command line: /usr/local/bin/rancher-desktop-guestagent -kubernetes=false -iptables=false -debug
Mar  2 17:41:40 alpine daemon.warn supervise-daemon[3443]: /usr/local/bin/rancher-desktop-guestagent, pid 6156, exited with return code 0

? I thought these were just from a restart or something?

quick-keyboard-83126

03/02/2023, 6:52 PM

slow moving train wreck?

Untitled

fast-garage-66093

03/02/2023, 6:52 PM

Yes, they were also coming every 10 seconds. I thought that was the issue we are talking about here. The getty one messages are just noise

quick-keyboard-83126

03/02/2023, 6:53 PM

imo they're all bugs

there's no reason any of these things should be restarting or thinking periodically if no one is doing anything to anything

And that's a description of the host:

jsoref@jsoref-mbp ~ % docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

fast-garage-66093

03/02/2023, 6:55 PM

I found https://gitlab.alpinelinux.org/alpine/aports/-/issues/11599 about the getty issue, but I don't understand why it only happens on aarch64 then.

calm-sugar-3169

03/02/2023, 6:55 PM

yeah, he tried that already

quick-keyboard-83126

03/02/2023, 6:55 PM

yeah, we all found that

fast-garage-66093

03/02/2023, 6:56 PM

Can you file a Github issue for this, maybe we can just disable the serial console in the image. I just don't want to break the ability to use

socat

serial.sock

to connect to the VM when

ssh

is broken. But maybe we no longer need this (and it doesn't work with vz anyways)

quick-keyboard-83126

03/02/2023, 6:57 PM

yeah, i guess i can, i'm currently baby-sitting some cleanup elsewhere

fast-garage-66093

03/02/2023, 6:57 PM

But this doesn't explain the guest-agent problem, which is what I'm more concerned about

Anyways, time for another meeting...

Hmm, I suspect we will also lose

serial.log

if we disable the tty, so not sure I want to do that

quick-keyboard-83126

03/02/2023, 7:01 PM

https://github.com/rancher-sandbox/rancher-desktop/issues/4085

where's that file?

lima-rancher-desktop:~# ls /var/log/
acpid.log                       dmesg                           lima-init.log                   openresty                       wtmp
cloud-init-output.log           docker.log                      messages                        rancher-desktop-guestagent.log

fast-garage-66093

03/02/2023, 7:22 PM

It is on the host; symlinked from

~/Library/Logs/rancher-desktop/lima.serial.log

quick-keyboard-83126

03/02/2023, 7:23 PM

that can't possibly be from tty stuff

it includes UEFI

fast-garage-66093

03/02/2023, 7:24 PM

Yeah, doesn't look like it:

-chardev socket,id=char-serial,path=/Users/jan/Library/Application Support/rancher-desktop/lima/0/serial.sock,server=on,wait=off,logfile=/Users/jan/Library/Application Support/rancher-desktop/lima/0/serial.log

Unfortunately we are wrong, When I build the ISO with this change, then

serial.log

is empty:

--- mkimg.lima.sh
+++ mkimg.lima.sh
@@ -10,8 +10,7 @@ profile_lima() {
        initfs_cmdline="modules=loop,squashfs,sd-mod,usb-storage"
        kernel_addons=
        kernel_flavors="virt"
-       kernel_cmdline="console=tty0 console=ttyS0,115200"
-       syslinux_serial="0 115200"
+       kernel_cmdline="console=tty0"
        apkovl="genapkovl-lima.sh"
        apks="$apks openssh-server-pam"
         if [ "${LIMA_INSTALL_CA_CERTIFICATES}" == "true" ]; then

So not going to make that change

quick-keyboard-83126

03/02/2023, 7:43 PM

does it really lose the uefi bits too?

fast-garage-66093

03/02/2023, 7:43 PM

$ ls -l ~/.lima/rd/serial.log
-rw-r--r--  1 jan  staff  0  2 Mar 11:39 /Users/jan/.lima/rd/serial.log

quick-keyboard-83126

03/02/2023, 7:43 PM

that makes absolutely no sense

ok, but can you just drop the

console=ttyS0,...

part and leave

syslinux_serial=....

fast-garage-66093

03/02/2023, 7:45 PM

I can try, but after that I will have to move on (for now). My bigger issue is that Alpine 3.17 doesn't work on aarch64 with Apple's virtualization framework. And I'm not dealing with it either, except by rolling back to 3.16 (for now).

Now I get this:

$ cat ~/.lima/rd/serial.log

ISOLINUX 6.04 6.04-pre1  Copyright (C) 1994-2015 H. Peter Anvin et al
boot:
Loading /boot/vmlinuz-virt... ok
Loading /boot/initramfs-virt...ok

So, not going to disable the serial console for now.

Did you have a way to stop the restarts later? Can you just comment it out in

/etc/inittab

and it will stop?

quick-keyboard-83126

03/02/2023, 7:50 PM

tried, didn't work

fast-garage-66093

03/02/2023, 7:50 PM

😞

quick-keyboard-83126

03/02/2023, 7:50 PM

yep

doesn't make any sense, but, also didn't work

fast-garage-66093

03/02/2023, 7:51 PM

Ok, will have to wait for later; it really is just an annoyance right now though, it doesn't actually break anything?

quick-keyboard-83126

03/02/2023, 7:52 PM

yeah, it's just an annoyance.... i mean, to some extent every stupid process that's actively waking up my poor computer hurts its battery

i'm only looking at logs because i'm trying to figure out how to convince docker to talk to a different internet egress and that's been all sorts of fail

fast-garage-66093

03/02/2023, 7:52 PM

Yeah, I know, there is a lot of polling going on that we should try to rein in

👍 1

quick-keyboard-83126

03/02/2023, 7:53 PM

if i wasn't trying (and failing) to get docker to do this, i wouldn't have looked under the hood

fast-garage-66093

03/02/2023, 7:53 PM

got it

At least we have a Github issue now

9 Views

#rancher-desktop Join Slack