sparse-monitor-30665
01/18/2023, 8:25 PMINFO[2023-01-18T19:28:39Z] Applying 'Cloud init' for stage 'boot'. Total stages: 1
INFO[2023-01-18T19:28:39Z] Processing stage step ''. ( commands: 2, files: 0, ... )
ERRO[2023-01-18T19:28:39Z] Udev database has incomplete information about device /dev/sda.
/dev/sda: Failed to get external handle [udev].
Udev database has incomplete information about device /dev/sda1.
/dev/sda1: Failed to get external handle [udev].
Udev database has incomplete information about device /dev/sda2.
/dev/sda2: Failed to get external handle [udev].
Udev database has incomplete information about device /dev/sda3.
/dev/sda3: Failed to get external handle [udev].
Udev database has incomplete information about device /dev/sda4.
/dev/sda4: Failed to get external handle [udev].
Udev database has incomplete information about device /dev/sda5.
/dev/sda5: Failed to get external handle [udev].
Udev database has incomplete information about device /dev/sdb.
/dev/sdb: Failed to get external handle [udev].
Device /dev/sdb excluded by a filter.
: failed to run if test -z "$(blkid /dev/rancher/longhorn)"; then vgcreate rancher /dev/sdb && lvcreate -l 100%FREE -n rancher/longhorn && mkfs -t ext4 -L longhorn /dev/rancher/longhorn; fi: exit status 5
Same with the mount statement after it
if test "$(blkid /dev/rancher/longhorn)"; then mount ... fi
Cheers!sticky-tailor-45974
01/19/2023, 8:38 AMsparse-monitor-30665
01/19/2023, 4:29 PMsticky-tailor-45974
01/19/2023, 4:37 PMsparse-monitor-30665
01/19/2023, 4:40 PMsticky-tailor-45974
01/19/2023, 4:44 PMsparse-monitor-30665
01/19/2023, 4:47 PMsticky-tailor-45974
01/20/2023, 10:47 AMsparse-monitor-30665
01/20/2023, 5:25 PMoverlay/oem/lvm_volumes_hook.yaml
could something as simple as:" instead of "_in_fstab" as you did earlier.
2. I can't get the actual hook to run. I never see "Running post-install" during the installation process. It finishes "after-install" then reboots.
3. The oem config does get copied to the /oem/... location and runs, but mounting fails as the Label doesn't exist.
4. If I use the default /var/lib/longhorn, will that have any conflict with the bind mount of the same listed prior in the fstab file. Should I be doing something like a 'sed -i ... ' type replace instead for this path?name: "Longhorn Data LVM"
stages:
initramfs:
- name: "Update fstab with new LVM path for Longhorn"
commands:
- |
sed -i "s|^/usr/local/.state/var-lib-longhorn.bind.*|/dev/rancher/longhorn /var/lib/longhorn xfs defaults 0 0|g" /etc/fstab
boot:
- name: "Unmount Longhorn in case it's mounted via bind"
commands:
- |
umount /var/lib/longhorn
- name: "Mount Longhorn via LVM"
commands:
- |
mount /var/lib/longhorn
I couldn't get it to mount the new path with just updating fstab in initramfs. Perhaps it would work if just adding a new path, and using that in Longhorn GUI, but that seems like more work. it appeared to be mounted already and I'm not sure at what stage that happened, but also couldn't unmount/remount there either. It worked in boot, so I made two stages.name: "Create Longhorn Storage Volume"
stages:
after-install:
- name: "Create LVM"
if: '[ -e "/dev/sdb" ]'
commands:
- |
# Create the physical volume, volume group and logical volume
pvcreate /dev/sdb
vgcreate vg0 /dev/sdb
lvcreate -l 100%FREE -n lv0 vg0
udevadm trigger
# Trigger udev detection
if [ ! -e "/dev/vg0/lv0" ]; then
udevadm settle
fi
- name: "Format LVM"
if: '[ -e "/dev/vg0/lv0" ]'
commands:
- |
# Format logical volume for later use in fstab
mkfs.xfs -L defaultDisk /dev/vg0/lv0
OEM: longhorn.yaml
name: "Longhorn Network and Data Configuration"
stages:
initramfs:
- name: "Setup eth1 network for Longhorn"
if: '[ "$(ip link | grep eth1)" ]'
files:
- path: /etc/sysconfig/network/ifcfg-eth1
content: |
BOOTPROTO='dhcp'
STARTMODE='auto'
permissions: 0600
owner: 0
group: 0
- name: "Create directory for Longhorn defaultDisk"
if: '[ "$(lvs | grep lv0)" ] && [ ! -e /var/lib/longhorn/defaultDisk ]'
commands:
- |
mkdir /var/lib/longhorn/defaultDisk
- name: "Update fstab for Longhorn defaultDisk"
if: '[ "$(lvs | grep lv0)" ] && [ -e /var/lib/longhorn/defaultDisk ]'
commands:
- |
echo "/dev/vg0/lv0 /var/lib/longhorn/defaultDisk xfs defaults 0 0" >> /etc/fstab
sticky-tailor-45974
01/24/2023, 2:32 PMpost-install
hook is indeed not available yet in stable version. Note in docs it has been added to the Next
version. Sorry my bad, should have warned about that. So first thing would be adapting the post-install
to after-install
hook.
I couldn't get it to mount the new path with just updating fstab in initramfs. Perhaps it would work if just adding a new path, and using that in Longhorn GUI, but that seems like more work. it appeared to be mounted already and I'm not sure at what stage that happened, but also couldn't unmount/remount there either. It worked in boot, so I made two stages.I am not really following you here sorry. But this could be related to the fact you already included such a mount point with immutable rootfs layout configuration in
/run/cos/cos-layout.env
. Note that in that file there is a list of persistent paths are mostly a bind mount from the persistent partition. If you have extra devices to be used as storage I would simply append the path in fstab as I documented and remove any reference to it in the rootfs stage cloud-init configuration.
The immutable-rootfs dracut module will create an fstab entry for all the mountpoint defined in /run/cos/cos-layout.env
to make sure systemd knows about it and it umounts them as expected on shutdown and also to make them obvious to any user logged into the system. So if you append new fstab lines including devices over an already existing mount point I understand systemd will complain on switch root and parsing the fstab file.rootfs:
- if: '[ ! -f "/run/cos/recovery_mode" ]'
name: "Layout configuration"
environment_file: /run/cos/cos-layout.env
environment:
VOLUMES: "LABEL=COS_OEM:/oem LABEL=COS_PERSISTENT:/usr/local"
OVERLAY: "tmpfs:25%"
RW_PATHS: "/var /etc /srv"
PERSISTENT_STATE_PATHS: >-
/etc/systemd
/etc/rancher
/etc/ssh
/etc/iscsi
/etc/cni
/home
/opt
/root
/usr/libexec
/var/log
/var/lib/elemental
/var/lib/rancher
/var/lib/kubelet
/var/lib/NetworkManager
/var/lib/longhorn
/var/lib/cni
/var/lib/calico
PERSISTENT_STATE_BIND: "true"
In that case you should remove the /var/lib/longhorn
path from the list as you want to handle that as an extra device. Extra devices I would not include them within the layout setup unless you have good reasons to do so. The immutable-rootfs module only sets the basic layout to ensure the system can switch root from initrd to the actual device keeping most of the OS in read only mode. Additional devices can be easily mounted and handled after the actual swtich root and let the underlying distribution handle it as it would on a traditional OS.sparse-monitor-30665
01/24/2023, 2:53 PM*udeadm* settle
...", but it should be "...ude*v*adm settle
...", it's missing the v in the middle. That one took me a bit to troubleshoot. 😉
Thanks a tonne for your assistance, and I'm glad we've been able to add more examples to help others accomplish the same in the future.sticky-tailor-45974
01/24/2023, 2:55 PM