Hey, we have a buch of bare metal server where we want to run k8s. I've experimented with https://github.com/metal3-io/baremetal-operator from metal3 and they combination of ironic with yaml driven configuration (which is quite gitops friendly!) is very attractive. Is there a way to install harvester in such an automatic fashion? We really don't want to run with USB sticks 🙂

You could try the PXE method: PXE Boot Installation | Harvester

@acoustic-addition-45641 I've read about it, but it's far from decent state managment of phisical servers that metal3 offers. So I wandered if anyone maybe actually tried to use metal3 with harvester? 😆

Due to Harvester’s unique way of booting it has to be the HOST os. PXE is the way : You can automate the install fairly easy.

@thousands-advantage-10804 I understand the pxe method, just with ironic/metal3 you first automated the configuration of bare metal (bios etc) and at the end you acutally boot and image. Hence I'm wondering (and will be willing to experiment) could I possibly put here: https://github.com/metal3-io/baremetal-operator/blob/main/examples/demo-hosts.yaml#L164 the url to the harvester image and then just provide to the harvester inataller the configuration it requirers

That operator uses libvirt to actually boot “images” from a qcow2. Harvester is designed to be on the host itself. There us not a qcow2 for harvester available because the install process is vital. It won’t work.

@thousands-advantage-10804 in this example there is qcow2, but the docs also mention raw (I guess it's the same as ISO?) https://book.metal3.io/bmo/introduction.html#deploying-baremetalhosts. libvirt? I might be mistake, but as I tested the metal3 flow to provision baremetal hosts with ubuntu image I just configured the bios/redfish (some hardware RADIS etc) and then just told it to boot ubuntu, ending up with exacly the same situation as I would if I configured the bios by hand and then boot ubuntu form iso/usb

I would love to see if works. It looked like it may be able to manage the host through the BMC.

I would love to hear that somone else tried it 😄 if not I will try soon 😄

@fancy-fountain-55453 please share those findings. I've been playing with the Dell Terraform Redfish provider recently in an effort to baremetal boot our Dell servers to a fully provisioned host OS (Ubuntu primarily) without the need to involve PXE. The general idea is to have the provider mount the installation ISO and also mount a separate IMG that contains a cloud config (Harvester config) CIDATA volume. However, Harvester does not have an autoinstall option when booting the ISO, so automating a Harvester install in this manner isn't an option. What would be nice is if Harvester had an autoinstall countdown timer immediately after ISO boot ("Looking for autoinstall file. Press any key to proceed.") that would at least scan other mounts for the presence of cloud-config (Harvester config) CIDATA volume... or something like that. I'm pretty sure that Redfish APIs are not Dell specific, so the Terraform provider may be applicable to the BMC of other hardware platforms. The Dell Redfish Terraform provider also allows for adjusting Dell's server configuration profile, which allows for (as an example) booting a RedHat golden image and providing answer files. All that to say, I wish Harvester had a means of autoinstalling with just an ISO + cloud-config (Harvester config). @thousands-advantage-10804 Love your YouTube content! Keep it coming!!!

You need to make a video showing Redfish.

can't you just set

kargs

like

harvester.install.automatic=true harvester.install.config_url

etc and it will install from ISO? we've basically done this before in a pinch

@acoustic-addition-45641 when provisioning (any image) with matal3 you can inject a config file to for example configure cloud-init. (https://book.metal3.io/bmo/instance_customization#userdata) as i read the docs

User and network data are passed to the instance via a so called _config drive_, which is a small additional disk partition created on the root device during provisioning.

so it also boils down auto mounting some config as a separate partition.

how are you passing the config drive to harvester installer?

since this is living rent free in my head.. Is there a way to pass the userdata into https://book.metal3.io/bmo/live-iso

since this is living rent free in my head..

- I know the feeling 😄 today I mobilized people to fix the bare metal network in our small DC so I can experiment 😄 @thousands-advantage-10804 regarding extra kernel params I found this https://github.com/metal3-io/baremetal-operator/blob/4409f7185cef78b4162da8a4c5acb[…]546/config/base/crds/bases/metal3.io_preprovisioningimages.yaml I need to dig into this, but even if some option is lacking it's just pull request to metal3 away from beeing added 🙂

crazy thought. use metal3 to boot the install kernel with the args to pull the iso from http/tftp like pxe.

IRONIC_KERNEL_PARAMS - Kernel parameters to pass to IPA (default console=ttyS0)

sounds like it ought to work

@prehistoric-morning-49258

IRONIC_KERNEL_PARAMS

but those parameters go to the IPA (ironic python agent) ramdisk that is first booted to inspect/configure baremetal, before actually OS happens. So far I failed to find a way to inject anything apart from mounted disk with

USER_DATA

and

NETWORK_DATA

which are later used by cloud-init and such.

something like ipxe should work. might need to bake an ISO per-server depending on reqs, and/or scripted takeover install. basically boot to whatever OS can chain/takeover into harvester installer, imo ipxe is usually best option

oh damm, abort? start drinking?

@prehistoric-morning-49258 with just ipxe it's indeed easy, I just had a dream of combining with ironic/meta3 etc for more gitops managment 🙂 But ironic serves its own boot.ipxe (for provisioning of ipa agent to gather inventory of bare metal) that is picked by netboot and brakes the netboot process ;-(

idk its like metal3 is whats preventing it being automated

@prehistoric-morning-49258 well with metal3 you would have a nice gitops repo with all bare metal and it's configuration there just waiting to be boostraped (or reinstalled in case or early very iterative development process 🙂 ) with ipxe it's more interactive automation 🙂 as you have to do a bit of manual picking (which host to start harvester with, which to join etc). Also, I'm not sure how harvester handles server with a lot of disks. Normally, a pre-provisioning tool - like ipa - does mdadm magic, or metal3 talkis via redfish to configure the hardware raid before the os is booted. Here I wonder what are actual capabilities of harvester are for configuring the hardware... looking at this sole variable https://docs.harvesterhci.io/v1.1/install/harvester-configuration#installdata_disk not soo big 😉

yeah you can pre-bake in a variety of ways, really it seems more like a metal3 support question than harvester support