I figured out the issue. Sorry for long response.
Issue was that Harvester was not showing the IP address even though the VM had an ip address. The IP address was not being shown due to the qemu-guest-agent not getting installed on boot by cloud init. Cloud init was not installing qemu-guest-agent because I found out cloud init was disabling itself on boot causing the cloud-init scripts to never get executed.
Cloud-init was disabling itself because in the boot process of cloud-init (explained in Cloud Init's docs here
https://cloudinit.readthedocs.io/en/latest/explanation/boot.html#generator ) it runs a tool called ds-identify during the first "Detect" stage. This stage basically determines what environment you're running cloud init in and configures it to use that environment's cloud init datasource. Here is a list of available cloud init data sources
https://cloudinit.readthedocs.io/en/latest/reference/datasources.html . This is why my image was working in AWS environments because cloud init could discover the AWS datasource. In the event that cloud init DOES NOT find a datasource like in the case of running in Harvester, cloud-init disables itself on boot.
The fix to my issue was that I had to configure cloud init to boot even in the case of it not finding a datasource. This can be done via kernel parameter or cloud init configuration file. Documentation on how to configure cloud init is here
https://github.com/canonical/cloud-init/blob/main/tools/ds-identify
After I configured cloud-init correctly, I was able to boot a VM in harvester with that custom image, cloud init started and installed the qemu-guest-agent package, and then an IP was received in the Harvester UI for my VM.