This is possible future architecture of my k3s clu...
# k3s
f
This is possible future architecture of my k3s cluster - just personal services for my family, friends, and the current one . The legend: 1. 1th Master - Oracle VM (1 vCPU, 1gb RAM, Availability Domain AD-1) - actually lanch some important telegram and discord bots, but mostly it's just master 2. 2th Master - Oractle VM(1 vCPU, 1gb RAM, AD-2) - actually lanch some important telegram and discord bots, but mostly it's just master 3. Google Cloud SQL HA (they have very cheap option) in the same city, but different datacenter (because Oracle and Cloud) for HA - so, I will not use etcd for HA 4. my home NAS - just fanless mini-itx x86_64 atom machine (4 weak CPU, 8gb RAM, a lot of space in the raid-z) - currently I don't have it in the k3s cluster 5. AWS ec2 spot instances + my own autoscaler - currently I don't have it. But I want in a case if I have powerloss at the home 6. cheap VM from OVH as agent node (not master) - I use it as main compute node. Want to decommission it Currently I use k3s with  machines 1, 2 and the cheap VM from OVH as non-compute node(let's assign to it number 6), as well as 3, because I was unable to create a connection between 4 and other nodes (because no public IP, and a double NAT). I have OpenWRT in my home router(let's call it 7), which is behind ISP router(let's call it number 8 ). I already have openvpn installed in it. I can also install here a wireguard, or install it in the NAS machine I use HA via round-robin DNS (and k3s supports this way) for both masters, and google cloud SQL HA instance - I use MySQL for k3s and my own apps, and the Oracle Cloud Free load balancer for one important app. Also I use AWS S3 for apps data. I don't use Persistent volumes. My goals are: because it's personal, I want to have a cheap HA Questions: a) Is it good, solid plan? What do you think? b) My main question and concern is about networking. How to deal with a double nat? Also I'm a little bit worry about security at home. I have a netgear semi-stupid switch with some vlan support. Is it good idea to make some sort of mapping from vlans to vxflans in the flannel? Or I need another CNI? Or maybe Cloudflare's tunnel could work better? c) How can I utilize 40gb of space in my 2 oracle VMs? How do you think, is it even possible to launch maybe minio in one of masters? I have only 1gb ram here... And I don't use etcd Also I think about cloudflare load balancer (in order to balance traffic between home compute node and possible ec2 spot) as well as about cloudflare argo tunnel (maybe it's good solution for my case of a double nat ?) Another thing which I'm going to use is wassabi S3 with minio operator as cache, because it's really cheap (but for backup I'll use AWS S3, oracle). What do you think about it?