This message was deleted.
# general
a
This message was deleted.
p
Heres mine for debian 12 : programs :
Copy code
apt install open-iscsi nfs-common net-tools htop sudo snmpd p7zip-full rsync
+ docker
Copy code
# Add Docker's official GPG key:
apt-get update
apt-get install ca-certificates curl
install -m 0755 -d /etc/apt/keyrings
curl -fsSL <https://download.docker.com/linux/debian/gpg> -o /etc/apt/keyrings/docker.asc
chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to Apt sources:
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] <https://download.docker.com/linux/debian> \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  tee /etc/apt/sources.list.d/docker.list > /dev/null
apt-get update
apt-get install docker-ce docker-ce-cli <http://containerd.io|containerd.io> docker-buildx-plugin docker-compose-plugin
Sysctls :
Copy code
fs.inotify.max_user_watches=2099999999
fs.inotify.max_user_instances=2099999999
fs.inotify.max_queued_events=2099999999
vm.max_map_count = 262144
The rest is as bonestock as it gets, so its simpler and more reproducible.
I guess there has to be an ansible book somewhere lmao
m
Thanks @powerful-librarian-10572 Mine on Ubuntu ( I need to use ipvs). I am not sure, when comes to Rancher/RKE2, RKE2 can do all for me?
Copy code
sudo swapoff -a
sudo sed -i '/swap/s/^/#/' /etc/fstab
sudo ufw disable
sudo apt install wget net-tools apt-utils lvm2 -y
sudo apt install -y chrony
sudo systemctl enable chrony
sudo systemctl start chrony

sudo tee -a /etc/security/limits.conf <<EOF
soft nofile 655360
hard nofile 131072
soft nproc 655350
hard nproc 655350
soft memlock unlimited
hard memlock unlimited
EOF

sudo apt install ipvsadm ipset sysstat conntrack libseccomp2 -y
# Load IPVS kernel modules
sudo modprobe -- ip_vs
sudo modprobe -- ip_vs_rr
sudo modprobe -- ip_vs_wrr
sudo modprobe -- ip_vs_sh
sudo modprobe -- nf_conntrack


sudo tee -a /etc/modules-load.d/ipvs.conf <<EOF
ip_vs
ip_vs_lc
ip_vs_wlc
ip_vs_rr
ip_vs_wrr
ip_vs_lblc
ip_vs_lblcr
ip_vs_dh
ip_vs_sh
ip_vs_fo
ip_vs_nq
ip_vs_sed
ip_vs_ftp
ip_vs_sh
nf_conntrack
ip_tables
ip_set
xt_set
ipt_set
ipt_rpfilter
ipt_REJECT
ipip
EOF
p
I don't know, ive taken the stance of "if it aint broken, don't fix it". My production cluster run well enough on my settings, and the least i fiddle around the least it would break in a complex manner.
1
Granted, i'm not using a stock debian 12 but the vendor image provided by my hoster (ovh) so it could be already optimized in ways i am not aware of.
1
m
I do not know if we need to make some necessary changes on Debian or Ubuntu. And It looks there is nothing about this part. Some people said, at least, we need to • disable swap • net.ipv4.ip_forward = 1 • net.bridge.bridge-nf-call-iptables = 1
For etcd, we need to keep all the machines with a synced time.
h
if you have not already - I would recommend starting here - it mentions some of the things you are asking for: https://docs.rke2.io/install/requirements
1
and keep an eye on support matrix: https://www.suse.com/suse-rke2/support-matrix/all-supported-versions/rke2-v1-30/ highly recommend reviewing that before doing OS / kubernetes updates
1
m
Thanks @hundreds-evening-84071 I read through the docs you sent before. what I am curious is: we do not even need to disable swap. Is it correct?
h
it used to be the case but I have not seen that requirement lately but that could be due to SSDs and NVME drives ---- full disclosure - I still disable SWAP on all my cluster nodes
1
m
That is why I have questions. From the official documentations, it looks RKE2 can help us handle everything, but I cannot find anything about the best practice or OS-level configurations. some people in the forums said we still need to refer to kubernetes official docs about this part.
w
@powerful-librarian-10572 why Docker? since RKE2 doesn't use Docker perhaps you're using RKE?
p
I don't know to be fair
Something install contnainerd maybe?
m
@witty-jelly-95845 Good afternoon, any suggestion for my questions in this session? thanks😃
h
I guess it also depends on your organizations requirements... We just install minimal OS and apply NIST controls this is required and only install what the installer complains about... so I do not have nfs-utils and docker etc (those are not required for RKE2)
1
m
Thanks @hundreds-evening-84071 Yes, it's true. For example, IP forwarding is necessary for Kubernetes pods to communicate across nodes, especially for services that rely on multi-host networking. If we do not have this, how does RKE2 do in this case. It looks RKE2 has done everything for us, but need to double confirm.
And if we use ipvs mode, the corresponding kernel should be loaded before we register the nodes. but I have not seen this in the official documents, maybe it is in somewhere else.
What I think is (I might be wrong): RKE2 is a Kubernetes distribution that bundles a lot of features (like containerd) for easier setup. However, some of the recommendations (such as kernel module loading or systemd limits) come from general Kubernetes best practices. These are not always required in every environment, but they help ensure stability, performance, and security in various scenarios. Or RKE2 will Not need these things any more?