many-journalist-64069
11/26/2022, 4:18 AMmany-journalist-64069
11/26/2022, 4:18 AMmany-journalist-64069
11/26/2022, 4:20 AMINSTALL_K3S_MIRROR=cn K3S_NODE_NAME=cubmaster01 K3S_KUBECONFIG_OUTPUT=/home/escape/.kube/config INSTALL_K3S_EXEC="--docker" | sh install.sh
agent:
INSTALL_K3S_MIRROR=cn K3S_NODE_NAME=cubnode01 K3S_KUBECONFIG_OUTPUT=/home/escape/.kube/config K3S_URL=<https://192.168.71.130:6443> K3S_TOKEN=K100.... sh install.sh
many-journalist-64069
11/26/2022, 4:24 AMjolly-waitress-71272
11/28/2022, 5:27 PMkubectl create secret generic kubeconfig --from-file=/etc/rancher/k3s/k3s.yaml
Is there a manifest (or literally any file at all) created somewhere I can reference? I'm trying to ansibilize it and I just want to run it once, with a creates
directive. So if there was a manifest, that'd be super easy.
I need to do this for some argo shenanigans if you're curious why I would yo dawg my kubeconfig.best-accountant-61831
11/28/2022, 8:02 PMsticky-summer-13450
11/29/2022, 3:08 PMTerminating
state and I want to know whether it's k3s, k8s, or me.
Example: I have cluster with 1 server
node and several worker
nodes, and I have workloads spread across the workers. Lets say a worker node dies - maybe it's never going to return.
$ kubectl get pods --context kube001 --all-namespaces -o=wide |grep Terminating
kube-system traefik-9c6dc6686-jdt9f 1/1 Terminating 0 24d 10.42.1.4 kube002 <none> <none>
active-mq active-mq-6665f5d8b9-ztwnq 1/1 Terminating 0 15d 10.42.1.82 kube002 <none> <none>
Some of the pods get stuck in the terminating state and don't get replaced on other worker nodes. This means the cluster is no-longer respecting the declarative state.
Is this a problem specific to me, a problem specific to k3s, a problem with k8s, or something else?faint-tomato-17392
11/30/2022, 7:36 AMfamous-flag-15098
11/30/2022, 10:32 PMcreamy-room-58344
12/02/2022, 9:07 PMwaiting for cluster agent to connect
Rancher v2.7.0
Machine Provider: VMware vSphere
Kubernetes version: v1.24.8+k3s+1
pool1: 3 master+etcd nodes
pool2: 4 worker nodes
Nodes image: Ubuntu Cloud 22.04 (same as used in RKE1 and RKE2)
Conditions shows:
Ready False 2.7 hours ago [Disconnected] Cluster agent is not connected
Reconciling True 2.7 hours ago [Reconciling]
RKECluster True 2.6 hours ago —
SecretsMigrated True 2.7 hours ago —
ServiceAccountSecretsMigrated True 2.7 hours ago —
Stalled False 2.7 hours ago —
SystemProjectCreated True 2.7 hours ago —
Updated Unknown 2.6 hours ago [Waiting] configuring bootstrap node(s) k3s-lab01-master-789d65b648-fxft7: waiting for cluster agent to connect
I'm able to deploy RKE1 and RKE2 clusters
What other info can I provide to help solve this issue?gentle-advantage-38637
12/03/2022, 2:06 PMprehistoric-diamond-4224
12/04/2022, 3:13 PMenough-carpet-20915
12/04/2022, 6:44 PMapiVersion: <http://helm.cattle.io/v1|helm.cattle.io/v1>
kind: HelmChartConfig
metadata:
name: traefik
namespace: kube-system
spec:
valuesContent: |-
additionalArguments:
- "--entrypoints.minecraft.address=:25565/tcp"
ports:
- containerPort: 25565
name: minecraft
protocol: TCP
I get this error from the helm-install-traefik-$foo pod:
Error: UPGRADE FAILED: template: traefik/templates/service.yaml:10:27: executing "traefik/templates/service.yaml" at <$name>: wrong type for value; expected string; got int
What am I doing wrong here?bright-london-1095
12/06/2022, 8:55 AM1.22 to 1.24
. and i have few questions in my mind
1. Is it fine to upgrade from 1.22 to 1.24 ?
2. Which minor version of 1.24.x
is a stable one to use it production k3s
cluster ?
3. I believe it is traefik-v2.x
will be shipped along with 1.24.x ?
TIAlively-night-78214
12/06/2022, 1:51 PMlively-night-78214
12/06/2022, 1:51 PMquiet-memory-19288
12/06/2022, 5:37 PMable-mechanic-45652
12/07/2022, 6:55 AMlively-night-78214
12/07/2022, 1:51 PMwide-author-88664
12/07/2022, 4:10 PMwide-author-88664
12/07/2022, 4:11 PMlively-night-78214
12/07/2022, 5:56 PMlively-night-78214
12/08/2022, 5:09 AMservers
actually run Traefik and this acts as the load balancer. I am planning to use a single server so I would like to understand if i actually need to spin my own load balancer solution, how and if not if indeed the loadbalancer is actually withing the Server Nodelively-night-78214
12/08/2022, 7:37 AMcurl -sfL <https://get.k3s.io> | INSTALL_K3S_VERSION=v1.25.4+k3s1 sh -s - server --datastore-endpoint=\"<mysql://user:pass@tcp>(host:3306)/my_k3s_test
STD out shows
"[INFO] Using v1.25.4+k3s1 as release",
"[INFO] Downloading hash <https://github.com/k3s-io/k3s/releases/download/v1.25.4+k3s1/sha256sum-amd64.txt>",
"[INFO] Downloading binary <https://github.com/k3s-io/k3s/releases/download/v1.25.4+k3s1/k3s>",
"[INFO] Verifying binary download",
"[INFO] Installing k3s to /usr/local/bin/k3s",
"[INFO] Skipping installation of SELinux RPM",
"[INFO] Creating /usr/local/bin/kubectl symlink to k3s",
"[INFO] Creating /usr/local/bin/crictl symlink to k3s",
"[INFO] Creating /usr/local/bin/ctr symlink to k3s",
"[INFO] Creating killall script /usr/local/bin/k3s-killall.sh",
"[INFO] Creating uninstall script /usr/local/bin/k3s-uninstall.sh",
"[INFO] env: Creating environment file /etc/systemd/system/k3s.service.env",
"[INFO] systemd: Creating service file /etc/systemd/system/k3s.service",
"[INFO] systemd: Enabling k3s unit",
"[INFO] systemd: Starting k3s"
However in std_err
"Created symlink /etc/systemd/system/multi-user.target.wants/k3s.service → /etc/systemd/system/k3s.service.",
"Job for k3s.service failed because the control process exited with error code.",
"See \"systemctl status k3s.service\" and \"journalctl -xe\" for details."
If i go to the server and run
ubuntu@my-k3s-server:~$ crictl version
FATA[0000] load config file: stat /var/lib/rancher/k3s/data/7c994f47fd344e1637da337b92c51433c255b387d207b30b3e0262779457afe4/bin/crictl.yaml: no such file or directory
From the journal
ubuntu@my-k3s-server:~$ journalctl -xe
Hint: You are currently not seeing messages from other users and the system.
Users in groups 'adm', 'systemd-journal' can see all messages.
Pass -q to turn off this notice.
No journal files were opened due to insufficient permissions.
ubuntu@myreviews-k3s-server:~$ sudo journalctl -xe
-- Subject: Automatic restarting of a unit has been scheduled
-- Defined-By: systemd
-- Support: <http://www.ubuntu.com/support>
--
-- Automatic restarting of the unit k3s.service has been scheduled, as the result for
-- the configured Restart= setting for the unit.
Dec 08 07:24:52 myreviews-k3s-server systemd[1]: Stopped Lightweight Kubernetes.
-- Subject: Unit k3s.service has finished shutting down
-- Defined-By: systemd
-- Support: <http://www.ubuntu.com/support>
--
-- Unit k3s.service has finished shutting down.
Dec 08 07:24:52 myreviews-k3s-server systemd[1]: Starting Lightweight Kubernetes...
-- Subject: Unit k3s.service has begun start-up
-- Defined-By: systemd
-- Support: <http://www.ubuntu.com/support>
--
-- Unit k3s.service has begun starting up.
Dec 08 07:24:52 myreviews-k3s-server sh[4543]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Dec 08 07:24:52 myreviews-k3s-server sh[4543]: /bin/sh: 1: /usr/bin/systemctl: not found
Dec 08 07:24:52 myreviews-k3s-server k3s[4555]: time="2022-12-08T07:24:52Z" level=info msg="Starting k3s v1.25.4+k3s1 (0dc63334)"
Dec 08 07:24:54 myreviews-k3s-server k3s[4555]: time="2022-12-08T07:24:54Z" level=info msg="Configuring mysql database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
Dec 08 07:24:54 myreviews-k3s-server k3s[4555]: time="2022-12-08T07:24:54Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
Dec 08 07:24:55 myreviews-k3s-server k3s[4555]: time="2022-12-08T07:24:55Z" level=info msg="Database tables and indexes are up to date"
Dec 08 07:24:56 myreviews-k3s-server sudo[4573]: ubuntu : TTY=pts/0 ; PWD=/home/ubuntu ; USER=root ; COMMAND=/bin/journalctl -xe
Dec 08 07:24:56 myreviews-k3s-server sudo[4573]: pam_unix(sudo:session): session opened for user root by ubuntu(uid=0)
From system CTL
systemctl status k3s.service
● k3s.service - Lightweight Kubernetes
Loaded: loaded (/etc/systemd/system/k3s.service; enabled; vendor preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Thu 2022-12-08 07:24:18 UTC; 3s ago
Docs: <https://k3s.io>
Process: 4373 ExecStart=/usr/local/bin/k3s server --datastore-endpoint=<mysql://OMISSIS:3306>)/my_k3s_test (code=exited, status=1/FAILURE)
Process: 4372 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
Process: 4370 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
Process: 4363 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service (code=exited, status=0/SUCCESS)
Main PID: 4373 (code=exited, status=1/FAILURE)
What am I doing wrong?lively-night-78214
12/08/2022, 9:56 AMClient Version: <http://version.Info|version.Info>{Major:"1", Minor:"18", GitVersion:"v1.18.2", GitCommit:"52c56ce7a8272c798dbc29846288d7cd9fbae032", GitTreeState:"clean", BuildDate:"2020-04-16T11:56:40Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Unable to connect to the server: x509: certificate is valid for kubernetes, kubernetes.default, kubernetes.default.svc, kubernetes.default.svc.cluster.local, localhost, myapp-k3s-server, not <http://k3s.domain.space|k3s.domain.space>
aloof-dusk-52700
12/08/2022, 6:17 PMcreamy-hospital-75658
12/08/2022, 11:05 PMquiet-chef-27276
12/09/2022, 5:22 AMchilly-telephone-51989
12/09/2022, 10:36 AMk -n kube-system logs traefik-7cd4fcff68-49cn2 -f
time="2022-12-09T09:42:28Z" level=info msg="Configuration loaded from flags."
time="2022-12-09T09:42:59Z" level=error msg="Error watching kubernetes events: could not retrieve server version: Get \"<https://10.43.0.1:443/version?timeout=32s>\": dial tcp 10.43.0.1:443: i/o timeout" providerName=kubernetes
I1209 09:42:59.042242 1 trace.go:205] Trace[1457098512]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167 (09-Dec-2022 09:42:29.039) (total time: 30002ms):
Trace[1457098512]: [30.002607632s] [30.002607632s] END
E1209 09:42:59.042279 1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1alpha1.IngressRoute: failed to list *v1alpha1.IngressRoute: Get "<https://10.43.0.1:443/apis/traefik.containo.us/v1alpha1/ingressroutes?limit=500&resourceVersion=0>": dial tcp 10.43.0.1:443: i/o timeout
I1209 09:42:59.042377 1 trace.go:205] Trace[1530918296]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167 (09-Dec-2022 09:42:29.041) (total time: 30000ms):
Trace[1530918296]: [30.00042368s] [30.00042368s] END
E1209 09:42:59.042389 1 reflector.go:138] pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167: Failed to watch *v1alpha1.MiddlewareTCP: failed to list *v1alpha1.MiddlewareTCP: Get "<https://10.43.0.1:443/apis/traefik.containo.us/v1alpha1/middlewaretcps?limit=500&resourceVersion=0>": dial tcp 10.43.0.1:443: i/o timeout
I1209 09:42:59.042468 1 trace.go:205] Trace[649799410]: "Reflector ListAndWatch" name:pkg/mod/k8s.io/client-go@v0.22.1/tools/cache/reflector.go:167 (09-Dec-2022 09:42:29.042) (total time: 30000ms):
Trace[649799410]: [30.000317588s] [30.000317588s] END
[3:00 PM] please note that requests are not being passed to the gateway or any other pod. here is my ingress file:
ingress.middleware.yaml
apiVersion: <http://traefik.containo.us/v1alpha1|traefik.containo.us/v1alpha1>
kind: Middleware
metadata:
name: strip-path
namespace: xplorie
spec:
stripPrefix:
prefixes:
- /api
[3:00 PM] ingress.yaml
apiVersion: <http://networking.k8s.io/v1|networking.k8s.io/v1>
kind: Ingress
metadata:
name: ingress
namespace: xplorie
annotations:
<http://traefik.ingress.kubernetes.io/router.middlewares|traefik.ingress.kubernetes.io/router.middlewares>: xplorie-strip-path@kubernetescrd
spec:
rules:
- http:
paths:
- path: "/api"
pathType: Prefix
backend:
service:
name: gateway
port:
number: 80
- path: "/"
pathType: Prefix
backend:
service:
name: portal
port:
number: 80