Hey folks! Shameless plug for a Shallow Dive video on Docker Desktop, Minikube and K3D:

can I not pass the config using heredoc ?

cat <<EOF | k3d cluster create -c -
apiVersion: <http://k3d.io/v1alpha4|k3d.io/v1alpha4>
kind: Simple
metadata:
  name: my-cluster
servers: 1
agents: 2
image: rancher/v1.24.3-k3s1
ports:
  # Drone CI
  - port: 127.0.0.1:30980:30980
    nodeFilters:
      - agent:*
  # Gitea
  - port: 127.0.0.1:30950:30950
    nodeFilters:
      - agent:*
  # Argo CD
  - port: 127.0.0.1:30080:30080
    nodeFilters:
      - agent:*
registries: 
    name: "${reg_name}"
    host: "0.0.0.0"
    hostPort: "${reg_port}"
    volumes:
      - "${PWD}/.k3s/registry":/var/lib/registry
EOF

Any recommendations for how to get hot-reloading for development? From initial googling I've found these alternatives: • devspace: https://devspace.sh/#development • garden: https://github.com/garden-io/garden/ • ksync: https://github.com/ksync/ksync • kubectl-warp: https://github.com/ernoaapa/kubectl-warp • kubed-sh: https://kubed.sh/ • okteto cli: https://www.okteto.com/docs/cloud/okteto-cli/ • skaffold: https://skaffold.dev/ • telepresence: https://www.telepresence.io/ • tilt: https://docs.tilt.dev/index.html Anyone here using any of these with k3d? I'd like to get hot reload working for python (fastapi) and typescript (nextjs).

I followed guide https://k3d.io/v5.4.4/usage/registries/#preface-referencing-local-registries to configure the local registry

👋 Hello, team!, I am trying to deploy an app on

k3d

via helm where cluster is created as below -

k3d cluster create --agents 2 test-cluster \
  --api-port 6445 \
  --port '8585:8585@loadbalancer' \
  --port '2376:2376@loadbalancer' \
  --port '80:80@loadbalancer' \
  --port '443:443@loadbalancer'

My app is using ports 80,443,2376 and 8585. All pods are up and running fine but when I am trying to access my application, on 80/443 I am getting a

404 not found

error from

traefik

, I can not change application port from 80/443 to some other port as there are some internal micro-services dependency. Can you help me to understand what might be wrong here?

is the LB(traefik) ip that is assigned to the services of LB type cant be reached like

TRAEFIK_LB_IP:<port>

from the host ?

Is this right way to mirror and cache images locally with k3d

registries:
  mirrors:
   "<http://docker.io|docker.io>":
     endpoint:
       - "<https://registry-1.docker.io>"
   "<http://quay.io|quay.io>":
     endpoint:
       - "<https://quay.io>"
  create:
    name: "k3d-myregistry.localhost"
    host: "0.0.0.0"
    hostPort: "5001"
    volumes:
      - "<localpath>/.k3s/registry:/var/lib/registry"

and then in all my references I use

k3d-myregistry.localhost:5000

is there a way to configure

insecure

registry with k3d using registry k8s service on my cluster

hey👋 is anyone running k3s on oracle oci cloud ( arm ) ? how can i bind api server to external public ip which is NAT-ed to vm instance? i tried --node-external-ip and --advertise-address but still can not connect to API (

no route to host

). Security Groups are ok, cuz if I delete iptable rules, its working correctly,....

Is it possible to setup a cluster with an external datastore using k3d?

Hi guys! I am fairly new to the whole kubernetes thing but figured it would be good to start labbing with k3d. However, after following the guide for setting up a cluster with a local registry (https://k3d.io/v5.4.4/usage/registries/) I ran into some trouble. The steps I have taken are:

k3d registry create registry.localhost --port 12345
k3d cluster create mycluster --registry-use k3d-registry.localhost:12345

docker pull nginx:latest
docker tag nginx:latest k3d-registry.localhost:12345/nginx:latest
docker push k3d-registry.localhost:12345/nginx:latest

cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-test-registry
  labels:
    app: nginx-test-registry
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-test-registry
  template:
    metadata:
      labels:
        app: nginx-test-registry
    spec:
      containers:
      - name: nginx-test-registry
        image: k3d-registry.localhost:12345/nginx:latest
        ports:
        - containerPort: 80
EOF

No errors so far, but when I run

kubectl describe pod nginx-test-registry

the event log looks like:

...

Events:
  Type     Reason                  Age   From               Message
  ----     ------                  ----  ----               -------
  Normal   Scheduled               13s   default-scheduler  Successfully assigned default/nginx-test-registry-5ff799bb8f-6mk6z to k3d-mycluster-server-0
  Warning  FailedCreatePodSandBox  3s    kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to get sandbox image "rancher/mirrored-pause:3.6":
                                                              failed to pull image "rancher/mirrored-pause:3.6": failed to pull and unpack image "<http://docker.io/rancher/mirrored-pause:3.6|docker.io/rancher/mirrored-pause:3.6>": 
                                                                failed to resolve reference "<http://docker.io/rancher/mirrored-pause:3.6|docker.io/rancher/mirrored-pause:3.6>": 
                                                                  failed to do request: Head "<https://registry-1.docker.io/v2/rancher/mirrored-pause/manifests/3.6>": dial tcp: lookup <http://registry-1.docker.io|registry-1.docker.io> on 127.0.0.11:53: read udp 127.0.0.1:34330->127.0.0.11:53: i/o timeout

I reckon that there is a problem with my network config on my host machine but I'm not sure and don't know where to start looking. Any ideas of what my problem is and how to fix it?

in some cases to do docker builds within a dind container I need to set mtu, how to know the right mtu value to set for a k3s cluster created with k3d

Is there a way to export images from an existing k3d based cluster

Does anyone know if you can use K3D to create multiple clusters from a single config file?

Has anyone tried to use multus cni on k3d?

Hello Is there easy and straightforward way to provide different helm chart values and helm chart version of k3d traefik after the installation without recreating k3d cluster

what is the best way to get the kubeconfig for k3d based cluster from within in another container ?

I'm using https://github.com/nolar/setup-k3d-k3s to run k3d in a github actions workflow. This mostly works well, but when something fails, I find that it's very difficult to reliably output the logs of a running container. If I do

kubectl logs ...

, sometimes I get output from that, but mostly I don't. Have anyone experienced anything similar?

hello — I’m trying to setup calico with k3d cluster — can anyone share their calico.yaml and any notes they may have?

how can we use nodeip with k3d cluster, I mean trying to reach a service of the cluster from say another container of the same docker network ?

🆕 https://github.com/k3d-io/k3d/releases/tag/v5.4.6 🆕

I'm looking for a way to create a cluster ONLY if it does not exist, otherwise start it. Is it possible to have

k3d cluster create --config=blah.yaml

behave this way (or

k3d cluster start

Also is there a way to configure the certificate authority for the cluster? (e.g. for local dev I want to share the ca cert so it can be used to authorize to Hashicorp Vault)

Is there a way to map forwarded ports to a specific IP address when creating a cluster? I have a made-up domain for development (

dev.local

) and I've previously made this work by adding a HOSTS entry to map to an always-consistent IP (

192.168.64.200

) and then having MetalLB + Traefik's Ingress LB mapped to that IP specifically on port 80. In my head that means I should be able to:

k3d cluster create foo-cloud --no-lb --k3s-arg="--disable=traefik@server:0" -p "192.168.64.200:80:80"

but k3d doesn't seem to agree:

FATA[0000] failed to transform ports: No nodefilters specified

Is there a more appropriate way to accomplish this?

How can I find out where the local storage provisioner is putting files? I want to use a host volume for some storage that would persist between cluster deletes (e.g. postgres db), but I don't know how to go about that?

Also is there a way to specify the current directory in the volume paths in a k3d config? I want to specify something like `pwd`/abc/def/:/var/lib/rancher/k3s/storage/ but I can't figure out how to add the current directory to the config without templating it.

Can I list the docker images available within the cluster, somehow?

is there anyway to prevent workload pods from scheduling on server (master) node? im using k3d v5.4.6. As you can tell one of the Pods is on

k3d-dev-server-0

node.

k3d cluster create dev --agents 2 --servers 1

kubectl get no
NAME               STATUS   ROLES                  AGE     VERSION
k3d-dev-agent-0    Ready    <none>                 2m13s   v1.24.4+k3s1
k3d-dev-agent-1    Ready    <none>                 2m13s   v1.24.4+k3s1
k3d-dev-server-0   Ready    control-plane,master   2m18s   v1.24.4+k3s1

kubectl create deploy web --image=nginx:alpine --replicas=3

kubectl get po -owide
NAME                   READY   STATUS    RESTARTS   AGE   IP          NODE               NOMINATED NODE   READINESS GATES
web-654c99587f-h9hr5   1/1     Running   0          27s   10.42.0.5   k3d-dev-server-0   <none>           <none>
web-654c99587f-xwkjg   1/1     Running   0          27s   10.42.1.5   k3d-dev-agent-0    <none>           <none>
web-654c99587f-4z45l   1/1     Running   0          27s   10.42.2.5   k3d-dev-agent-1    <none>           <none>

Hi all, I am evaluating K3d with Neuvector. Does anybody have a clue how to get this up and running? I have the following config

apiVersion: <http://k3d.io/v1alpha4|k3d.io/v1alpha4>
kind: Simple
metadata:
  name: test
servers: 1
agents: 2
kubeAPI:
  host: localhost
  hostIP: 0.0.0.0
  hostPort: "6443"
volumes:
  - volume: /tmp:/tmp/host
    nodeFilters:
      - all
ports:
  - port: 0.0.0.0:8080:80
    nodeFilters:
      - loadbalancer
  - port: 0.0.0.0:8443:443
    nodeFilters:
      - loadbalancer
options:
  k3d:
    wait: true
    timeout: "150s"
    disableImageVolume: false
    disableLoadbalancer: false
    disableRollback: false
  k3s:
    extraArgs:
      - arg: --tls-san=127.0.0.1
        nodeFilters:
          - server:*
  kubeconfig:
    updateDefaultKubeconfig: true
    switchCurrentContext: true

I have installed Neuvector with helm

NAMESPACE="neuvector"

helm upgrade neuvector neuvector/core -n ${NAMESPACE} \
  --install \
  --create-namespace \
  --set k3s.enabled=true \
  --set manager.svc.type="ClusterIP"

It is up and running, but I am not able to get the UI of the management pod. Not with a port forward and not on the service. Any help would be appreciated. Thanks!

Hi everyone, trying to figure out if k3d is the right tool for what I'm looking to do. I have a single host I want to be able to run multiple kubernetes clusters on. K3d can clearly create multiple clusters but is it possible to setup MetalLB to make things in each cluster accessible to other machines on my network?