https://rancher.com/ logo
Join the conversationJoin Slack
Channels
academy
amazon
arm
azure
cabpr
chinese
ci-cd
danish
deutsch
developer
elemental
epinio
espanol
events
extensions
fleet
français
gcp
general
harvester
harvester-dev
hobbyfarm
hypper
japanese
k3d
k3os
k3s
k3s-contributor
kim
kubernetes
kubewarden
lima
logging
longhorn-dev
longhorn-storage
masterclass
mesos
mexico
nederlands
neuvector-security
office-hours
one-point-x
onlinemeetup
onlinetraining
opni
os
ozt
phillydotnet
portugues
rancher-desktop
rancher-extensions
rancher-setup
rancher-wrangler
random
rfed_ara
rio
rke
rke2
russian
s3gw
service-mesh
storage
submariner
supermicro-sixsq
swarm
terraform-controller
terraform-provider-rancher2
terraform-provider-rke
theranchcast
training-0110
training-0124
training-0131
training-0207
training-0214
training-1220
ukranian
v16-v21-migration
vsphere
windows
Powered by Linen
k3d
  • s

    swift-monitor-11764

    08/10/2022, 1:46 AM
    Hey folks! Shameless plug for a Shallow Dive video on Docker Desktop, Minikube and K3D:

    https://www.youtube.com/watch?v=oeIp3MsI3Uc▾

    👍 1
  • p

    polite-engineer-55788

    08/10/2022, 5:06 AM
    can I not pass the config using heredoc ?
    cat <<EOF | k3d cluster create -c -
    apiVersion: <http://k3d.io/v1alpha4|k3d.io/v1alpha4>
    kind: Simple
    metadata:
      name: my-cluster
    servers: 1
    agents: 2
    image: rancher/v1.24.3-k3s1
    ports:
      # Drone CI
      - port: 127.0.0.1:30980:30980
        nodeFilters:
          - agent:*
      # Gitea
      - port: 127.0.0.1:30950:30950
        nodeFilters:
          - agent:*
      # Argo CD
      - port: 127.0.0.1:30080:30080
        nodeFilters:
          - agent:*
    registries: 
        name: "${reg_name}"
        host: "0.0.0.0"
        hostPort: "${reg_port}"
        volumes:
          - "${PWD}/.k3s/registry":/var/lib/registry
    EOF
    w
    c
    • 3
    • 9
  • m

    melodic-market-42092

    08/10/2022, 7:28 AM
    Any recommendations for how to get hot-reloading for development? From initial googling I've found these alternatives: • devspace: https://devspace.sh/#development • garden: https://github.com/garden-io/garden/ • ksync: https://github.com/ksync/ksync • kubectl-warp: https://github.com/ernoaapa/kubectl-warp • kubed-sh: https://kubed.sh/ • okteto cli: https://www.okteto.com/docs/cloud/okteto-cli/ • skaffold: https://skaffold.dev/ • telepresence: https://www.telepresence.io/ • tilt: https://docs.tilt.dev/index.html Anyone here using any of these with k3d? I'd like to get hot reload working for python (fastapi) and typescript (nextjs).
    w
    • 2
    • 5
  • p

    polite-engineer-55788

    08/10/2022, 2:08 PM
    I followed guide https://k3d.io/v5.4.4/usage/registries/#preface-referencing-local-registries to configure the local registry
    w
    • 2
    • 24
  • f

    fierce-magician-45581

    08/11/2022, 12:55 AM
    👋 Hello, team!, I am trying to deploy an app on
    k3d
    via helm where cluster is created as below -
    k3d cluster create --agents 2 test-cluster \
      --api-port 6445 \
      --port '8585:8585@loadbalancer' \
      --port '2376:2376@loadbalancer' \
      --port '80:80@loadbalancer' \
      --port '443:443@loadbalancer'
    My app is using ports 80,443,2376 and 8585. All pods are up and running fine but when I am trying to access my application, on 80/443 I am getting a
    404 not found
    error from
    traefik
    , I can not change application port from 80/443 to some other port as there are some internal micro-services dependency. Can you help me to understand what might be wrong here?
    w
    m
    • 3
    • 7
  • p

    polite-engineer-55788

    08/12/2022, 3:37 PM
    is the LB(traefik) ip that is assigned to the services of LB type cant be reached like
    TRAEFIK_LB_IP:<port>
    from the host ?
    w
    • 2
    • 3
  • p

    polite-engineer-55788

    08/13/2022, 2:45 AM
    Is this right way to mirror and cache images locally with k3d
    registries:
      mirrors:
       "<http://docker.io|docker.io>":
         endpoint:
           - "<https://registry-1.docker.io>"
       "<http://quay.io|quay.io>":
         endpoint:
           - "<https://quay.io>"
      create:
        name: "k3d-myregistry.localhost"
        host: "0.0.0.0"
        hostPort: "5001"
        volumes:
          - "<localpath>/.k3s/registry:/var/lib/registry"
    and then in all my references I use
    k3d-myregistry.localhost:5000
    • 1
    • 5
  • p

    polite-engineer-55788

    08/16/2022, 12:21 PM
    is there a way to configure
    insecure
    registry with k3d using registry k8s service on my cluster
    w
    • 2
    • 20
  • r

    red-queen-31013

    08/17/2022, 8:36 AM
    hey👋 is anyone running k3s on oracle oci cloud ( arm ) ? how can i bind api server to external public ip which is NAT-ed to vm instance? i tried --node-external-ip and --advertise-address but still can not connect to API (
    no route to host
    ). Security Groups are ok, cuz if I delete iptable rules, its working correctly,....
  • h

    high-morning-12231

    08/17/2022, 9:44 PM
    Is it possible to setup a cluster with an external datastore using k3d?
    w
    • 2
    • 4
  • d

    dazzling-zoo-91458

    08/18/2022, 7:32 AM
    Hi guys! I am fairly new to the whole kubernetes thing but figured it would be good to start labbing with k3d. However, after following the guide for setting up a cluster with a local registry (https://k3d.io/v5.4.4/usage/registries/) I ran into some trouble. The steps I have taken are:
    k3d registry create registry.localhost --port 12345
    k3d cluster create mycluster --registry-use k3d-registry.localhost:12345
    
    docker pull nginx:latest
    docker tag nginx:latest k3d-registry.localhost:12345/nginx:latest
    docker push k3d-registry.localhost:12345/nginx:latest
    
    cat <<EOF | kubectl apply -f -
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx-test-registry
      labels:
        app: nginx-test-registry
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: nginx-test-registry
      template:
        metadata:
          labels:
            app: nginx-test-registry
        spec:
          containers:
          - name: nginx-test-registry
            image: k3d-registry.localhost:12345/nginx:latest
            ports:
            - containerPort: 80
    EOF
    No errors so far, but when I run
    kubectl describe pod nginx-test-registry
    the event log looks like:
    ...
    
    Events:
      Type     Reason                  Age   From               Message
      ----     ------                  ----  ----               -------
      Normal   Scheduled               13s   default-scheduler  Successfully assigned default/nginx-test-registry-5ff799bb8f-6mk6z to k3d-mycluster-server-0
      Warning  FailedCreatePodSandBox  3s    kubelet            Failed to create pod sandbox: rpc error: code = Unknown desc = failed to get sandbox image "rancher/mirrored-pause:3.6":
                                                                  failed to pull image "rancher/mirrored-pause:3.6": failed to pull and unpack image "<http://docker.io/rancher/mirrored-pause:3.6|docker.io/rancher/mirrored-pause:3.6>": 
                                                                    failed to resolve reference "<http://docker.io/rancher/mirrored-pause:3.6|docker.io/rancher/mirrored-pause:3.6>": 
                                                                      failed to do request: Head "<https://registry-1.docker.io/v2/rancher/mirrored-pause/manifests/3.6>": dial tcp: lookup <http://registry-1.docker.io|registry-1.docker.io> on 127.0.0.11:53: read udp 127.0.0.1:34330->127.0.0.11:53: i/o timeout
    I reckon that there is a problem with my network config on my host machine but I'm not sure and don't know where to start looking. Any ideas of what my problem is and how to fix it?
    w
    • 2
    • 5
  • p

    polite-engineer-55788

    08/19/2022, 3:09 AM
    in some cases to do docker builds within a dind container I need to set mtu, how to know the right mtu value to set for a k3s cluster created with k3d
    🙌 1
    w
    • 2
    • 3
  • p

    polite-engineer-55788

    08/19/2022, 3:12 AM
    Is there a way to export images from an existing k3d based cluster
    w
    • 2
    • 30
  • b

    brash-businessperson-80975

    08/19/2022, 11:06 PM
    Does anyone know if you can use K3D to create multiple clusters from a single config file?
    m
    w
    • 3
    • 6
  • f

    flat-glass-90449

    08/20/2022, 1:04 PM
    Has anyone tried to use multus cni on k3d?
  • m

    millions-alarm-86298

    08/22/2022, 8:29 AM
    Hello Is there easy and straightforward way to provide different helm chart values and helm chart version of k3d traefik after the installation without recreating k3d cluster
    w
    • 2
    • 3
  • p

    polite-engineer-55788

    08/22/2022, 5:08 PM
    what is the best way to get the kubeconfig for k3d based cluster from within in another container ?
    w
    • 2
    • 16
  • m

    melodic-market-42092

    08/23/2022, 6:36 AM
    I'm using https://github.com/nolar/setup-k3d-k3s to run k3d in a github actions workflow. This mostly works well, but when something fails, I find that it's very difficult to reliably output the logs of a running container. If I do
    kubectl logs ...
    , sometimes I get output from that, but mostly I don't. Have anyone experienced anything similar?
  • c

    clever-pizza-61378

    08/25/2022, 1:49 AM
    hello — I’m trying to setup calico with k3d cluster — can anyone share their calico.yaml and any notes they may have?
    f
    • 2
    • 2
  • p

    polite-engineer-55788

    08/25/2022, 8:32 AM
    how can we use nodeip with k3d cluster, I mean trying to reach a service of the cluster from say another container of the same docker network ?
    b
    w
    • 3
    • 11
  • w

    wide-garage-9465

    08/29/2022, 10:52 AM
    🆕 https://github.com/k3d-io/k3d/releases/tag/v5.4.6 🆕
    🎊 1
    🎉 1
  • f

    fancy-angle-74076

    08/31/2022, 5:11 PM
    I'm looking for a way to create a cluster ONLY if it does not exist, otherwise start it. Is it possible to have
    k3d cluster create --config=blah.yaml
    behave this way (or
    k3d cluster start
  • f

    fancy-angle-74076

    08/31/2022, 5:28 PM
    Also is there a way to configure the certificate authority for the cluster? (e.g. for local dev I want to share the ca cert so it can be used to authorize to Hashicorp Vault)
  • q

    quick-midnight-5000

    08/31/2022, 8:36 PM
    Is there a way to map forwarded ports to a specific IP address when creating a cluster? I have a made-up domain for development (
    dev.local
    ) and I've previously made this work by adding a HOSTS entry to map to an always-consistent IP (
    192.168.64.200
    ) and then having MetalLB + Traefik's Ingress LB mapped to that IP specifically on port 80. In my head that means I should be able to:
    k3d cluster create foo-cloud --no-lb --k3s-arg="--disable=traefik@server:0" -p "192.168.64.200:80:80"
    but k3d doesn't seem to agree:
    FATA[0000] failed to transform ports: No nodefilters specified
    Is there a more appropriate way to accomplish this?
    w
    • 2
    • 9
  • f

    fancy-angle-74076

    09/06/2022, 5:48 PM
    How can I find out where the local storage provisioner is putting files? I want to use a host volume for some storage that would persist between cluster deletes (e.g. postgres db), but I don't know how to go about that?
    w
    • 2
    • 3
  • f

    fancy-angle-74076

    09/06/2022, 5:57 PM
    Also is there a way to specify the current directory in the volume paths in a k3d config? I want to specify something like `pwd`/abc/def/:/var/lib/rancher/k3s/storage/ but I can't figure out how to add the current directory to the config without templating it.
    w
    • 2
    • 1
  • m

    melodic-market-42092

    09/08/2022, 9:39 AM
    Can I list the docker images available within the cluster, somehow?
    w
    • 2
    • 15
  • b

    breezy-electrician-44168

    09/14/2022, 8:35 PM
    is there anyway to prevent workload pods from scheduling on server (master) node? im using k3d v5.4.6. As you can tell one of the Pods is on
    k3d-dev-server-0
    node.
    k3d cluster create dev --agents 2 --servers 1
    
    kubectl get no
    NAME               STATUS   ROLES                  AGE     VERSION
    k3d-dev-agent-0    Ready    <none>                 2m13s   v1.24.4+k3s1
    k3d-dev-agent-1    Ready    <none>                 2m13s   v1.24.4+k3s1
    k3d-dev-server-0   Ready    control-plane,master   2m18s   v1.24.4+k3s1
    
    kubectl create deploy web --image=nginx:alpine --replicas=3
    
    kubectl get po -owide
    NAME                   READY   STATUS    RESTARTS   AGE   IP          NODE               NOMINATED NODE   READINESS GATES
    web-654c99587f-h9hr5   1/1     Running   0          27s   10.42.0.5   k3d-dev-server-0   <none>           <none>
    web-654c99587f-xwkjg   1/1     Running   0          27s   10.42.1.5   k3d-dev-agent-0    <none>           <none>
    web-654c99587f-4z45l   1/1     Running   0          27s   10.42.2.5   k3d-dev-agent-1    <none>           <none>
    c
    w
    m
    • 4
    • 8
  • p

    purple-dream-76106

    09/16/2022, 2:46 PM
    Hi all, I am evaluating K3d with Neuvector. Does anybody have a clue how to get this up and running? I have the following config
    apiVersion: <http://k3d.io/v1alpha4|k3d.io/v1alpha4>
    kind: Simple
    metadata:
      name: test
    servers: 1
    agents: 2
    kubeAPI:
      host: localhost
      hostIP: 0.0.0.0
      hostPort: "6443"
    volumes:
      - volume: /tmp:/tmp/host
        nodeFilters:
          - all
    ports:
      - port: 0.0.0.0:8080:80
        nodeFilters:
          - loadbalancer
      - port: 0.0.0.0:8443:443
        nodeFilters:
          - loadbalancer
    options:
      k3d:
        wait: true
        timeout: "150s"
        disableImageVolume: false
        disableLoadbalancer: false
        disableRollback: false
      k3s:
        extraArgs:
          - arg: --tls-san=127.0.0.1
            nodeFilters:
              - server:*
      kubeconfig:
        updateDefaultKubeconfig: true
        switchCurrentContext: true
    I have installed Neuvector with helm
    NAMESPACE="neuvector"
    
    helm upgrade neuvector neuvector/core -n ${NAMESPACE} \
      --install \
      --create-namespace \
      --set k3s.enabled=true \
      --set manager.svc.type="ClusterIP"
    It is up and running, but I am not able to get the UI of the management pod. Not with a port forward and not on the service. Any help would be appreciated. Thanks!
    b
    • 2
    • 2
  • g

    gorgeous-battery-45190

    09/24/2022, 7:29 PM
    Hi everyone, trying to figure out if k3d is the right tool for what I'm looking to do. I have a single host I want to be able to run multiple kubernetes clusters on. K3d can clearly create multiple clusters but is it possible to setup MetalLB to make things in each cluster accessible to other machines on my network?
    w
    • 2
    • 2
Powered by Linen
Title
g

gorgeous-battery-45190

09/24/2022, 7:29 PM
Hi everyone, trying to figure out if k3d is the right tool for what I'm looking to do. I have a single host I want to be able to run multiple kubernetes clusters on. K3d can clearly create multiple clusters but is it possible to setup MetalLB to make things in each cluster accessible to other machines on my network?
w

wide-garage-9465

09/25/2022, 8:34 AM
Hey 👋 It's definitely possible to use MetalLB on k3d, but you can also use simple port-forwarding built into k3d. What's your use-case?
g

gorgeous-battery-45190

09/25/2022, 4:24 PM
I have a few web applications I've built that at home are accessible via some CNAME. Today I've been able to use minikube with a none (bare metal) driver and MetalLB to easily expose all my apps which all listen on ports 80 and 443 but on their own IP. The management of the single cluster with all the different apps is kinda a pain so I'm looking for something that supports creating multiple clusters which k3d sounds perfect for so I stop accidentally deploying to the same namespace and breaking things 😅. I also don't want to have to create a ton of VMs and the resources of my machine should scale better with docker.
View count: 28