what is the best way to get the kubeconfig for k3d based clu rancher-users #k3d

Title

polite-engineer-55788

08/22/2022, 5:08 PM

what is the best way to get the kubeconfig for k3d based cluster from within in another container ?

wide-garage-9465

08/22/2022, 6:21 PM

Another docker container next to the cluster?

Just

docker cp

maybe?

polite-engineer-55788

08/23/2022, 4:29 AM

I am running a drone pipeline that will deploy apps to k3d cluster. as part of my step container I need to have access to the kubeconfig file, right now I am mounting the config from the source folder. but that has localhost/127… ip which might not work from within the container right?

I see kind has some thing like

kubeconfig get --internal

which gives the kube config with Docker IP range for API Server

I saw the k3d-demo for the drone CI example but for that the cluster gets stripped down after pipeline, in my case I need to deploy to the running cluster

wide-garage-9465

08/23/2022, 6:08 AM

So you have a persistent k3d cluster running on the Docker Host where you also run the DroneCI containers? You could open a Feature Request or start a PR to implement a similar flag to what kind has or make use of some grep/sed magic, since apparently you don't have to do it that often if you keep the cluster running? 🤔

👍🏽 1

➕ 1

polite-engineer-55788

08/23/2022, 4:06 PM

@wide-garage-9465 I did opened a PR for the

stdin

not sure if you had time to take a look at it

https://github.com/k3d-io/k3d/issues/1131

wide-garage-9465

08/24/2022, 10:13 AM

I'll get to your PR today 👍

🙌🏽 1

polite-engineer-55788

08/24/2022, 11:56 AM

@wide-garage-9465 I am trying to apply k8s on a standing cluster from within drone step like

- name: deploy app to k8s
  image: kameshsampath/kube-dev-tools
  pull: never
  commands:
    - k3d kubeconfig get my-demos > $KUBECONFIG
    - kubectl config set clusters.k3d-my-demos.server <http://host.k3d.internal:49244>
    - kubectl apply -f k8s/
  volumes:
    - name: dockersock
      path: /var/run/docker.sock

I see I cant resolve the

host.k3d.internal

any tip on how to reach the k8s API from step. container ??

I guess that the feature request but then thought to see any hacks available

I did this and its working. May be useful for some others

- name: deploy app to k8s
  image: kameshsampath/kube-dev-tools
  pull: never
  commands:
    - k3d kubeconfig get my-demos > $KUBECONFIG
    - export DOCKER_INTERNAL_IP=$(docker inspect k3d-my-demos-server-0 --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}')
    - kubectl config set clusters.k3d-my-demos.server https://$DOCKER_INTERNAL_IP:6443
    - kubectl apply -f k8s/
  volumes:
    - name: dockersock
      path: /var/run/docker.sock

wide-garage-9465

08/24/2022, 2:30 PM

host.k3d.internal

is only available within the k3d cluster (nodes + pods via coredns). Also, it's the same as

host.docker.internal

, so it gives you the IP of the docker host (which would work I guess since the API port is mapped to some port on your docker host).

polite-engineer-55788

08/24/2022, 4:10 PM

yeah on the host it should be some random port right. so its more concrete via

internal IP:6443

. I guess we can cover that in the feature req

btw thanks for merging my pr @wide-garage-9465 🙂

17 Views

#k3d Join Slack