This message was deleted.

Hi @hundreds-easter-25520, did you mean you re-install one of the harvester node from v1.2.0 -> v1.2.1 and encountered the problem with getting the kubeconfig? If not, could you describe it in more detail? Thanks! cc @red-king-19196, could you help to check it?

Sure. I had a one node cluster running 1.2.0 for my homelab, and decided to re-install rather than upgrade. This is a bare-metal install on a HP DL380, and has worked fine with both 1.1.1 and 1.2.0. However, when I do a fresh install of 1.2.1 on the hardware, everything appears to work fine, but when I go to Support -> Download Kubeconfig, the

local.yaml

that downloads a cluster cert that has the Subject line of

Subject: O = dynamiclistener-org, CN = dynamiclistener-ca@1698527436

and no SANs. When attempting to use the downloaded file as a kubeconfig I get

$ k get pod --kubeconfig ~/Downloads/local.yaml                                                                           <aws:saas-development-admin>
I1030 22:06:37.965645   58405 versioner.go:58] Get "<https://harvester.domain.local/k8s/clusters/local/version?timeout=5s>": x509: certificate is not valid for any names, but wanted to match harvester.domain.local
E1030 22:06:38.072280   58405 memcache.go:265] couldn't get current server API group list: Get "<https://harvester.domain.local/k8s/clusters/local/api?timeout=32s>": tls: failed to verify certificate: x509: certificate is not valid for any names, but wanted to match harvester.domain.local
E1030 22:06:38.089072   58405 memcache.go:265] couldn't get current server API group list: Get "<https://harvester.domain.local/k8s/clusters/local/api?timeout=32s>": tls: failed to verify certificate: x509: certificate is not valid for any names, but wanted to match harvester.domain.local
E1030 22:06:38.106775   58405 memcache.go:265] couldn't get current server API group list: Get "<https://harvester.domain.local/k8s/clusters/local/api?timeout=32s>": tls: failed to verify certificate: x509: certificate is not valid for any names, but wanted to match harvester.domain.local
E1030 22:06:38.125057   58405 memcache.go:265] couldn't get current server API group list: Get "<https://harvester.domain.local/k8s/clusters/local/api?timeout=32s>": tls: failed to verify certificate: x509: certificate is not valid for any names, but wanted to match harvester.domain.local
E1030 22:06:38.141503   58405 memcache.go:265] couldn't get current server API group list: Get "<https://harvester.domain.local/k8s/clusters/local/api?timeout=32s>": tls: failed to verify certificate: x509: certificate is not valid for any names, but wanted to match harvester.domain.local
Unable to connect to the server: tls: failed to verify certificate: x509: certificate is not valid for any names, but wanted to match harvester.domain.local

Hi James, did you configure the

ssl-certificates

setting and provide any custom SSL certificate on the Harvester dashboard? The PEM format cert you saw in the

local.yaml

is the CA. It does not contain the SAN extension. The SAN extension is in the certificate on the server side. You can grab it by the following command:

echo | openssl s_client -connect <vip-of-harvester-cluster>:443 -showcerts | openssl x509 -noout -text

So the downloaded

local.yaml

seems fine. And I’m curious about the domain name in the log:

harvester.domain.local

. Is it resolved to the VIP address? If yes, did you modify the value of the

.clusters[0].cluster.server

field to the custom domain name in

local.yaml

?

harvester.domain.local

is a made up domain, but yes, the real name resolves to the VIP address. I do not have a custom SSL cert on the dashboard.

This is expected. If you don’t get a valid certificate configured for the cluster,

kubectl

will keep complaining that the domain name requested does not match any names in the certificate. The default certificate only has the VIP and other internal IP addresses in the SAN extension. However, as a workaround, you can still provide the

--insecure-skip-tls-verify=true

flag to

kubectl

. Or instead, provide the

--tls-server-name=<vip-of-harvester-cluster>

flag.

This worked in 1.2.0, so something changed in the behavior of 1.2.1. I’m perfectly happy with the self-signed cert from the server, but only being able to access the server by IP feels so 1990s. The other option would be to support/install cert manager in the harvester cluster itself, which is how I manage every other cluster, both personally and professionally. It’s supported in Rancher, so it would make a huge amount of sense to support it here.

That’s odd. I can verify this is still the case back in v1.2.0, as I have a fresh v1.2.0 cluster provisioned. Integrating cert-manager could be a good idea. 👍 There seems to be no such place for configuring the domain name during the installation AFAIK.

I’m going to re-install 1.2.0, the only difference between my 1.2.0 and 1.2.1 setup is a

s/1.2.0/1.2.1/

on the installation configuration file. I’ll report back on what I find

Ah, I get it. I must have misunderstood your original question. So the problem here is that, with v1.2.1, the downloaded kubeconfig file contains the hostname as the URL, while with v1.2.0, it is the VIP address. Is that correct?

Yes. I misunderstood my own problem when I first posted 🙂

yep, I’m checking v1.2.1 right now.

Is that an upgrade from 1.2.0 -> 1.2.1 or a fresh install?

hmm, good point. It was upgraded from 1.2.0

I’ve only got a single node cluster, so I don’t really expect an upgrade to work well for me. I’ve got everything scripted with PXE so it’s pretty easy for me to switch back and forth for now

If you’re willing to install v1.2.1 again, could you help check the

vip

ConfigMap in the

harvester-system

namespace? The downloaded kubeconfig is constructed using that as a source.

It’s pretty late here, I’m going to clean up my terraform, then I can start the reinstall, but I won’t be able to check it until the morning

Sure, thanks for the help 🙌

Install is running. I’ll report back in the morning

With a freshly installed v1.2.1 cluster, it’s still a VIP address instead of the hostname. The downloaded kubeconfig:

apiVersion: v1
kind: Config
clusters:
- name: "local"
  cluster:
    server: "<https://10.84.132.32/k8s/clusters/local>"
    certificate-authority-data: "<redacted>"

users:
- name: "local"
  user:
    token: "kubeconfig-user-7dxlrgmwgn:<redacted>"


contexts:
- name: "local"
  context:
    user: "local"
    cluster: "local"

current-context: "local"

And the

vip

ConfigMap:

apiVersion: v1
data:
  enabled: "true"
  hwAddress: <redacted>
  ip: 10.84.132.32
  loadBalancerIP: ""
  mode: dhcp
  serviceType: LoadBalancer
kind: ConfigMap
metadata:
  annotations:
    <http://meta.helm.sh/release-name|meta.helm.sh/release-name>: harvester
    <http://meta.helm.sh/release-namespace|meta.helm.sh/release-namespace>: harvester-system
    <http://objectset.rio.cattle.io/id|objectset.rio.cattle.io/id>: default-mcc-harvester-cattle-fleet-local-system
  creationTimestamp: "2023-11-01T04:30:21Z"
  labels:
    <http://app.kubernetes.io/managed-by|app.kubernetes.io/managed-by>: Helm
    <http://objectset.rio.cattle.io/hash|objectset.rio.cattle.io/hash>: e852fa897f5eae59a44b4bfe186aad80b10b94b3
  name: vip
  namespace: harvester-system
  resourceVersion: "4401"
  uid: 67873575-64ce-4af4-962a-acc6843279c6

And I got a hostname on my fresh install

Seems not relevant. I’ve provisioned another v1.2.1 cluster with the static VIP address, and the URL was still the VIP address 😵‍💫

harvester.domain.local

is the VIP, not the node. The VIP actually does have a PTR record. I’ll create the ticket later this morning

Thanks! I’m checking if I can add a PTR record in my environment.