Channels
academy
amazon
arm
azure
cabpr
chinese
ci-cd
danish
deutsch
developer
elemental
epinio
espanol
events
extensions
fleet
français
gcp
general
harvester
harvester-dev
hobbyfarm
hypper
japanese
k3d
k3os
k3s
k3s-contributor
kim
kubernetes
kubewarden
lima
logging
longhorn-dev
longhorn-storage
masterclass
mesos
mexico
nederlands
neuvector-security
office-hours
one-point-x
onlinemeetup
onlinetraining
opni
os
ozt
phillydotnet
portugues
rancher-desktop
rancher-extensions
rancher-setup
rancher-wrangler
random
rfed_ara
rio
rke
rke2
russian
s3gw
service-mesh
storage
submariner
supermicro-sixsq
swarm
terraform-controller
terraform-provider-rancher2
terraform-provider-rke
theranchcast
training-0110
training-0124
training-0131
training-0207
training-0214
training-1220
ukranian
v16-v21-migration
vsphere
windows
Title
q
quaint-alarm-7893
03/23/2023, 4:46 AMi'm trying to get a custom rke2 or k3s cluster working on-top of harvester. but i'm having issues getting the cloud-provider working.
i followed: https://docs.harvesterhci.io/v1.1/rancher/cloud-provider
if i deploy a 1.24.4 RKE2 cluster with Harvester for the cloud provider, the cluster never finishes spinning up.
if i try to do a custom RKE2/K3S cluster with no provider, then add the Harvester-Cloud-provider later, i get this error:
Error getting instance metadata for node addresses: Get
"<https://192.168.5.100/k8s/clusters/local/apis/kubevirt.io/v1/namespaces/k8s/virtualmachines/k3s-m1>": x509: certificate signed by unknown authorityf
full-plastic-79795
03/23/2023, 5:51 AMThank you for bringing this issue to our attention. Based on our investigation, it appears that the user may have been using a kubeconfig downloaded from the UI, rather than obtaining it from the /etc/rancher/rke2/rke2.yaml file after SSH into the Harvester node. Using a kubeconfig downloaded from the UI can cause this problem. We recommend that the user try obtaining the kubeconfig from within the node instead. Please let us know if this resolves the issue. Thank you.
q
quaint-alarm-7893
03/23/2023, 5:52 AMare you talking about the kubeconfig used with the generate_addon.sh script?
f
full-plastic-79795
03/23/2023, 5:52 AMYes
q
quaint-alarm-7893
03/23/2023, 5:53 AMgreat! i'll try that. thanks!
is it just me or can you not use the harvester provider w/ a custom rke2 cluster? it seems like every time i try that method, it just hangs on the install
f
full-plastic-79795
03/23/2023, 6:15 AMCould you describe the step to reproduce in detail？
q
quaint-alarm-7893
03/23/2023, 6:16 AMsure. standby. 🙂
🙏 1
ignore this one.
In Rancher Cluster Management click Create Cluster
Pick “Custom” for the provider:
Set cluster version to v1.24.4, because that’s what all the docs show. Newer versions seem to exhibit the same behavior.
Click “Create”
Grab the Registration command and save it.
curl --insecure -fL https://{rancher-url} /system-agent-install.sh | sudo sh -s - --server https://{rancher-url} --label 'cattle.io/os=linux' --token {token} --ca-checksum {ca-checksum} --etcd --controlplane –worker
Create a VM in harvester. I used a Ubuntu Cloud-img. Let me know if you’d like the YAML
login to the VM once it’s ready, and run the curl command from the Registration Page from your Custom RKE2 Cluster.
Watch logs with: sudo journalctl -u rke2-server -f
Most of my clusters take about 8 minutes to stand up. I can let this run over-night and it never finishes.
👍 1
image.png
logs.txt
looks like it might have to do w/ it not creating the cloud-config files here too. similar issue as to when you do not select harvester as the provider, and you have to add it manually.
f
full-plastic-79795
03/23/2023, 6:42 AMThank you for providing us the steps. After reviewing your scenario, we have found that it is not covered in our current testing procedures, as we typically use the Harvester node driver to automatically create clusters.
q
quaint-alarm-7893
03/23/2023, 6:43 AMwhich makes sense, but i'm trying to get a windows node up and running, that's why i'm running into this. seems the best way to get a windows node is the custom cluster, w/ the windows registration command (after bootstrapping your cluster w/ atleast 1 linux master)
do you guys have anything you are doing currently to get a windows node working with a rke2 guest-cluster?
f
full-plastic-79795
03/23/2023, 6:53 AMI appreciate your question, but unfortunately, I don't have the relevant experience to provide you with a satisfactory answer.
If you don't mind, could you please submit your question as a GitHub issue in harvester/harvester repo ?
Please let me know if you have any further questions or concerns.
q
quaint-alarm-7893
03/23/2023, 7:24 AMwill do. thanks!
👍 1
@full-plastic-79795 and @salmon-city-57654 i created a custom cluster, dropped the new cloud-config generated from the rke.yaml file off one of the harvester servers onto the node, added the helm chart for the Harvester-CSI, and created a PVC. i get this error now:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Provisioning 4s (x7 over 68s) driver.harvesterhci.io_harvester-csi-driver-controllers-779c557d47-srfwv_6e6919cb-2c85-4eef-ab22-88caf890c870 External provisioner is provisioning volume for claim "natimark/sync-pvc"
Warning ProvisioningFailed 4s (x7 over 67s) driver.harvesterhci.io_harvester-csi-driver-controllers-779c557d47-srfwv_6e6919cb-2c85-4eef-ab22-88caf890c870 failed to provision volume with StorageClass "harvester": rpc error: code = Internal desc = persistentvolumeclaims is forbidden: User "system:serviceaccount:k8s:default" cannot create resource "persistentvolumeclaims" in API group "" in the namespace "k8s"
Normal ExternalProvisioning 1s (x6 over 68s) persistentvolume-controller waiting for a volume to be created, either by external provisioner "<http://driver.harvesterhci.io|driver.harvesterhci.io>" or manually created by system administratoralso worth noting, i'm using the service_account_name default, and namespace k8s for generate_addon.sh. i'm not sure if i should be maybe passing it something else? and do i have to do anything to create the service account first maybe?
another note. if i create a cluster w/ harvester as the cloud provider, grab that cloud-config, and use it in my custom cluster, it seems to provision the pvc just fine. so it must have something to do w/ that config generation.