This message was deleted.

does removing the changes in kubevirt object work?

/spec/customizeComponents/patches/

BTW, can you share the reason you want to evaluate a different image? Thanks.

I will take a look at it. Could you provide some examples of what you want to apply? (like what image you would like to patch

@prehistoric-balloon-31801 @salmon-city-57654, my harvester cluster is based on version 1.1.1 and the corresponding version for the kubevirt is v0.54.0 so therefore , I have cloned the kubevirt code (v0.54.0) and made some changes to the virt-launcher code (this is the only component I am interested in) The image has been pushed to our local internal repository and to update the virt-launcher image across both virt-controller and virt-handler I updated the following in kubevirt CR Also please let me know if there's any other place I need to update the virt-launcher image apart from these two

customizeComponents:                     
    patches:                               
    - patch: '{"spec":{"template":{"spec":{"containers":[{"name":"virt-controller", "args":["--launcher-image","<http://test-registry.example.com/myrepo/virt-launcher@sha256:9144b0697edd1e26a23282007a017af7564b9813eb32fd5356afaff244c470c5|test-registry.example.com/myrepo/virt-launcher@sha256:9144b0697edd1e26a23282007a017af7564b9813eb32fd5356afaff244c470c5>","--port","8443", "-v", "2"]}]}}}}'
      resourceName: virt-controller
      resourceType: Deployment
      type: strategic                             
    - patch: '{"spec":{"template":{"spec":{"initContainers":[{"name":"virt-launcher","image":"<http://test-registry.example.com/myrepo/virt-launcher@sha256:9144b0697edd1e26a23282007a017af7564b9813eb32fd5356afaff244c470c5|test-registry.example.com/myrepo/virt-launcher@sha256:9144b0697edd1e26a23282007a017af7564b9813eb32fd5356afaff244c470c5>","imagePullPolicy":"Always"}]}}}}'
      resourceName: virt-handler
      resourceType: Daemonset
      type: strategic

I tried removing

/spec/customizeComponents/patches/

and it did kind of work but wanted to be sure if there's any other process to be followed here

@salmon-city-57654 This isn't my thread, but to give you a heads up, one reason to want a newer/updated KubeVirt is because SR-IOV is unusable in a lot of situations in the current version of Harvester due to a bug that existed in 0.54, is fixed in the 0.54 release branch, but not yet pulled back into the 0.54.0-150400.3.19.1 version. Basically the SR-IOV devices are not presented to the VM in the order they are assigned and they may not even be presented to the correct VMs due to the mapping bug. I think https://github.com/kubevirt/kubevirt/pull/8415 is a link to one of the related fixes that hasn't been pulled in to the builds yet.

Hi @powerful-soccer-11224, yes, if you want to patch the

virt-launcher

image. These two parts should be patched. Did you try after you patched the

virt-launcher

image? Also, if you want to rollback, just remove your added parts from the

/spec/customizeComponents/patches/

on the

KubeVirt

CR. Did it work now?

I removed them but still no luck while rolling back to previous image registry.suse.com/suse/sles/15.4/virt-launcher:0.54.0-150400.3.3.2

Thanks, @thousands-action-31843; that’s a good example of why we need to patch the Kubevirt image.

I tried that too by updating the kubevirt-controller in

/spec/customizeComponents/patches/

to use the image registry.suse.com/suse/sles/15.4/virt-launcher:0.54.0-150400.3.7.1 but the controller pods are in CrashLoopBackOff state and on inspecting the container logs , I found this :

exec /usr/bin/virt-controller: exec format error

Could you check the virt-controller of deployment? What is the virt-controller image you are using

From the virt-controller deployment

Events:                                                                                                                                                                                        │
│   Type     Reason          Age                   From               Message                                                                                                                    │
│   ----     ------          ----                  ----               -------                                                                                                                    │
│   Normal   Scheduled       38m                   default-scheduler  Successfully assigned harvester-system/virt-controller-9c6f7799-lwhsj to iaas-node-002                                     │
│   Normal   AddedInterface  38m                   multus             Add eth0 [10.52.2.151/32] from k8s-pod-network                                                                             │
│   Normal   Pulled          37m (x4 over 38m)     kubelet            Container image "registry.suse.com/suse/sles/15.4/virt-controller:0.54.0-150400.3.7.1" already present on machine          │
│   Normal   Created         37m (x4 over 38m)     kubelet            Created container virt-controller                                                                                          │
│   Normal   Started         37m (x4 over 38m)     kubelet            Started container virt-controller                                                                                          │
│   Warning  BackOff         3m7s (x179 over 38m)  kubelet            Back-off restarting failed container

Could you check the following command output?

kubectl get deployment virt-controller -n harvester-system -o yaml |yq -e .spec.template.spec.containers

here's the O/P of command

- args:
    - --launcher-image
    - <http://registry.suse.com/suse/sles/15.4/virt-launcher:0.54.0-150400.3.7.1|registry.suse.com/suse/sles/15.4/virt-launcher:0.54.0-150400.3.7.1>
    - --port
    - "8443"
    - -v
    - "2"
  command:
    - virt-controller
  image: <http://registry.suse.com/suse/sles/15.4/virt-controller:0.54.0-150400.3.7.1|registry.suse.com/suse/sles/15.4/virt-controller:0.54.0-150400.3.7.1>
  imagePullPolicy: IfNotPresent
  livenessProbe:
    failureThreshold: 8
    httpGet:
      path: /healthz
      port: 8443
      scheme: HTTPS
    initialDelaySeconds: 15
    periodSeconds: 10
    successThreshold: 1
    timeoutSeconds: 10
  name: virt-controller
  ports:
    - containerPort: 8443
      name: metrics
      protocol: TCP
  readinessProbe:
    failureThreshold: 3
    httpGet:
      path: /leader
      port: 8443
      scheme: HTTPS
    initialDelaySeconds: 15
    periodSeconds: 10
    successThreshold: 1
    timeoutSeconds: 10
  resources:
    requests:
      cpu: 10m
      memory: 150Mi
  terminationMessagePath: /dev/termination-log
  terminationMessagePolicy: File
  volumeMounts:
    - mountPath: /etc/virt-controller/certificates
      name: kubevirt-controller-certs
      readOnly: true
    - mountPath: /profile-data
      name: profile-data

And If I use my custom image , it works very well . The only time it's causing the issue it while rolling back to the same image registry.suse.com/suse/sles/15.4/virt-controller:0.54.0-150400.3.7.1, one of the pods is in crashLoopBackOff state

Interesting… I will try to reproduce this on my v1.1.1 cluster.

Hi @salmon-city-57654 sorry for the late reply. It was on leave yesterday. I tried the above steps again and things seems to be working fine. The problem was with one of the nodes on harvester where I am encountering the error : exec

/usr/bin/virt-controller: exec format error

. I am still not able to find out why it's working on the remaining 2 nodes and only failing on this node even though all the configuration is same for all nodes

Hi @powerful-soccer-11224, could you try the workaround on the https://github.com/harvester/harvester/issues/4434 mentioned?

@salmon-city-57654 I followed the steps mentioned in the workaround and it seems to be working. Thanks a ton for all your help 🙂