That's coming from the shim

I have everything in 'hobbyfarm-system'

I'll redeploy later into 'hobbyfarm' and try again

If I export the manifests and rename them, can I do so as long as I keep the prefixes on the name that is 'vmt-myname`

s-myname

etc?

I'm closer now. I'm stuck here this is coming from hf-shim when trying to provision vms dynamically from ec2

2023-03-17T13:33:40.346Z    INFO    controller-runtime.metrics    metrics server is starting to listen    {"addr": ":8080"}                                                                                                                                                │
│ 2023-03-17T13:33:40.354Z    INFO    setup    starting manager                                                                                                                                                                                                              │
│ 2023-03-17T13:33:40.357Z    INFO    controller-runtime.manager    starting metrics server    {"path": "/metrics"}                                                                                                                                                          │
│ 2023-03-17T13:33:40.361Z    INFO    controller-runtime.controller    Starting EventSource    {"controller": "virtualmachine", "source": "kind source: /, Kind="}                                                                                                           │
│ 2023-03-17T13:33:40.469Z    INFO    controller-runtime.controller    Starting EventSource    {"controller": "virtualmachine", "source": "kind source: /, Kind="}                                                                                                           │
│ 2023-03-17T13:33:40.570Z    INFO    controller-runtime.controller    Starting EventSource    {"controller": "virtualmachine", "source": "kind source: /, Kind="}                                                                                                           │
│ 2023-03-17T13:33:40.672Z    INFO    controller-runtime.controller    Starting EventSource    {"controller": "virtualmachine", "source": "kind source: /, Kind="}                                                                                                           │
│ 2023-03-17T13:33:40.774Z    INFO    controller-runtime.controller    Starting EventSource    {"controller": "virtualmachine", "source": "kind source: /, Kind="}                                                                                                           │
│ 2023-03-17T13:33:40.876Z    INFO    controller-runtime.controller    Starting EventSource    {"controller": "virtualmachine", "source": "kind source: /, Kind="}                                                                                                           │
│ 2023-03-17T13:33:40.979Z    INFO    controller-runtime.controller    Starting Controller    {"controller": "virtualmachine"}                                                                                                                                               │
│ 2023-03-17T13:33:40.979Z    INFO    controller-runtime.controller    Starting workers    {"controller": "virtualmachine", "worker count": 20}                                                                                                                              │
│ 2023-03-17T13:35:47.261Z    ERROR    controller-runtime.controller    Reconciler error    {"controller": "virtualmachine", "name": "dynamic-hywxhhkaqj-02260ede", "namespace": "hobbyfarm", "error": "Operation cannot be fulfilled on <http://virtualmachines.hobbyfarm.io|virtualmachines.hobbyfarm.io> \"dyna │
│ <http://github.com/go-logr/zapr.(*zapLogger).Error|github.com/go-logr/zapr.(*zapLogger).Error>                                                                                                                                                                                                                                 │
│     /go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128                                                                                                                                                                                                                 │
│ <http://sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler|sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler>                                                                                                                                                                                      │
│     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.14/pkg/internal/controller/controller.go:257                                                                                                                                                                           │
│ <http://sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem|sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem>                                                                                                                                                                                   │
│     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.14/pkg/internal/controller/controller.go:231                                                                                                                                                                           │
│ <http://sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker|sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker>                                                                                                                                                                                                │
│     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.14/pkg/internal/controller/controller.go:210                                                                                                                                                                           │
│ <http://k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1|k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1>                                                                                                                                                                                                                        │
│     /go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:152                                                                                                                                                                                                      │
│ <http://k8s.io/apimachinery/pkg/util/wait.JitterUntil|k8s.io/apimachinery/pkg/util/wait.JitterUntil>                                                                                                                                                                                                                              │
│     /go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:153                                                                                                                                                                                                      │
│ <http://k8s.io/apimachinery/pkg/util/wait.Until|k8s.io/apimachinery/pkg/util/wait.Until>                                                                                                                                                                                                                                    │
│     /go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:88                                                                                                                                                                                                       │
│ time="2023-03-17T13:35:47Z" level=info msg="creating new keypair"                                                                                                                                                                                                          │
│ 2023-03-17T13:35:47.784Z    ERROR    controller-runtime.controller    Reconciler error    {"controller": "virtualmachine", "name": "dynamic-hywxhhkaqj-02260ede", "namespace": "hobbyfarm", "error": "Secret \"dynamic-hywxhhkaqj-02260ede-secret\" not found"}            │
│ <http://github.com/go-logr/zapr.(*zapLogger).Error|github.com/go-logr/zapr.(*zapLogger).Error>                                                                                                                                                                                                                                 │
│     /go/pkg/mod/github.com/go-logr/zapr@v0.1.0/zapr.go:128                                                                                                                                                                                                                 │
│ <http://sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler|sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler>                                                                                                                                                                                      │
│     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.14/pkg/internal/controller/controller.go:257                                                                                                                                                                           │
│ <http://sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem|sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem>                                                                                                                                                                                   │
│     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.14/pkg/internal/controller/controller.go:231                                                                                                                                                                           │
│ <http://sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker|sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).worker>                                                                                                                                                                                                │
│     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.5.14/pkg/internal/controller/controller.go:210                                                                                                                                                                           │
│ <http://k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1|k8s.io/apimachinery/pkg/util/wait.JitterUntil.func1>                                                                                                                                                                                                                        │
│     /go/pkg/mod/k8s.io/apimachinery@v0.17.2/pkg/util/wait/wait.go:152

I'm completely stuck @faint-optician-47536 or @worried-fountain-60974 if y'all have some zoom time this morning. I'm sure I'm missing something that should be obvious

unfortunately i am not quite sure how to get hf-shim running. we are still using the tf-controller here. i know that some guy got aws + tf running, but he had to update the module manually (this is for terraform only)

This message from hf-shim would indicate that the secret is not created, i guess you have checked that already and it is indeed missing?

I think it eventually created the secret

yeah let's mess with this JvB. I'm tied up until top of the hour (10a CST) then we could

Looking through the shim reconciler. vm.status is not being set to SecretCreated. It's stuck in readyforprovisioning, even though it looks like it should work just fine

oh is hfShim not calling updateStatus yet @worried-fountain-60974. as of https://github.com/hobbyfarm/gargantua/commit/27779a010998aa86adf1145113469065041fdb1d status is added as a subresource in gargantua. maybe this effects hf-shim too

I think it's doing it wrong looking at the code now

It should be working fine. But it's not, I walked through in the debugger

It's been a good while since I worked controller code, so I might be missing something

That might be it @faint-optician-47536

What you said

Yes that was a change in the underyling CRDs.

and i guess that hf-shim is not yet doing this. so we would have to update hf-shim.

you might use terraform + aws module to get it running for now. but lets wait for what eamon has to say 🙂

ahh crap if it isn't that's going to be part of the problem

I can probably fix that momentarily if you are okay running a very fresh RC

Give me 10?

It's not implementing UpdateStatus, that's correct

Yeah I can give an RC a go

Good cuz the alternative is it stays broken 😉

ugh https://github.com/hobbyfarm/hf-shim-operator/issues/12

I was fixing this lol

Thread for this here. So we stop bothering people unnecessarily.