This message was deleted.
# kubernetesa
adamant-kite-43734
11/08/2022, 11:29 PMThis message was deleted.
c
creamy-pencil-82913
11/08/2022, 11:39 PMthis is more of an nvidia question than a kubernetes question… but no it doesnt’ work that way
creamy-pencil-82913
11/08/2022, 11:39 PMcreamy-pencil-82913
11/08/2022, 11:39 PMMIG stands for Multi-Instance-GPU. It is a mode of operation for future Nvidia GPUs that allows one to partition a GPU into a set of MIG devices, each of which appears to the software consuming them as a mini-GPU with a fixed partition of memory and a fixed partition of compute resources.
creamy-pencil-82913
11/08/2022, 11:41 PMAssuming you’re using MIG-capable devices
creamy-pencil-82913
11/08/2022, 11:41 PMif it’s a single-instance device, then you can’t partition it at all, and whatever’s using it gets to use it.
👍 1
a
abundant-gpu-72225
11/08/2022, 11:42 PMI see, thank you very much for the response
c
creamy-pencil-82913
11/08/2022, 11:44 PMalso, are you sure that you’re OOMing on GPU resources, and not OOMing on host memory? Are you setting traditional memory requests/limits in addition to requesting GPU?
a
abundant-gpu-72225
11/08/2022, 11:46 PMYes definitely running our of video memory. I set too large of a batch size for inference for some model and got a cuda error. I was just wondering if it was possible to request a certain amount of vram to avoid this.
5 Views