Sometimes Receiving Models without CUDA

[sturdy wharf]

Depending on the GPU I choose and availability regions, I get this error up to around 50% of the time.

{"requestId": null, "message": "Fitness check failed: _cuda_init_check | RuntimeError: CUDA initialization failed: Failed to initialize GPU 0: CUDA error: no kernel image is available for execution on the device\nCUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.\nFor debugging consider passing CUDA_LAUNCH_BLOCKING=1\nCompile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.\n", "level": "ERROR"}

It seems to me like either the image isn't always loading with CUDA, or I'm receiving instances that don't have GPUs attached. Happy to DM with info about the particular endpoint I'm running this on. Anyone experienced something similar? And RunPod team, please help me out!

[final crow]

To help others find answers, you can mark your question as solved via Right click solution message -> Apps -> ✅ Mark Solution

[gray lotus]

It looks like the GPU either isn’t properly initialized or the instance doesn’t have a compatible GPU. Make sure the image supports your GPU architecture and the instance actually has a GPU attached. You can also try setting CUDA_LAUNCH_BLOCKING=1 to get more precise errors. I can DM you to troubleshoot the specific endpoint.