For a few days we were stuck with this error on serverless worker startup, tried many torch, cuda combinations, turns out that this particular region EUR-NO-1 was giving 5090s that were causing the issue i.e CUDA Not Available, just filtering it out solved it.
So maybe we can have some sort of service that can filter these out on it's own to stay safe and save time?