trying to get data on speeds across gpu types for our jobs, and i'm wondering if the api exposes this anywhere, and if not, what the best way to sort it out would be.
#serverless: any way to figure out what gpu type a job ran on?
limber badger
hollow vortex
You can have your handler log the job ID (or whatever), along with the GPU info (either locally calling the API or asking via e.g. nvidia-smi or framework-specific introspection methods), to wherever you like, and retrieve the results later.
Also consider looking through the GraphQL schema if anything serverless job log related exposes what you want. Have you looked?
zenith vine
dont need nvidia-smi, we expose an env variable to the worker with the gpu name in it, look at our docs for env variable names
spice galleon
dont need nvidia-smi, we expose an env variable to the worker with the gpu name ...
is this what you're referring to?
i dont think thats the one, after browsing for some mins i still cant be able to find it
please send the docs url for that
hollow vortex
is this what you're referring to?
That would be where it should be; just run the native/CLI command env and see which one has a reasonable name/content.
If none do, then that's a wrong support claim, not just outdated docs.
You could also look at the (system/OS) environment from python or so, but I don't know off-hand how exactly.
spice galleon
what is the env variable key/name for it?
spice galleon
That would be where it should be; just run the native/CLI command `env` and see ...
yeah lets just wait for support
im too lazy changing and rebuilding my image for that
hollow vortex
im too lazy changing and rebuilding my image for that
Fair; if you wanted you could look by just launching a stock Jupyter notebook image.
(I don't yet have done the detailed budget for second half December, and thus not yet loaded credit into my RunPod account.)
spice galleon
ye lol
dusk patrol
Those are GPU cloud environment variables, these are serverless ones:
RUNPOD_WEBHOOK_POST_STREAM=https://api.runpod.ai/v2/12345657890/job-stream/12345657890/$ID?gpu=NVIDIA+L4
RUNPOD_ENDPOINT_ID=mpoacd7wrmv2fc
RUNPOD_CPU_COUNT=6
RUNPOD_POD_ID=p8btjjjjq865pi
RUNPOD_GPU_SIZE=AMPERE_24
RUNPOD_MEM_GB=62
RUNPOD_GPU_COUNT=1
RUNPOD_VOLUME_ID=hbsp3mav9e
RUNPOD_POD_HOSTNAME=p8btjjjjq865pi-64410f26
RUNPOD_DEBUG_LEVEL=INFO
RUNPOD_ENDPOINT_SECRET=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
RUNPOD_DC_ID=EU-RO-1
RUNPOD_AI_API_ID=mpoacd7wrmv2fc
RUNPOD_AI_API_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
RUNPOD_WEBHOOK_GET_JOB=https://api.runpod.ai/v2/12345657890/job-take/12345657890?gpu=NVIDIA+L4
RUNPOD_WEBHOOK_PING=https://api.runpod.ai/v2/12345657890/ping/12345657890?gpu=NVIDIA+L4
RUNPOD_WEBHOOK_POST_OUTPUT=https://api.runpod.ai/v2/12345657890/job-done/12345657890/$ID?gpu=NVIDIA+L4
RUNPOD_PING_INTERVAL=4000
CUDA_VERSION=11.8.0
NV_CUDNN_VERSION=8.9.6.50
Doesn't seem to have the GPU type set directly but you can get it from the end of RUNPOD_WEBHOOK_POST_STREAM, RUNPOD_WEBHOOK_GET_JOB, RUNPOD_WEBHOOK_PING, RUNPOD_WEBHOOK_POST_OUTPUT as shown above.
dusk patrol
Not sure how that translates to L4 though
spice galleon
Ah ye no full gpu type
zenith vine
looks like it isn't, i will plan to add it, this is good to expose in pod env
spice galleon
Wow you're a Rp developer
dusk patrol
Wow you're a Rp developer
He isn't just a developer, he is the CTO 😉