Run Unsloth on a different GPU

[quaint shore]

Hey guys, I know the free version can only run on single GPU, but if I'm trying to run it on GPU 1 or 2 instead of 0, how would I do that? Looks like I'm missing a setup step since I always get a RunTime error on gpu>0

os.environ['CUDA_VISIBLE_DEVICES'] = '2'
gpu = 1
inputs = tokenizer(d_prompt.format(text, ''), return_tensors="pt").to("cuda:%d" % gpu)

RuntimeError: CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

[bold rune]

it should default to one automatically mm weird

[nocturne imp]

Maybe this can help: on multiGPU server I choose one of them by id, bash script:

uuids to device numbers

device_ids=""
for device in $(echo $CUDA_VISIBLE_DEVICES | tr ',' ' '); do
device_id=$(nvidia-smi -L | grep $device | egrep -o 'GPU [0-3]' | sed 's/GPU //g')
device_ids+=$device_id,
done
export CUDA_VISIBLE_DEVICES=${device_ids::-1}