Select specific GPU to use

[ornate shuttle]

I have 2 GPUs on my machine, primary being 6GB card and secondary 8GB. The secondary is being used by another service which is why I want to set InvokeAI to only use the primary 6GB card. But for some reason, it keeps returning an error whenever I have "cuda:1" selected, and forces the 8GB card to be used with "cuda" resulting in simultaneous use which slows everything down drastically.

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cuda:1! (when checking argument for argument weight in method wrapper_CUDA_cudnn_convolution)

It does seem to utilize the VRAM of the primary card, even though it's refusing to actually work. As soon as I try to invoke, 60% of the primary card's VRAM is preserved and won't be released until I shut down the console.

I'd also like to note that it's only possible to select either "cuda" or "cuda:1", setting "cuda:0" puts the whole project into a "bootloop" lol

[kindred nimbus]

Hi @ornate shuttle. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

---

Hmm, I don't know enough to give you a confident answer yet. The information extracts provided do not contain specific instructions on how to select a specific GPU for InvokeAI to use.

However, it's clear that InvokeAI does support GPU usage, as mentioned in the installation guide (https://invoke-ai.github.io/InvokeAI/installation/030_INSTALL_CUDA_AND_ROCM/). It's also evident from the GitHub issues that some users have faced problems with GPU usage, but the extracts do not provide solutions for your specific issue.

I recommend checking the official InvokeAI documentation or reaching out to the community for more specific guidance on this issue.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

@ornate shuttle kapa.ai is still learning and improving, please let us know how it did by reacting below

[tacit ravine]

is there a error message for the bootloop ?

[ornate shuttle]

Ok for some reason it doesn't do that anymore, but the problem still is that it's either using the unwanted GPU to generate or giving that error

This is the result with cuda:1

As you can see, it preserves the ~60% VRAM of the correct card and begins to generate, but fails immediately. When setting just "cuda", it tries to generate concurrently with the secondary card.

Now the preservation won't be cleared until I manually terminate the console. Retrying to invoke just repeats the error

cuda:0 setting simply refuses to work as it's expecting only cuda or cuda:1, giving that "bootloop" which I mentioned earlier (it doesnt actually loop afterall, but just refuses to work regardless)

[tacit ravine]

Can you try CPU noise and Set your Cache value Low for a test

[ornate shuttle]

CPU Noise is on by default

Which cache value do you mean?

Setting CPU noise makes no difference, I also set RAM cache to 4GB instead of ~7GB but no difference. System has enough RAM, and the GPU has all VRAM available and is not being used by anything else other than Windows itself.

Screen photo because I'm too lazy to open remote desktop for a proper SS, and the screen&phone are right here xD

[tacit ravine]

what is your vram cache setting value? the 60% is weird

[ornate shuttle]

Default, 0.25GB

[tacit ravine]

mhm, i dont think invoke runs concurrently on both cards.

but i cant really test this

[ornate shuttle]

Auto1111 also caps at around 60-70% while generating, it only maxes out when reconstructing in the end

So basically I have auto1111 running and using my secondary (8GB) card, and I want to use InvokeAI with my primary (6GB) card which is utilized only by Widows itself. But it keeps giving this error whenever I have cuda:1 selected, while "cuda" forces InvokeAI to use the same secondary card as auto1111 is using which causes simultaneous use

And when they are running simultaneously, both generations slow down drastically and when the end is reached, other appears to wait until the other one is finished lol

[tacit ravine]

can you try this: https://www.reddit.com/r/invokeai/comments/105ft27/comment/j3o0qw9/?utm_source=reddit&utm_medium=web2x&context=3

[ornate shuttle]

It's confusing why InvokeAI is refusing to use the primary card. All it's saying is something about finding 2 cards

Even more confusion lol, the primary card is 0 and secondary 1. So by logic, setting "cuda:1" should force InvokeAI to use the secondary card, right? And setting "cuda" should use the primary card, right? Well, it seems to be the opposite lol

Though, could it be that since both programs are utilized by the same executable Python.exe?

I'll investigate this further tonight, please tell me your thoughts

[ornate shuttle]

Yeah, cuda:1 seems to utilize the primary card even though the number for the primary card is actually 0 while the secondary is 1. The error just persists, really weird

It would make sense to not being able to use the card which is already utilized by some other program, but it's indeed the opposite. Why could this be? I might try to run InvokeAI on a VM tonight and see if that makes a difference, I feel like Windows could be behind nonsense like this lmao

[tacit ravine]

did you try this ?

[ornate shuttle]

No difference, it only multiplies one python.exe in the smi list but no difference to anything

[edgy meadow]

go through nvida control panel

cuda

or create a batch script to do it

`
:: Set CUDA_VISIBLE_DEVICES to the ID of the GPU you want to use for the first AI application
set CUDA_VISIBLE_DEVICES=1

:: Ensure we are in the correct directory for the first AI application
pushd G:\4060invoke
start "4060" cmd /c "title 4060 & python .venv\Scripts\invokeai-web.exe %*"
popd`

(is what I use)

might work, I had a similar issue

I have a startup script that switches between 3080 and 4060 using that