#error out of memory cuda

20 messages · Page 1 of 1 (latest)

vocal cloak
#

hello im building a model with pytorch but i get this error.
i tried lowering my batch size, loading less images in my dataset, nothing worked.
Any ideas how to fix it ?
Thanks!

rough spire
#

Hi, A couple of questions:

  1. Are you getting the error in your first run or after few runs?
  2. what is the minimum batch size you tried.

If you are getting this issue after first batch, try to clear the GPU memory with avaiable commands in the pyTorch.

proper forge
#

I think you can begin with batch size=1,then try to increase batch size gradually or use gradient accumulation.

vocal cloak
#

I tried batch size 2

#

And it didn't work

#

I get it on the first run when the kernel is clean

#

I tried this one one command pytorch cuda empty cache but didn't work

vocal cloak
fervent solstice
#

Restart your kernel completely and reduce the batch size to 1. Also reduce your eval batch size to 1

#

Monitor the memory usage and increase the batch size accordingly.

vocal cloak
#

isn't it really bad to have a batch size on 1 ?

vocal cloak
#

it's working with bs=1 but is completely frozen

#

with gpu on 100%

#

there must be something wrong in the way i coded it

simple crown
#

Whats the dimensions

#

Try lowering the dimensions by half and see what happens

vocal cloak
#

didn't work

#

nop i can try what does it do ?