deepseek-r is loading for >1h into vram.

[rain idol]

Seems it is related to nmap on network drive. How do you solve it?

[south helm]

What error or what's the log did you get

[unborn cove]

are you using our vllm with network volume? it might downloading the model which could take a while.

[rain idol]

I use my own docker container with sglang inside. For rocm you have only pytorch, no vllm or sglang.

I use loaded to the disk model. There is no space to load the 2nd one: 670gb total. and it is downloading for 4 hours, not for 1

[south helm]

Okay, if it's loading, is there an oom error?

[rain idol]

loading safetensors checkpoint shards for 1 hour

no oom, this is 8xMI300X

I read on the github that it is often related on nmap over network drives, but not sure

[south helm]

Nmap? What's that

What's taking so long, can you debug to see the progress ( loading)

[rain idol]

function, that reads .tensor and load it to gpu, takes extremally long time. mmap is mapping file as memory to load data directly from ssd to vram with no ram consumption

[south helm]

Oh Im not experienced on these model downloading area, so I don't know much

[south helm]

Makes sense, people report network storage is slow to load big models

[fallow hornet]

dont use network storage to load the models, instead move them to container disk or pod volume disk, see if that loads them any faster

[rain idol]

It is on pod volume disk