#Serverless container storage

15 messages · Page 1 of 1 (latest)

quiet jay
#

I made a script that downloads various models and packages and incorporated it in the docker image that is being run on serverless endpoint, i want it to run only once and download all the packages and store them for use, do i have to use network volume storage or is there another way to store the packages in container storage so they do not download every time the container is removed and started again?

kind forge
#

Yes

#

Let me know how big are your models In total

#

That will be stored

#

If you're using network storage just download them into /runpod-volume in serverless and then check if it has been downloaded then Dont re-download , or in pods you can download it once manually to /workspace

quiet jay
#

the total download data amounts to about 25GBs

kind forge
#

Okay yeah just use network storage

quiet jay
#

I was hoping for another solution 😩

kind forge
#

Ah okay here is it but it's not really a solution because it's slower, Another solution is to download it when your docker is building so execute a script that downloads them when your building from the dockerfile

torn cliff
#

I would suggest creating a version with the models built into the image and another version that uses network volume and compare the response time. There is an open issue with network volumes that cause them to add delaytime to the process so you might be better of with your 25gb image.

quiet jay
kind forge
#

As I said before it would be slower if your model sums up to 25gb but yeah sure try it out

torn cliff
#

With either choice you can speed up your requests by having at least 1 active worker.

#

and since you are only charged when your requests are being processed I suggest you set max workers to 30.

kind forge
#

Yup but if you're using active workers ig the request will be faster if you have the models inside the image