Am I left to understand the only way to send files to my network storage in a command line? I have spent two days on this. I am paying for storage I have no understanding how to access, so I am loading my files to hugging face in desperation.
#the lack of a drag and drop or file structure is not ok.
late aurora
nimble sentinel
@dawn rose
https://docs.runpod.io/pods/storage/transfer-files-with-scp
I think you guys need an updated documentation with how to upload / download files:
- Options of using CLI (SCP / SSH) (using current method, but also just using the
runpod ssh add-keymethod since that exists.
https://github.com/justinwlin/Runpod-Tips-and-Tricks/tree/main/SSH On Runpod
-
Using CROC (until RunpodCTL file transfer isn't so slow) (this prob the easiest for most people).
-
Giving community resource suggestions such as from @orchid depot :
https://github.com/kodxana/RunPod-FilleUploader/tree/main
But basically ways to download / upload I think is important, and definitely not known best practices, especially a lot of people will try to just use jupyter notebook not knowing there are problems with larger files and so-on.
Maybe also a section on telling people use HuggingFace CLI / linking people to civitai docs, cause this also happens a lot where people never use their token to download civitai
dawn rose
Thanks for the ping!
I know of this method:
[https://github.com/runpod/runpodctl?tab=readme-ov-file#to-send-a-file]
runpodctl send data.txt
But you're saying that is pretty slow?
I'll log this request and investigate a bit more too.
nimble sentinel
Thanks for the ping!
I know of this method:
[https://github.com/runpod/runpodct...
Yup! This is slow. I forgot where, but flash / justin should know about it. The problem is that runpodctl is using own private relay servers, so its been heavily bottlenecked:
https://discord.com/channels/912829806415085598/1207427186974396556
So the current idea has been to use croc, since they use public servers and are significantly faster
orchid depot
@dawn rose issue is cause all pods are consider being behind cgnat so all data is transfered by relays
nimble sentinel
I think the solution is to use runpodctl and tell everyone to use it, but right now croc seems the best directly (which is what runpodctl uses anyways under the hood)
orchid depot
My RunPod File Uploader is only one way but it works with pods that not have tcp
nimble sentinel
Yeah, I think end of the day, either runpodctl gets fixed and docs gets added to use that for send / download data as the "easy way". And telling people to zip it up before sending it.
Or telling people to use croc cause the relays right now for runpod is too slow
And other methods can be more "advance".
And @orchid depot method can be a "community option" for simply uploading only
orchid depot
My tool is already integrated in templates made by ashleyk