#My POD is not accessible + lost access to my data

25 messages · Page 1 of 1 (latest)

eager sedge
#

2 days ago a notice appeared next to my pod:

PLEASE CHECK ATTACHED TXT FILE FOR FULL INFO

We have detected a critical error on this machine which may affect some pods. We are looking into the root cause and apologize for any inconvenience. We would recommend backing up your data and creating a new pod in the meantime.

I guess this is a HW error. Since then, trying to boot with GPU gives me this error in log:

error creating container: nvidia-smi: parsing output of line 6: failed to parse ([GPU requires reset]) into int: strconv.Atoi: parsing "": invalid syntax

And wont boot up at all. If I try to bootup in CPU mode, the server seems to go online - with 512MB RAM which is immediately 100% utilized and 0.5 vCPU. Web terminal fails to launch and when I try to connect from my OSX terminal, I get through the authentication but end up with ...

FULL INFO IN THE ATTACHMENT

I am really desperate, I've been using runpod for over a month and have though what a great service it is. I've built and configured a perfect pod for my work workflow. Was currently running a big job for a client (which I have now loosed for not delivering on time). Despite the notice (quoted above) nobody is proactively looking into the issue, no updates. I have cotacted RunPods customer service and created a ticket (and have read the whole documentation). The support was completely useless - replying with some template answer telling me to create a network storage and migrate my data there, pointing me to two knowledgebase articles. But..

FULL INFO IN THE ATTACHMENT

This is very unfortunate situation for me and terrible customer experience. I've though "This is it" when I first discovered runpod but if this is how they care about their customers and the level of SLA they provide ..

Any ideas? Please help.

PLEASE CHECK ATTACHED TXT FILE FOR FULL INFO

📎 runpod_issue.txt
clever kernel
#

looks like community cloud

eager sedge
# clever kernel looks like community cloud

It is not. I'm in the secure cloud. Would expect some reasonable SLA for a service costing me around 450 usd / month. Not being able to access my data for 48+ hour plus stil having to pay for it - is ridiculous at least. Runpod had offered a compensation of 35 usd, what a (bad) joke. And I still dont have a resolution ETA.

gilded knot
#

Message me your ticket number, but publicly I'll recommend you manually expose TCP port 22 so you don't send your SSH through the proxy.

#

Please be advised this is me providing support outside of our standard support hours I may not be able to get back to you immediately.

eager sedge
eager sedge
gilded knot
#

I accepted a friend request.

eager sedge
clever kernel
#

@eager sedge have you tried to run pod with command: bash -c 'sleep infinity' that should let you access pod by blocking running any apps

eager sedge
#

I did not, let me try that

eager sedge
clever kernel
#

the FUSE wont work as it requires provilaged container, about your bash script idk

eager sedge
clever kernel
#

web terminal, basic ssh, tcp ssh?
rsync should work

eager sedge
#

web terminal and tcp ssh

eager sedge
eager sedge
#

ok, CROC seems to be the only way to get the job done

#

actually .. not. Worked for a small set of files, now I get: peer error: refusing files

#

getting desperate

clever kernel
#

@eager sedge would you give a try to sftp with like filezilla?

eager sedge
clever kernel
#

try open web terminal and type
pip install OhMyRunPod

then run OhMyRunPod

then select File Trnansfer and then SFTP it will give you details for Filezilla

eager sedge
#

will do that