How to avoid Cloudflare timeouts on pods? | Runpod | Page 1

glass barn Aug 21, 2024, 3:45 PM

#

I saw a previous post mentionning using the public IP but it doesn't seem to work for me?

I'm using runpod to host a vLLM server (the serverless endpoint doesn't work for me).

I'm running batch workloads and those timeout (cloudflare)

How to avoid that please?

wintry wind Aug 21, 2024, 5:29 PM

#

Use tcp or ssh tunnel

glass barn Aug 21, 2024, 8:36 PM

#

Understood for ssh tunnel, but I don't understand for tcp, would it still accept https connection?

tranquil furnace Aug 22, 2024, 6:41 AM

#

i don't think so but you can try it

tranquil furnace Aug 22, 2024, 6:42 AM

#

serverless doesn't work? why? maybe if you can post the problem i can try to help

glass barn Aug 22, 2024, 10:17 AM

#

I tried with the NousHermes model llama 3.1 8b and it doesn't work out of the box

#

the requests hang for ever

tranquil furnace Aug 22, 2024, 11:00 AM

#

no it doesn't hang forever, i loaded 70b model in like 5mins ish

#

then the next ones are faster

#How to avoid Cloudflare timeouts on pods?