#Empty Responses from Hermes 405b (Lambda)

27 messages · Page 1 of 1 (latest)

untold panther
#

Hi, there are certain issues I've faced while using Hermes 405b (Provider: Lambda)

I keep getting empty responses since three hours ago (approximately).

Normally, there would be an error notice (like 529 error or something that includes numbers), but no, nothing pops up. All I get is an empty response.

DeepInfra Hermes is working well, but it seems like the Hermes with the Lambda provider is going through these problems.
(This includes Hermes 405 extended as well)

I tried changing my prompts, reducing the max context (tokens), and obtaining a new API key....but it still isn't working.

I checked Latency and Throughput of Hermes via OpenRouter website, just in case. At least from what I know, everything seems fine.
The last time I checked it, it said that the provider is likely operational, and the Latency/Throughput 0.62/19.37

  • Also, I keep getting Error Null: Received no content" when talking to Hermes 405b in the chat on OpenRoute (except the free version of Hermes)

So I'm starting to wonder if it could be a problem with the provider, perhaps?

I'd really appreciate any kind of help, thanks.

shadow marsh
#

Their upstream endpoint is very slow at times. It is working right now. Is there any chance OR is dropping the request due to the latency.
Sending "hi" to their upstream took 56 seconds for a response.

#

via OR it just seems to send back an empty response after half a second

untold panther
shadow marsh
#

It has been unreliable for a while. I wish someone else would host it, but the compute resources are significant. Given its place on the leaderboards I would think people would be willing to pay.

#

No worries!

wet juniper
#

i recommend you try out our paid provider deepinfra for better response outcomes. (currently, we cannot guarantee you a speed for a free provider)

untold panther
untold panther
wet juniper
untold panther
# wet juniper yeah since it's upstream related, switching your account wouldn't help. https://...

I see, thanks for providing the docs link. Only, there are a few more questions I would like to ask.

  1. So I can do nothing other than use DeepInfra for now?
  2. I've seen some other users from a month ago that have been going through the same problem. However, others are using Hermes 405b Lambda with no problem while I can't use both the extended and instructed versions. Is this a common problem that happens when using Lambda Hermes? And even if it is, is waiting an only option in this situation?

Thanks.

wet juniper
#

Unfortunately, we cannot guarantee quality and consistent uptime for the free providers of a model.
If you find a better provider for the model, please let us know.

However, others are using Hermes 405b Lambda with no problem, while I can't use both the extended and instructed versions.

If others are using it without problems, it shouldn't be an issue for you either (unless you have used up your free rate limits).

shadow marsh
#

@cinder plank fyi this is still the case

#

direct to them ie https://api.lambdalabs.com/v1 with my api key works fine

cinder plank
#

just straight up Hi???

shadow marsh
wet juniper
shadow marsh
#

oh, right

#

i hope lambda releases a chargeable endpoint, then it wouldnt be overloaded like this 😦
on OR only :free works
but both direct with lambda are working rn. I assume because it is >5am in the US

wet juniper
#

if direct api call works, i think it's due to their internal quota limitation 😢

untold panther
buoyant bison
#

Came here to report the same issue. NH405B isn't giving me responses, both in the chatroom and on the frontend I use. The frontend says it's a 526 error (???)

gloomy grotto
#

they said they're fixing it

#

Lambda had downtime today

lavish snow
#

hope it’ll be up again soon :’)

mighty carbon
#

Lambda-provided hermes 3.1 405b once again not working. Times out