#SillyTavern "group chat" messages to Deepseek (provider) fail silently

1 messages · Page 1 of 1 (latest)

fervent spear
#

When using the "group chat" feature in SillyTavern and requesting a reply from Deepseek v3 with Deepseek as the provider, SillyTavern shows that it is waiting for a response which never comes (streaming on or off). There is no error shown, absolutely nothing in the console. OpenRouter does not show a request sent on my Activity page. When I switch to Fireworks as the provider, everything works as normal.

dawn frigate
#

yes i am seeing something odd with deepseek too

lavish niche
#

+1 (it never generates past the first token which is typically within 10 seconds or so)

No errors, just an endless wait.

dawn frigate
#

this might be linked to your sillytavern configuration and not openrouter. I was having the exact same problem with avante (a neovim extension) and only openrouter . Fixed it by fixing my avante lua configuration.

fervent spear
#

This was a completely fresh ST install

#

I don't even understand how it could be a thing on a technical level. As far as I'm aware, everything except the parameters / samplers is a giant text blob that gets text-completed

#

If I send in 5000 zeroes it should still give me a response

lavish niche
sage harbor
#

Same - working perfectly with Fireworks but none of the other providers

lavish niche
sage harbor
#

Testing for RP using ST.

lavish niche
#

(not that it matters since I'm getting an error via Fireworks too, provider returned error)

Ah well

dawn frigate
#

so don't discredit any hypothesis yet.

#

Here's a question: If you go to your activity on openrouter,

#

do you see requests that return 0 completion tokens?

#

and if you click on these requests to see the metadata , what is the status code returned by the API provider (is it 200?)

#

Also second test you might wanna try is sending a curl request to openrouter and seeing if the problem persists

#

example:

# 
fetch("https://openrouter.ai/api/v1/chat/completions", {
  method: "POST",
  headers: {
    "Authorization": `Bearer ${OPENROUTER_API_KEY}`,
    "HTTP-Referer": `${YOUR_SITE_URL}`, // Optional, for including your app on openrouter.ai rankings.
    "X-Title": `${YOUR_SITE_NAME}`, // Optional. Shows in rankings on openrouter.ai.
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    "model": "mistralai/mixtral-8x7b-instruct",
    "messages": [
      
      {"role": "user", "content": "Hello"},
      
    ],
    "provider": {
      "order": [
        "OpenAI",
        "Together"
      ]
    },
  })
});

taken from a post by someone else in general..
pay special attention to the providers you elect.
run this test up to 10 times during the day.
If it generates properly , it's most probable that sillytavern might be the problem.
If it doesn't always generate properly. it means there is a service problem somewhere int he pipeline between openrouter and the differnet providers.

lavish niche
#

If it worked before, and now it doesn't, and there were no changes on my end, I am, along with any other users of the service, going to assume it's OpenRouter and not me. DeepSeek V3 worked for me earlier, now it doesn't. Thank you for the guidance though, if I was seeing the behavior described it'd help tremendously

Sucks that discord has such poor searchability of information, it's actual, literal garbage

lavish niche
#

@languid heart this was the last successful generation for me (UTC+0530) from OR. Is this a known issue?

No further generations resulted in even zero-token output on the activity log. Really sorry to ping you

image.png
languid heart
#

hm, it's working fine for me now, but DeepSeek's API was surging with upstream hangups yesterday (an issue with their API rather than OpenRouter) - that could have been it. everything ok now?

fervent spear
#

I'm...getting even more confused. I created a new group chat and it's working fine. But when I try to have a character speak in an older group message, I get:

#

So I now have one group chat where things work properly. One group chat where ST loads forever, never returns an error, and the request never shows on OR. And one chat where it fails with a 404 🙃

hot portal
#

I thought it was Sillytavern too, so I did a fresh installation, checked out various other fixes, made sure there was nothing interfering, tested it on other models, everything else seems to be working fine.

It has gotten to the point that only one in out of five swipes or requests are working with Deepseek v3, which is sad because it is such a good model. :(

potent sand
fervent spear
potent sand
#

The max context should be 64k, which is what I have set in sillytavern as well. So it's below max context for sure. If you have one group chat that works and one that doesn't, I wouldn't be surprised if the failing one is around 10k tokens long.

fervent spear
potent sand
#

Hmm, that means it was working at 29k though, so maybe not the same issue. If you wanted to test, you could fill up the working chat with garbage messages to see if it stops working at around 10k-13k. Alternatively, take the 30k chat, change your sillytavern context length to 10k (so it automatically truncates the middle of the chat) and see if it starts working.

fervent spear
#

It wasn't working at 29K, it was falling back to Fireworks without me knowing

fervent spear
#

Something still going on. DeepSeek provider still fails with a 404, but it accidentally fell back to the "NovitaAI" provider and it replied to the roleplay with just "pop. limply" lol