#Empty response from OpenRouter api for grok-4-fast

37 messages · Page 1 of 1 (latest)

distant fog
#

I am using SillyTavern for roleplay with the grok-4-fast free model via OpenRouter api.
It works mostly for single chats, but almost never work for group chats.

Example response is as follows:
{
id: 'gen-1758500363-1vize67QJwKV4tN8kjW0',
provider: 'xAI',
model: 'x-ai/grok-4-fast:free',
object: 'chat.completion',
created: 1758500363,
choices: [
{
logprobs: null,
finish_reason: 'stop',
native_finish_reason: 'stop',
index: 0,
message: {
role: 'assistant',
content: '',
refusal: null,
reasoning: null
}
}
],
system_fingerprint: 'fp_9362061f30',
usage: {
prompt_tokens: 1606,
completion_tokens: 204,
total_tokens: 1810,
prompt_tokens_details: {
cached_tokens: 1591,
audio_tokens: 0
},
completion_tokens_details: {
reasoning_tokens: 204
}
}
}

Does this mean the provider returned an empty content, or that OpenRouter failed to parse a content from the provider's response? Or any other possibilities?

junior bramble
distant fog
# junior bramble Is this error still happening?

Hi Eternal,
It continues but can be mitigated to some extent.
I am only guessing, as I don't know what happened at AI provider and OpernRouter.
I think it is likely a provider problem.
I tried further and found out that, in a group chat, if I add a user message (even an empty) to be the last message, it will respond ok (as it worked in single chat).
For example,
mesages:[
...,
{role:user,content:...}
]
Instead of,
mesages:[
...,
{role:assistant/system,content:...}
]

If the message roles are delivered to providers, grok-4-fast may somehow depend on user message to work properly.

junior bramble
distant fog
junior bramble
#

Its an OpenRouter issue

distant fog
junior bramble
#

@solid cape care to take a look at this? OpenRouter is failing to normalize messages for grok-4-fast

solid cape
#

Can we get a full request that was sent to OpenRouter?

#

I assume on a group chat, there are multiple users hence the last chat might not come from the user role

#

Curious what the shape of this os

junior bramble
# solid cape Can we get a full request that was sent to OpenRouter?

Here's the reproduction


async function main() {
const url = "https://openrouter.ai/api/v1/chat/completions";
const headers = {
  "Authorization": `Bearer No API key for you today :D hehe`,
  "Content-Type": "application/json"
};
const payload = {
  "model": "x-ai/grok-4-fast:free",
  "messages": [
    {
      "role": "assistant",
      "content": "If you built the world's tallest skyscraper, what would you name it?"
    }
  ]
};

const response = await fetch(url, {
  method: "POST",
  headers,
  body: JSON.stringify(payload)
});

const data = await response.json();
console.log(util.inspect(data, { depth: null }));
}

main()```

# Note
This works fine with `nvidia/nemotron-nano-9b-v2:free` and `x-ai/grok-3` but using it with Grok-4-fast returns an empty string

### Response from `nvidia/nemotron-nano-9b-v2:free` 

```{
  id: 'gen-1759009846-GEHD09AxCrpX1MGVPFja',
  provider: 'Nvidia',
  model: 'nvidia/nemotron-nano-9b-v2:free',
  object: 'chat.completion',
  created: 1759009846,
  choices: [
    {
      logprobs: null,
      finish_reason: 'stop',
      native_finish_reason: 'stop',
      index: 0,
      message: {
        role: 'assistant',
        content: '',
        refusal: null,
        reasoning: " I want you to give the skyscraper a name that's unique, and that somehow reflects deadlines, planning and the process of building something.\n",
        reasoning_details: [
          {
            type: 'reasoning.text',
            text: " I want you to give the skyscraper a name that's unique, and that somehow reflects deadlines, planning and the process of building something.\n",
            format: 'unknown',
            index: 0
          }
        ]
      }
    }
  ],
  usage: { prompt_tokens: 28, completion_tokens: 32, total_tokens: 60 }
}```
#

Response from Grok 3

{
  id: 'gen-1759010012-CIZnIyVRxGct8uKbc94h',
  provider: 'xAI',
  model: 'x-ai/grok-3',
  object: 'chat.completion',
  created: 1759010012,
  choices: [
    {
      logprobs: null,
      finish_reason: 'stop',
      native_finish_reason: 'stop',
      index: 0,
      message: {
        role: 'assistant',
        content: `If I were to name the world's tallest skyscraper, I’d call it "Skyspire Apex." The name combines "Skyspire," evoking the image of a towering structure piercing the heavens with elegance and ambition, and "Apex," signifying the ultimate peak or pinnacle of human achievement. It conveys a sense of grandeur, innovation, and dominance in both design and purpose. What do you think—got any names in mind?`,
        refusal: null,
        reasoning: null
      }
    }
  ],
  system_fingerprint: 'fp_f02e6df795',
  usage: {
    prompt_tokens: 21,
    completion_tokens: 86,
    total_tokens: 107,
    prompt_tokens_details: { cached_tokens: 0, audio_tokens: 0 },
    completion_tokens_details: { reasoning_tokens: 0 }
  }
}```
#

Response from Grok 4 fast

Notice the empty response

{
  id: 'gen-1759010143-Cl5xtOla6tU9GS7nSwSy',
  provider: 'xAI',
  model: 'x-ai/grok-4-fast:free',
  object: 'chat.completion',
  created: 1759010143,
  choices: [
    {
      logprobs: null,
      finish_reason: 'stop',
      native_finish_reason: 'stop',
      index: 0,
      message: {
        role: 'assistant',
        content: '',
        refusal: null,
        reasoning: null
      }
    }
  ],
  system_fingerprint: 'fp_9362061f30',
  usage: {
    prompt_tokens: 131,
    completion_tokens: 45,
    total_tokens: 176,
    prompt_tokens_details: { cached_tokens: 110, audio_tokens: 0 },
    completion_tokens_details: { reasoning_tokens: 45 }
  }
}```
stiff inlet
#

same issue that's only present with grok4-fast:free any other model responds fine

junior bramble
#

@solid cape did you triage this yet?

solid cape
#

Hmm.. we can add a padding user role yeah

#

cc @craggy nimbus

craggy nimbus
#

if it doesn't get fixed soon then we will fix it on our end with prefill

craggy nimbus
# junior bramble Here's the reproduction ```const util = require('util') async function main() ...

So the "real" issue here is misuse of the assistant role by SillyTavern.

In a group chat scenario with multiple AI characters, you shouldn't be using the assistant role for all the characters.

Rather the assistant role should only be used on messages authored by the current model that you're passing the request to.

So the "correct" form of your request would be to simply use the user role instead, e.g.:

  "messages": [
    {
      "role": "user",
      "content": "If you built the world's tallest skyscraper, what would you name it?"
    }
  ]
distant fog
# craggy nimbus So the "real" issue here is misuse of the `assistant` role by SillyTavern. In a...

Hi, thanks for the diagnosis and clarification. Please correct me if I misunderstood.
Assume 1 user and 2 bots in a group,
If bot 1 is supposed to generate a message, all messages of user and bot 2 are considered "user input" for bot 1, as if the user is inpersonating multiple characters.
The same goes for bot 2, as if user is impersonating all characters except bot2.

Based on the result, I suppose I should report this to SillyTavern for a fix?

BTW, I am curious if the role information is passed to providers. If passed, would they possibly consider marking AI messages as user role a violation of the OpenRouter API?

distant fog
# craggy nimbus So the "real" issue here is misuse of the `assistant` role by SillyTavern. In a...

Hi, I had a second thought while writing the reproduction steps for a bug report to SillyTavern, and wonder if the issue is really resolved in this way.

Based on your suggestion, let's say a minimal case, a regular chat, or a group chat with 1 bot.
If I want the bot to generate an initial message, what should be the roles in messages?
There is no assistant message at the moment, and system ones only (e.g., scenario, rules, and backgrounds).

Similarly, in a chat, if I want the bot to continue with more messages after its response, what should be the roles?
The last message is from the same bot. It should be assistant as I understood. However, this has the similar request role labelling as the group chat scenario, except for the difference in message contents.

These makes me confused.

craggy nimbus
#

Notice how user is described as "an end user", meaning its more open-ended and can be any user.

While assistant is very specifically for messages authored by the model.

#

Similarly, in a chat, if I want the bot to continue with more messages after its response, what should be the roles?

Ideally your app would keep track of which messages were authored by each model. And then the next turn when you send a request to a model, make all messages from that specific model assistant role, and the rest user

#

If multiple models are responding on a single turn then you'd have to adapt this for each model

distant fog
# craggy nimbus > Similarly, in a chat, if I want the bot to continue with more messages after i...

I think I understand your suggestion but the confusion is for exmaple.
Case 1: Only 1 bot in either group or single chat.
User: blahblah
Bot: 1111

Now, user want bot to generate another message.
The request messages should be:
role: user, content: "user:blahblah",
role: assistant, content: "1111"

Case 2: Invalid role labelling in a group chat (this post reported issue).
User: blahblah
Bot 1: 1111

Now user want bot 2 to respond, but the role is labelled wrong.
The request messages are:
role: user, content: "user:blahblah",
role: assistant, content: "1111"

As the second results in empty response from the AI model, the first will respond empty.

#

If we believe the grok-4-fast is capable to continue with more messages in a regular chat (case 1), it would be weird it cannot in case 2 (the reported issue).

craggy nimbus
#

Yeah your app would have to handle labeling each message role properly on each turn AND for each model

craggy nimbus
distant fog
craggy nimbus
craggy nimbus
#

Update: looks like xAI fixed this behavior on their end!

junior bramble
#

@distant fog pinging you to let you know xAI fixed it