#Perplexity r1-1776 sometimes returns empty responses

9 messages · Page 1 of 1 (latest)

novel arrow
#

Rencetly I've been trying to benchmark perplexity/r1-1776 model, but:

  1. In responses there's a ton of whitespace looking like this in front of the actual json text:
00000000  20 20 20 20 20 20 20 20  20 0a 0a 20 20 20 20 20  |         ..     |
00000010  20 20 20 20 0a 0a 20 20  20 20 20 20 20 20 20 0a  |    ..         .|
00000020  0a 20 20 20 20 20 20 20  20 20 0a 0a 20 20 20 20  |.         ..    |
00000030  20 20 20 20 20 0a 0a 20  20 20 20 20 20 20 20 20  |     ..         |
00000040  0a 0a 20 20 20 20 20 20  20 20 20 0a 0a 20 20 20  |..         ..   |
00000050  20 20 20 20 20 20 0a 0a  20 20 20 20 20 20 20 20  |      ..        |
00000060  20 0a 0a 20 20 20 20 20  20 20 20 20 0a 0a 20 20  | ..         ..  |
00000070  20 20 20 20 20 20 20 0a  0a 20 20 20 20 20 20 20  |       ..       |
00000080  20 20 0a 0a 20 20 20 20  20 20 20 20 20 0a 0a 20  |  ..         .. |
00000090  20 20 20 20 20 20 20 20  0a 0a 20 20 20 20 20 20  |        ..      |

All are spaces and line feed characters. Example responses in attachments were saved with the code below (no processing at all after post() call):

            response = requests.post(
                url = "https://openrouter.ai/api/v1/chat/completions",
                headers = { "Authorization": f"Bearer {api_key}" },
                data=json.dumps(request_data),
            )

            if is_verbose:
                with open(f"/tmp/{quiz_id}-response.txt", "w") as f:
                    f.write(response.text);
                print(f"[{quiz_id}] Response: {response.text.strip()}", file=sys.stderr)
  1. For longer prompts sometimes all I get are the mentioned whitespace characters, there's no json in response (response.text.strip() is empty). Then I get json parse errors and of course my money still gets eaten.

Please fix this crap.

📎 6835caf5d0ad9c81e41cdb1544daebd5-response.txt 📎 528c0353136bf9e655783a6370755cae-response.txt
undone ibex
#

If you're not using streaming, then this is intentional. OpenRouter will send back periodic whitespace in the response to ensure that the connection stays alive.

novel arrow
novel arrow
# undone ibex If you're not using streaming, then this is intentional. OpenRouter will send ba...

I ran the script again and checked each request and response. The script made 12 requests, got 4 correct responses:

  • "id":"gen-1740497624-aKWLVc6J1K5bMw5vRe5G"
  • "id":"gen-1740497624-TtmaZ2UaIYja2jIJn3ls"
  • "id":"gen-1740497624-rhzN2SQSpvjDoy3coshK"
  • "id":"gen-1740497624-dl2Mx4xhxVobl1pgxWIt"

then the mentioned error happened:

[6835caf5d0ad9c81e41cdb1544daebd5] Response: 
[6835caf5d0ad9c81e41cdb1544daebd5] Response status code: 200
[6835caf5d0ad9c81e41cdb1544daebd5] Caught exception: Expecting value: line 1423 column 1 (char 7821), trying again.

As you can see the response (stripped) is blank, status code is 200, json parsing failed.

BUT maybe I was wrong about this eating money from my account. I don't see this response in activity, it's like it didn't exist at all. Still weird.

#

Perplexity r1-1776 returns empty responses

#

Perplexity r1-1776 sometimes returns empty responses

undone ibex
#

I'm not OpenRouter staff so I can't debug this for you, but if you log the time it takes for you to get a response, I wouldn't be surprised if there's a timeout somewhere, potentially between OpenRouter and Perplexity. My suggestion is to switch to streaming responses, especially when outputs are large (like with reasoning models) and can take multiple minutes to generate.

novel arrow