0 token completion from Lambda

[dense roost]

I've been receiving a lot of 0 token completions from the Lambda provider for deepseek-chat-v3-0324 model.
Generation IDs:
https://openrouter.ai/api/v1/generation?id=gen-1744865894-ywnmbfM61oHEGd11hJJF
https://openrouter.ai/api/v1/generation?id=gen-1744856548-duKnHQemc0iik1Xy8GLz
https://openrouter.ai/api/v1/generation?id=gen-1744857457-HJLS1cHk1vjlWHptXKJg
https://openrouter.ai/api/v1/generation?id=gen-1744858913-0PeHIQj5DGwuBZUcxMPn
https://openrouter.ai/api/v1/generation?id=gen-1744859158-UXx8ZM7OotgjCwxxqwA9

All of them also had a "stop" finish reason so they're billed. I did filter them out now, but when DeepSeek was down all requests were being routed to Lambda. @thin junco

The inference seems to be broken too when it does return something, comparing to other providers with the same input

[thin junco]

Hey @dense roost can you share any sample code for these kind of requests? specifically around sampling params like max tokens or stop sequence

[dense roost]

your API defaults. The only sampling params I'm passing are:

"temperature": 0.7,
"top_p": 0.95,

[tulip scaffold]

Send your fetch request

[dense roost]

data = {
    "model": "deepseek/deepseek-chat-v3-0324",
    "messages": [
        {"role": "system", "content": TRANSLATION_PROMPT},
        {"role": "user", "content": text},
    ],
    "provider": {"ignore": ["DeepInfra"], "sort": "price"},
    "temperature": 0.7,
    "top_p": 0.95,
}

response = requests.post(
    "https://openrouter.ai/api/v1/chat/completions",
    headers=headers,
    data=json.dumps(data),
)

[tulip scaffold]

import requests
import json

Assuming 'headers' is a dictionary containing necessary headers like Authorization

Assuming 'data' is the Python dictionary payload you want to send as JSON

Example definitions (replace with your actual data and headers):

headers = {
'Authorization': 'Bearer YOUR_API_KEY',
# Content-Type is NOT needed here when using the json parameter
}
data = {
"model": "deepseek/deepseek-chat-v3-0324",
"messages": [
{"role": "system", "content": "TRANSLATION_PROMPT"},
{"role": "user", "content": "text"},
],
"provider": {"ignore": ["DeepInfra"], "sort": "price"},
"temperature": 0.7,
"top_p": 0.95,
}

response = requests.post(
"https://openrouter.ai/api/v1/chat/completions",
headers=headers,
json=data # Use the 'json' parameter here
)

You can then check the response

print(response.status_code)

print(response.json())