Structured Output with DeepSeek-R1: How to account for differences between providers?

[ancient jasper]

I am trying to understand which providers of the DeepSeek-R1 model provide support for structured output, and, if so, in what form, and how to request it from them. Given that this seems to be quite different from one provider to the next, I am also trying to understand how to account for those differences when using DeepSeek-R1 via OpenRouter (i.e., not knowing which provider will end up serving my request). I did a detailed write-up of my question on Reddit - I will post the link here, as it is too long for this Discord: https://www.reddit.com/r/openrouter/comments/1inpby4/structured_output_with_deepseekr1_how_to_account/ @fallow perch

[ancient jasper]

Nevermind, I figured it out: https://openrouter.ai/docs/features/provider-routing?error=true#requiring-providers-to-support-all-parameters-beta

[rapid terrace]

@ancient jasper what was the confusion? Would love to know how we can improve our docs.

[ancient jasper]

I described the problem in detail in the reddit post above. Really, though, I was kind of an idiot, because the solution is plainly stated on your page about structured outputs. I guess the only thing I find slightly confusing / perhaps less than ideal is that, when you search for a model in the models catalog and filter by some feature, e.g. structured outputs, the models that get listed may be provided by several providers, and some of those providers may, in fact, not support the feature you were filtering for. That's not immediately obvious from the layout of the model catalog page though. The implication, to me, when filtering for a feature in the catalog is that I can use any and all of the models that get listed as having that feature without needing to worry, when in fact, I need to ensure I only use the model hosted by a provider that supports the feature, see e.g. https://openrouter.ai/models?fmt=cards&order=newest&supported_parameters=structured_outputs

[fallow perch]

Thanks for the writeup, super valuable feedback :)

[rapid terrace]

@ancient jasper you don't need to worry about the provider because if you pass in json_schema we will filter for providers that support it

[ancient jasper]

oh, i didn't realize that. Then the whole problem I was trying to figure out how to avoid really does not exist in the first place. But then, what is the require_parameters field even for? Actually, it says there: "With the default routing strategy, providers that don’t support all the LLM parameters specified in your request can still receive the request, but will ignore unknown parameters. When you set require_parameters to true, the request won’t even be routed to that provider." https://openrouter.ai/docs/features/provider-routing?error=true#requiring-providers-to-support-all-parameters-beta

[fallow perch]

this is for “less important” parameters like topk and stuff, but that’s a good point, we should make that clearer

cc @rapid terrace correct me if i’m wrong

[rapid terrace]

Yup correct. We should fix that documentation.

[gilded abyss]

That doesn't seem to be actually true? It seems like it falls back to json_object when not supported or something? Because it generates outputs that do not always fit the schema and I have to retry etc

[rapid terrace]

@gilded abyss I added an internal ticket to verify this will get back to you on Monday

[rapid terrace]

@gilded abyss Just checked you need to specify "require_parameters" to true. Docs: https://openrouter.ai/docs/features/provider-routing#requiring-providers-to-support-all-parameters-beta

[gilded abyss]

Hmmm, I can't specify that with openai client.beta.chat.completions.parse though?