JSON response mod | Learn AI Together | Page 1

quick spindle Jun 30, 2024, 9:44 PM

#

How does it work? I've seen other apis implement it, but I'm not sure if it's something related to the model or something applied to the response of the model afterwords or what. I'm looking at adding it to my api but not sure how.

#

https://docs.together.ai/docs/json-mode#:~:text=JSON mode corrals the LLM,the schema property of response_format .

Together

JSON Mode

JSON mode corrals the LLM into outputting JSON conforming to a provided schema. To activate JSON mode, provide the <button aria-label="Copy Code" class="rdmd-code-copy fa"></button>response_format parameter to the Chat Completions API with <button aria-label="Copy Code" class="rdmd-co...

astral night Jul 1, 2024, 2:12 PM

#

This is called structured-json-response and OpenAI calls it function calling, in this model generates a valid JSON object as response which is further used for calling python-function or making some other API calles, this is very useful in alot of tasks like rendering some LLM generated data in plots, multi-agent workflow etc etc, these models like mistral, mixtral, gpt3.5 are explicitly trained on function calling data, while you can use other models too to generate this kind of response but then you might need to do another step of processing by maybe using regex to extract json

#

Latest open source instructions tuned LLM (llama,gemma, etc)are capable of doing the same, by giving sysPromt to generate JSON data.

quick spindle Jul 2, 2024, 12:31 AM

#

astral night Latest open source instructions tuned LLM (llama,gemma, etc)are capable of doing...

could you provide a link to this. thanks

#

to an example? or just post one

astral night Jul 2, 2024, 2:35 AM

#

not sure how these API providers like together.ai,open-router, firework, does this, but if I find something later I'll share 👍

#JSON response mod