stream mode splits SSE message mid-json, causing problems with some http client/json parser

[autumn dew]

I am getting messages where chunking happens mid-json like:

data: {"id": "cmpl-8238bb566a84454bb3f1b5b872377e82", "object": "chat.completion.chunk", "created": 1702644750, "model": "mistral-medium", "choices": [{"index": 0, "delta": {"role": null, "content": " squad"}, "finish_reason": null}]}

data: {"id": "cmpl-8238bb566a84454bb3f1b5b872377e82", "object": "chat.completion.chunk", "created": 1702644750, "model": "mistral-medium", "choices": [{"inde

[-----------------new chunk-------------------]

x": 0, "delta": {"role": null, "content": ":"}, "finish_reason": null}]}

data: {"id": "cmpl-8238bb566a84454bb3f1b5b872377e82", "object": "chat.completion.chunk", "created": 1702644750, "model": "mistral-medium", "choices": [{"index": 0, "delta": {"role": null, "content": " Ar"}, "finish_reason": null}]}

data: {"id": "cmpl-8238bb566a84454bb3f1b5b872377e82", "object": "chat.completion.chunk", "created": 1702644750, "model": "mistral-medium", "choices": [{"index": 0, "delta": {"role": null, "content": "sen"}, "finish_reason": null}]}

I don't know if this is valid by spec or not, but it would simplify things for clients if every chunk only contained valid self-contained json.

[heady iris]

Definitely not expected, we’re looking at it

[heady iris]

Hello @autumn dew , we’ve made some changes. Can you let me know if this is still an issue?