#Azure OpenAI interrupting mid-stream on latest main branch?

1 messages · Page 1 of 1 (latest)

oblique dragon
#

Hi there, if I try to chat with gpt-4o with Azure OpenAI using the latest main branch it breaks mid-stream, with an unhandled error:
2024-10-14T16:15:48.628Z debug: [saveConvo] api/app/clients/BaseClient.js - saveMessageToDatabase #saveConvo 2024-10-14T16:15:54.068Z warn: [OpenAIClient.chatCompletion][stream] Unhandled error type 2024-10-14T16:15:55.756Z warn: [OpenAIClient.chatCompletion][finalChatCompletion] Unhandled error type 2024-10-14T16:15:55.757Z error: [OpenAIClient.chatCompletion] Unhandled error type Error: Cannot read properties of undefined (reading 'content') 2024-10-14T16:15:55.757Z error: [handleAbortError] AI response error; aborting request: Error: Cannot read properties of undefined (reading 'content') 2024-10-14T16:15:55.758Z error: [handleAbortError] error while trying to abort message Cannot set headers after they are sent to the client

I tried to change API version (now using 2024-09-01-preview, but same result with older ones), remove the streamRate parameter in the config, switched the model to the gpt-4o-2024-08-06 version, but no dice. Anybody else has faced this?

dark crater
#

Same issue here

#

seems to be soemthing with azure

#

(having issues with my personal projects too, with 4o and 4o-mini

dark crater
cursive mirage
#

same here, most likely azure issues... its been pretty unstable past few days

dark crater
#

just find out how to do, but still the inference is quite slow: I use sweden central for the location

cursive mirage
#

also using sweden central and yeah... its a mess rn

oblique dragon
#

Same, also on sweden central. Been broken for a week now - wtf are MS doing?? And it's not a librechat issue, even trying to generate a completion in the chat playground will stop halfway through.

oblique dragon
oblique dragon
oblique dragon
#

@dark crater @cursive mirage I fixed it eventually. The problem is that i had a content filter using the 'new' asynchronous mode. This was causing it to just error out halfway through the completion. By replacing it with the older default mode it now manages to generate the full completion without errors.

#

Also, Librechat atm doesn't handle content filtering errors so you get a generic error. I had made a change to handle them but I never deployed it to my instance, as I needed to test it a bit better and didn't have much time to do it. If I finish it I might PR it.

cursive mirage
oblique dragon
# cursive mirage how do you replace it? older api version?

go to your azure openai resource -> Deployments -> manage deployments, then in azure ai studio, on the left choose content filters, if you have multiples, check which one your model is using atm and edit that, then in the second page on the bottom change that streaming mode from asynchronous to default. That's what fixed it for me.

image.png image.png
cursive mirage
#

thats just fkn stupid jesus christ

#

i love microsoft

oblique dragon
cursive mirage
cursive mirage
oblique dragon
cursive mirage
#

classic microsoft

#

why make it easy when you can make it this way

mild pollen
#

also last week sweden central was down for the newest gpt-4o version. they didnt even tell anyone. fix was to go back to june version of gpt-4o (which works better than newest one but costs double)

oblique dragon
#

Are you guys able to upload any files to an assistant in azure openai now? I think this issue is librechat-specific, as it works in the assistants playground. Now I'm getting 400 unknown parameter error, messages.file_ids related. I think it started happening as part of the last changes to the assistants?

mild pollen
#

we dont use assistants on azure sry

oblique dragon
#

Ok fixed this one too. I had even made a PR changing how the assistants work, then danny told me i was mixing v2 and v1 assistants api. So I looked closely into it, and it looks like MS fixed an oversight where 2024-05-01-preview, which was the first to support v2 assistants, would still accept v1 file attachments, now they corrected it and it errors out. Switching further back to 2024-02-15-preview - which was the last api with v1 assistants support - fixed the issue.