I'm getting api responses that are inconsistently incoherent responses for the same inputs. Here's a minimal example of what I'm seeing:
user input: "hi"
api response: "! How I assist today?"
It's a response like this 7 out of 10 times. I don't set a temperature, just default. Also, my system prompt is extremely simple: "You are a helpful assistant."
3 out of 10 times it works, ex:
user input: "hi"
api response: "Hello! How can I assist you today?"
It feels like there's some A/B testing on this api, and there's a broken model in one of the tests. Anyone else seeing these issues. Specifically on gpt-4 only, same on all gpt-4 model options.