Unexpected role RoleEnum.system only for one model?

[normal crypt]

i, I am currently evaluating the mistral API and the different models available and I am confused about an error message I get when using open-mixtral-8x7b but not when using open-mistral-7b.
The error is Unexpected role RoleEnum.system

I am sending messages in the order user, assistant, system, user. With the system message being a specific prompt to be used to answer the following user message. As I said, it works fine with open-mistral but not on open-mixtral, everything else is exactly the same. Why would the API behave differently?

@hollow blaze suggested that system messages are only allowed as the first message. But that's neither documented nor does it seem to be true for all models?

[silk rune]

Strange that it works for some models, but I would always argue to not try that at all, as it is rarely an intended format with any model.

E.g. anthropic doesn’t even allow system messages and instead you have to pass the system prompt as a separate, single parameter as it is always the prefix of the whole conversation. And most models are trained that way, so deviating from that structure likely won’t work as you expect it to or degrade performance

I would use system, user, assistant, user, assistant, … with any model, regardless of the API allowing otherwise or not