Incompatibility between Mistral and OpenAI API for image content

[livid spindle]

I was trying to use the OpenAI .NET client to call Pixtral with image content, since we'd already successfully used it for text generation.

I found that Mistral expects 'image_url' to be formatted as a string:

{
                "type": "image_url",
                "image_url": f"data:image/jpeg;base64,{base64_image}" 
}

While OpenAI expects it as an object:

{
          "type": "image_url",
          "image_url": {
            "url": f"data:image/jpeg;base64,{base64_image}"
          }
}

Was this intentional for any reason?

I was getting back an HTTP 400, and narrowed it down to the image content. If I removed the image content, the API call succeeded.

It now makes calling Pixtral not compatible with OpenAI clients, I believe?

[crystal dome]

Hi ! Actually, even if not mentionned in the documentation, both options should work 👍

[livid spindle]

Ah, that's good to know. I'm still unclear what would be causing the error then. The image was <1MB in size.

The Mistral API is returning HTTP 400 with this response.

{
"code": 2210,
"param": null,
"object": "error",
"type": "invalid_request_filter",
"message": "Could not parse body - request body did not fit into client body buffer, consider raising 'client_body_buffer_size'"
}