How to pass image URL to Vision models

[half escarp]

Hey guys, I'm testing LlaVa and Nous-Hermes Vision.

Below is the code I'm currently passing to the payload (The format of GPT-4 vision).

I get this error response {'error': {'message': 'Missing image input', 'code': 400}}

What is it?

Code:

        url="https://openrouter.ai/api/v1/chat/completions",
        headers={
            "Authorization": f"Bearer {OPENROUTER_API_KEY}",
            "HTTP-Referer": "localhost",  # Optional, for including your app on openrouter.ai rankings.
            "X-Title": "Testing",  # Optional. Shows in rankings on openrouter.ai.
        },
        data=json.dumps({
            "model": "haotian-liu/llava-13b",  # Optional
            "messages": [
                {
                    "role": "system",
                    "content": [
                        {"type": "text", "text": system_message},
                    ],
                },
                {
                    "role": "user",
                    "content": [
                        {
                            "type": "image_url",
                            "image_url": {
                                "url": image_url,
                                "detail": detail
                            }
                        },
                    ],
                },
                {
                    "role": "user",
                    "content": [
                        {"type": "text", "text": prompt},
                    ],
                }
            ],
        })
    )```

[lethal meadow]

cc @late prism

[late prism]

Ah we are grabbing just the "last Multipart message"

Will fix this behavior

I wasn't sure if the array content would be used this way, was thinking a mix and match.

[half escarp]

Thanks, let us know!