#Gemini 1.5: 500 errors when processing >15 images (docs say 3,600 supported)

14 messages · Page 1 of 1 (latest)

dull glacier
#

Hi! I'm trying to use Gemini 1.5 with image batches and running into some issues. According to the docs, Gemini 1.5 supports up to 3,600 images, but I'm getting 500 errors when trying to process more than ~15 images at once. For my current request, I'm trying to send 128 images. even 20 images pretty consistently fails.
When i print my completion, we see:

ChatCompletion(id=None, choices=None, created=None, model=None, object=None, service_tier=None, system_fingerprint=None, usage=None, error={'message': 'Internal Server Error', 'code': 500}, user_id='user_id')

Here's a minimal reproducible example:

import json
from openai import OpenAI

# Read messages containing multiple images
with open('debug_messages.txt', 'r') as f:
    messages = json.loads(f.read())

client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key="api_key",
)

completion = client.chat.completions.create(
    model="google/gemini-flash-1.5",
    messages=messages
)
print(completion.choices[0].message.content)

This works fine with 15 or fewer images but fails with 500 errors when trying to process more. The messages follow the format outlined in the docs:

{
    "role": "user",
    "content": [
        {"type": "text", "text": "Some text"},
        {"type": "image_url", "image_url": {"url": "https://example.com/image1.jpg"}},
        {"type": "image_url", "image_url": {"url": "https://example.com/image2.jpg"}},
        # ... more images
    ]
}

Is this a known limitation? Any suggestions for handling larger batches of images? For now I've worked around it by processing in chunks of 15 images, but wanted to check if there's a better approach.

dull glacier
#

Hi there— any guidance on this? I know support sometimes takes time, so no worries if an answer is still cooking 🙂

shut mulch
#

Hey @dull glacier yeah let me look into this for you, sorry for the delay

dull glacier
shut mulch
#

Hey @dull glacier, I looked into this a bit for you - it seems that what the model supports and what Google actually supports on their infra is a bit different. Various reports from users trying to do similar things to you note that this is an intermittent problem, and that possibly swapping models or re-trying helps.

Also, per their docs (which I assume you've already read, so apologies if this is redundant):

image.png
dull glacier
#

Ok, cool! Thanks for the info. I’ll definitely add in (more) retries, and try Logan. Thanks!

shut mulch
dull glacier
#

I can work around it for now by just batching, but it would be really great if you would poke around a little more and/or escalate! lmk if you need any more info of course.

dull glacier
#

Ah, so this is an interesting tidbit. After a couple tests, it's looking like for the really long ones, setting provider to Google (Which looks like it directs to Google Vertex) takes a while (O(1.5 minute)), but works. Seems like it fails when it's Google AI Studio.
added this:

extra_body={
    "provider": {
        "order": ["Google"],
        "allow_fallbacks": False
    }

}

dull glacier
#

@shut mulch ok, one last question and this can be closed. Slow and getting content in a response is good enough for now. Is there a way to choose the provider in typescript using the OpenAI sdk? It looks like they don't have or plan on having an extra_body parameter.

shut mulch
dull glacier
#

ok, i didn't really scour the docs. i'll use that as a starting point! thanks for all the help 🙂