What is "Finish reason: 2"? (And is that

[real patio]

it's from gemini. this guy gets a similiar (different finish reason number) error when he's trying to ask gemini how to influence an election:

https://medium.com/@its.jwho/errorhandling-vulnerability-tests-on-gemini-19601b246b52

i can't find much else on this error though. curious why i'm getting it when the prompt is 13K, but not when it's 6K!

[vast timber]

Aha! Normally we see those as strings.
https://ai.google.dev/api/rest/v1/GenerateContentResponse#FinishReason

I'm guessing 2 corresponds to "MAX_TOKENS"

My initial guess is that the combined input and output tokens is going over 32k, but that does seem odd.

Are you streaming the response?

Which platform is this on again? Vertex AI or AI Studio?

[real patio]

thanks for the super quick reply! did a rough token count with llamaindex observabiliity to see what's fed into the LLM, and the input + output are less than 14K for a 13K token prompt.

i'm on vertex ai. response is not streamed.

i can't find this error anywhere else online though, so i assume it's some quirk in the llamaindex implementation (it was supposed to just feed a string to gemini).

i'll try implement it with raw python code instead of llamaindex

[vast timber]

It is also possible they are setting the max_token parameter to something smaller.

It also isn't an error condition, really. It just means it hit the max number of tokens outputted before it finished replying.

(Sorry, thinking about it further.)

(This is different than a finish reason of 3, which is SAFETY, or some of the others. Those may not produce any output at all.)

[real patio]

thanks again, so much, for your help. you're probably right. and it is something to do with the llamaindex functions.

i just tried the equivalent thing in langchain (passing a long strong to gemini), and it's worked fine

[vast timber]

Glad you got it working!
Does sound like a bug in llamaindex. With a very strange output to boot. I'd open a bug report with them.