OpenAI Agents SDK token checking on-the-fly

[jaunty vale]

hi, in OpenAI Agents SDK, is there a way to monitor token output in real time to monitor it for errors, and cut it off early before it's potentially used as input into a tool call?

i have these agents connected to some database search tools, and it occasionally generates a very repetitive search string (such as what's attached). this is both a massive waste of tokens generated, it takes extra time for the LLM to produce (plus the search won't work anyway), and i would like to be able to detect this early and throw an exception with a message instructing the agent not to do that.

i'm unable to find anything in the OpenAI Agents SDK documentation about detecting this on-the-fly. does anybody have any advice for this?

[velvet sorrel]

There are some ways to get the token output (with traces for example). However, what you want is to prevent the LLM to output such big content.

1. You can hard cap it with the model settings (I’ll send some code in a minute)
2. You can soft cap it with a good tool prompt, instructing with examples what good inputs look like.
3. If the search will not work with that long string, the tool should reject the task with a message like {success:False, reason:”query too long”}

For me, this is a steering problem. Let’s find out if we can improve the tool prompt to prevent the LLM to generate such long strings at all.

Also, correct me if I’m wrong, but the query in the image feels like it’s the same part repeated many times. Right? This may be a symptom, looks like the LLM is not understanding something, and freaks out.