manic thorn
Hey ya'll. Question:
I'm making a web app talking to a custom gpt assistant. I got everything working and whatnot, but I need the API calling highly optimized. Right now, this is how my system works:
I create a thread
I create a message
I run the assistant
Every 2,5 seconds I retrieve the message list to see if the AI answered, until I get an answer, then the polling stops
How can I just print out the answer as soon as it is done running. Is that even possible? I'm doing this in PHP
pure oasis
Hey ya'll. Question:
I'm making a web app talking to a custom gpt assistant. I g...
It is not possible natively with the API
There is no hook in the assistants API to wait for a message to return
So your best bet is to asynchronously check the state of the assistant
You can write something on your own that acts like an await sorta
it shouldn't be too hard to implement
check every 0.5 seconds
sleep asynchronously after checking
and then u can wrap it all around a function and make it usable in a way like a function call
manic thorn
I have a polling system that checks every 2,5 seconds to reduce api calls and costs, but I’m getting the entire message list to check. Is there a more efficient way of checking?
pure oasis
@manic thorn check out https://platform.openai.com/docs/assistants/how-it-works/runs-and-run-steps
Polling for updates
In order to keep the status of your run up to date, you will have to periodically retrieve the Run object. You can check the status of the run each time you retrieve the object to determine what your application should do next. We plan to add support for streaming to make this simpler in the near future.
The status of the Run object tells you what you have to do next, so you don't have to retrieve all messages
e.g if you retrieve the run, and the run state is completed, you know that the assistant has answered you
Or if your assitant makes function calls, the run state can return as requires_action, in which case you can pick it up and kick off a function call
does that help?
manic thorn
Riight gotcha gotcha that’s amazing. Yeah that helps a lot. That should reduce the cost significantly aswell from gathering all messages right?
pure oasis
I don't think gathering messages has any cost?
manic thorn
Oh word? I thought every API call came with a cost, and the bigger the response, the higher the cost
pure oasis
It only incurs a cost for the first time that the model generates the text
So when it first creates a message, it will cost you the amount of tokens it generated for that message + the input tokens (not sure how they do this behind the scenes)
but if you're just retrieving already-generated messages from the assistant, it's free
manic thorn
Well I appreciate the help, I’ll implement checking the run instead. I have a few more questions you might be able to answer if you have time?
pure oasis
Sure!
manic thorn
but if you're just retrieving already-generated messages from the assistant, it'...
Gotcha that’s amazing, thanks
manic thorn
Sure!
Right so sometimes when im retrieving the AI answer, the text value is an empty string. Is this simply an AI error or something wrong with my API calls..
pure oasis
It may be an issue with your prompt or the way that you are collecting your messages together or the way you're sending it to the assitant
sometimes it'll return empty text if it thinks it doesn't have to respond
You can try printing out the conversation history before the API call to make sure the prompting and the messages are what you expect
manic thorn
I’ll get on the train real quick and pull out my laptop so I can further provide you with the responses I get etc, will you be available in about 15 minutes?
pure oasis
Probably not but I'll check back in when I have some time and respond
manic thorn
Probably not but I'll check back in when I have some time and respond
That’s amazing, thank you so much