Does it cost more tokens if you put multiple messages in the request or does it just use tokens for the last message in the array?
#Does it cost more tokens if you put multiple messages in the request?
rich hazel
covert bane
API charges for all input tokens. API requires you to local buffer yoir past prompts/responses.
rich hazel
API charges for all input tokens. API requires you to local buffer yoir past pr...
What does local buffer past prompts/responses mean?
covert bane
OpenAI API does not store your prompts after response. If you want the prior prompts/ responses as context then you have to save them ( buffer, store, not discard) and resend in next prompt.
rich hazel
Yes, I would save them in an array of messages. So when I send that whole array of messages back to the API, will it charge me for all the messages, not just the last one?
@covert bane
covert bane
Yes. All tokens sent/ received are billed as tokens. Why would you expect OpenAI to distinguish the tokens you send via the API? They could not do it. P.S. you could use a FIFO array. Or you could use prior_message.append and limit its length in FIFO manner.
rich hazel
What is a FIFO array?
covert bane
Ask Chat GPT ( free) your basic questions. Or ask chat GPT to write the python script to do this as described.