can someone explain to me something how to chatgpt with gpt 3.5 when he only had 4096 memory tokens to make a discussion and always remember the beginning when he only had 4096 memory tokens 🧐
#gpt 3.5
shy yarrow
full starBOT
Hi @shy yarrow. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
When interacting with GPT-3.5 Turbo models, you need to manage the conversation history to ensure that the model has the necessary context to generate meaningful responses. The model has a context window of 16,385 tokens, which includes both the input and output tokens. However, the maximum number of output tokens that can be returned in an API call is 4,096 tokens. This means that if your conversation history (input tokens) is too long, it might limit the length of the model's responses.
Here's an example of how you might structure your API call:
$data = '{
"model": "gpt-3.5-turbo",
"messages": [
{
"role": "user",
"content": "'. $question. '"
}
]
}';
In this example, $question would be replaced with the conversation history and the current user message. You need to ensure that the total number of tokens (including both the conversation history and the model's response) does not exceed the model's context window.
If your conversation exceeds the model's context window, you might need to truncate or omit some parts of the conversation. However, be aware that very long conversations are more likely to receive incomplete replies. For example, a gpt-3.5-turbo conversation that is 4090 tokens long will have its reply cut off after just 6 tokens.
As for remembering the beginning of the conversation, OpenAI is testing memory with ChatGPT. You can explicitly tell it to remember something, ask it what it remembers, and tell it to forget conversationally or through settings. However, the memory feature is still in testing and might not be available to all users.
Sources:
- Forum Post: API Remembering conversation (https://community.openai.com/t/732733)
- Docs > Guides > Text generation models > Reproducible outputs Beta Part 3 (https://platform.openai.com/docs/guides/text-generation)
- Help Article > En > Articles > Memory and new controls for ChatGPT (Feb 13, 2024) (https://help.openai.com/en/articles/6825453-chatgpt-release-notes#h_30ceb8d7ba)
- Docs > Models > GPT-3.5 Turbo (https://platform.openai.com/docs/models)