Whats the recommends amount
#max_tokens
undone isle
minor oceanBOT
Hi @undone isle. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
The recommended amount for the max_tokens parameter depends on the specific model you're using and the context of your task. For instance, for the gpt-3.5-turbo-1106 model, the maximum context length is 16,385 tokens, but the output tokens are limited to 4,096. Therefore, when calling the API, you should ensure that max_tokens <= 4096 and input_tokens + max_tokens <= 16385 source (https://community.openai.com/t/589155).
However, there is no way to set a minimum number of tokens source (https://help.openai.com/en/articles/5072518-controlling-the-length-of-openai-model-responses).
The max_tokens parameter is a way to control the length of a model response. You can also influence the length of the model response by providing instructions to generate the desired output length, adding examples of a specific length, or using strategic stop sequences source (https://help.openai.com/en/articles/5072518-controlling-the-length-of-openai-model-responses).
Please note that the token limits can vary depending on the model used. Some models, like GPT-4 Turbo, have different limits on input and output tokens source (https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them#h_051eb08805).
For models like GPT-4 Turbo with vision, there is currently a low max_tokens default which you can override source (https://community.openai.com/t/500636).
In the future, OpenAI plans to add support for specifying a maximum number of tokens the Assistant or tool should use source (https://community.openai.com/t/534869).
Remember to always check the documentation for the specific model you're using to understand the token limits and how to count them. You can compute token counts using OpenAI's counting tokens notebook (https://platform.openai.com/docs/guides/fine-tuning).
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
undone isle
----------------------
The recommended amount for the `max_tokens` parameter dep...
Is 100 tokens enough for a question of the day messae
@minor ocean Is 100 tokens enough for a question of the day message
minor oceanBOT
Let me check the knowledge base for you. One moment! :books:
The number of tokens needed for a "question of the day" message would depend on the length and complexity of the question.
In English, a token can be as short as one character or as long as one word. For example, "ChatGPT is great!" is encoded into six tokens: ["Chat", "G", "PT", " is", " great", "!"]. So, a 100-token limit could potentially accommodate a question of moderate length and complexity.
However, it's important to note that both the input and output tokens count towards these quantities. For example, if your API call used 10 tokens in the message input and you received 20 tokens in the message output, you would be billed for 30 tokens source (https://platform.openai.com/docs/guides/text-generation).
To see how many tokens are used by an API call, you can check the usage field in the API response (e.g., response**'usage']['total_tokens']). To count tokens in a text string without making an API call, you can use OpenAI’s [tiktoken** (https://github.com/openai/tiktoken) Python library source (https://platform.openai.com/docs/guides/text-generation).