Max size limit of prompt
#ae
severe torrent
stray stagBOT
Hi @severe torrent. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
The maximum size limit of a prompt depends on the specific API you are using.
For the Whisper API, the prompt is limited to only 224 tokens. If the prompt is longer than 224 tokens, only the final 224 tokens of the prompt will be considered; all prior tokens will be silently ignored. The tokenizer used is the multilingual Whisper tokenizer (https://github.com/openai/whisper/blob/main/whisper/tokenizer.py#L361). source (https://cookbook.openai.com/examples/whisper_prompting_guide)
For the Assistants API, you can control the token usage in a single Run by setting max_prompt_tokens when creating the Run. This limit applies to the total number of tokens used in all completions throughout the Run's lifecycle. For example, initiating a Run with max_prompt_tokens set to 500 means the first completion will truncate the thread to 500 tokens. source (https://platform.openai.com/docs/assistants/deep-dive)
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
severe torrent
@stray stag whats the max i can sent the max to
stray stagBOT
Let me check the knowledge base for you. One moment! :books:
The maximum limit you can set for max_prompt_tokens is not explicitly mentioned in the provided knowledge sources. However, it's important to note that the total number of tokens (including both input and output tokens) in a single API call to the Chat models is limited to 4096 tokens. If a conversation has more than 4096 tokens, you will have to truncate, omit, or otherwise reduce your text until it fits. source (https://platform.openai.com/docs/guides/chat/)
For the Batch API, a single batch may include up to 50,000 requests, and a batch input file can be up to 100 MB in size. source (https://platform.openai.com/docs/guides/batch/model-availability)
For file uploads, all files uploaded to a GPT or a ChatGPT conversation have a hard limit of 512MB per file. All text and document files uploaded to a GPT or to a ChatGPT conversation are capped at 2M tokens per file. source (https://help.openai.com/en/articles/8983719-what-are-those-file-upload-size-restrictions)
Please refer to the specific API documentation for the most accurate and up-to-date information.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
severe torrent
@stray stag if 4096 is max token limit for input then how many different psoible inputs are there? Do the math
stray stagBOT
Let me check the knowledge base for you. One moment! :books:
The number of possible inputs is not straightforward to calculate because it depends on the language and the specific tokens in that language.
In English, a token can be as short as one character or as long as one word. For example, "a" or "apple" are both considered one token. As a rough rule of thumb, 1 token is approximately 4 characters or 0.75 words for English text. source (https://platform.openai.com/docs/advanced-usage/managing-tokens)
If we simplify the problem and consider each token as a single character, and if we only consider the ASCII characters (128 characters), then the number of possible inputs of length 4096 would be 128^4096.
However, this is a vast oversimplification. In reality, the number of possible inputs is much larger because tokens can represent whole words, and there are many more than 128 words in the English language. Furthermore, OpenAI's models are capable of understanding multiple languages, each with their own set of tokens.