what is the difference between context windows (gpt-3.5/gpt4 models) and tokens (text embedding models)
#Context window
balmy gull
frigid wraithBOT
Hi @balmy gull. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
The context window and tokens are two different concepts in the operation of OpenAI's language models like GPT-3.5 and GPT-4.
Context Window: The context window of a model refers to the maximum number of tokens from the input that the model can consider when generating a response. For instance, the GPT-3.5 Turbo model has a context window of 16,385 tokens, while the GPT-4 Turbo model has a larger context window of 128,000 tokens [source (https://platform.openai.com/docs/models)]. This means that GPT-4 Turbo can consider a larger chunk of input text when generating a response compared to GPT-3.5 Turbo.
Tokens: Tokens are the chunks of text that the model processes. Text generation and embeddings models process text in chunks called tokens. Tokens represent commonly occurring sequences of characters. For example, the string " tokenization" is decomposed as " token" and "ization", while a short and common word like " the" is represented as a single token. Note that in a sentence, the first token of each word typically starts with a space character [source (https://platform.openai.com/docs/introduction)].
One limitation to keep in mind is that for a text generation model the prompt and the generated output combined must be no more than the model's maximum context length. For embeddings models (which do not output tokens), the input must be shorter than the model's maximum context length [source (https://platform.openai.com/docs/introduction)].