#Embeddings Input Array Max Length

1 messages · Page 1 of 1 (latest)

normal delta
#

https://api.openai.com/v1/embeddings

The documentation says "Input text to embed, encoded as a string or array of tokens. To embed multiple inputs in a single request, pass an array of strings or array of token arrays. The input must not exceed the max input tokens for the model (8192 tokens for text-embedding-ada-002), cannot be an empty string, and any array must be 2048 dimensions or less."

I want to batch-embed an array of strings, but the API docs don't say what the maximum length this array can be.

static mortarBOT
#

Hi @normal delta. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

The OpenAI API documentation specifies that when you're passing an array of strings or an array of token arrays to the Embeddings API, the array must be 2048 dimensions or less. This means that the maximum length of the array of strings you can pass for batch embedding is 2048.

Remember, the total number of tokens across all strings in the array must not exceed the maximum input tokens for the model, which is 8192 tokens for the text-embedding-ada-002 model.

Here is the relevant extract from the API documentation:

#

{'required': True, 'content': {'application/json': {'schema': {'type': 'object', 'additionalProperties': False, 'properties': {'input': {'description': 'Input text to embed, encoded as a string or array of tokens. To embed multiple inputs in a single request, pass an array of strings or array of token arrays. The input must not exceed the max input tokens for the model (8192 tokens for text-embedding-ada-002), cannot be an empty string, and any array must be 2048 dimensions or less. Example Python code (https://cookbook.openai.com/examples/how_to_count_tokens_with_tiktoken) for counting tokens.\n', 'example': 'The quick brown fox jumped over the lazy dog', 'oneOf': {'type': 'string', 'title': 'string', 'description': 'The string that will be turned into an embedding.', 'default': '', 'example': 'This is a test.'}, {'type': 'array', 'title': 'array', 'description': 'The array of strings that will be turned into an embedding.', 'minItems': 1, 'maxItems': 2048, 'items': {'type': 'string', 'default': '', 'example': "['This is a test.']"}}, {'type': 'array', 'title': 'array', 'description': 'The array of integers that will be turned into an embedding.', 'minItems': 1, 'maxItems': 2048, 'items': {'type': 'integer'}, 'example': '[1212, 318, 257, 1332, 13]'}, {'type': 'array', 'title': 'array', 'description': 'The array of arrays containing integers that will be turned into an embedding.', 'minItems': 1, 'maxItems': 2048, 'items': {'type': 'array', 'minItems': 1, 'items': {'type': 'integer'}}, 'example': '[[1212, 318, 257, 1332, 13]]'}], 'x-oaiExpandable': True}, 'model': {'description': 'ID of the model to use. You can use the [List models (/docs/api-reference/models/list) API to see all of your available models, or see our Model overview (/docs/models/overview) for descriptions of them.\n', 'example': 'text-embedding-ada-002', 'anyOf': [{'type': 'string'},

#

{'type': 'string', 'enum': ['text-embedding-ada-002']}], 'x-oaiTypeLabel': 'string'}, 'encoding_format':

#

@normal delta kapa.ai is still learning and improving, please let us know how it did by reacting below

normal delta
#

@static mortar It's not clear if a single input's max is 8192 tokens or if all inputs must not collectively exceed 8192 tokens.

static mortarBOT
#

Let me check the knowledge base for you. One moment! :books:

#

The maximum token limit applies to the total number of tokens across all inputs in the batch, not to each individual input. This means that if you're using the text-embedding-ada-002 model, the total number of tokens in all the strings in your array should not exceed 8192 tokens.

This is evident from a forum post where a user was trying to upload an array of texts to the OpenAI Embedding API using the text-embedding-ada-002 model. The user was encountering an error message indicating that the total number of tokens requested exceeded the model's maximum context length of 8191 tokens, even though each individual text was within the limit.

Here is the relevant extract from the forum post:

I’m trying to upload an array of texts to the OpenAI Embedding API using the text-embedding-ada-002 model, which should have a token limit of 8191, but it sometimes tells me I have gone over the limit even though I am not.
I even tried lowering the token size of each array to be way lower, like 5000 tokens, but it’s telling me the size of tokens I sent is 8500+.
Am I correctly assuming that the token limit for the batched request is cumulative of every element in the array? and not individual elements in an array?

Source (https://community.openai.com/t/316546)

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@normal delta kapa.ai is still learning and improving, please let us know how it did by reacting below

normal delta
#

@static mortar In that post it seems like that wasn't actually true later on

static mortarBOT
#

Let me check the knowledge base for you. One moment! :books:

#

I apologize for the confusion. Based on the information available in the extracts, it's not clear whether the token limit applies to each individual input or the total tokens across all inputs in the batch. The forum post does suggest that the token limit might be cumulative across all elements in the array, but it doesn't provide a definitive answer.

To get a clear answer, I recommend checking the official OpenAI API documentation or reaching out to OpenAI support. They should be able to provide a definitive answer on how the token limit applies when using the Embeddings API with batch inputs.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@normal delta kapa.ai is still learning and improving, please let us know how it did by reacting below

lime field
#

@static mortar how many dimensions does text-embedding-3-small model have?

static mortarBOT
#

Let me check the knowledge base for you. One moment! :books:

#

The text-embedding-3-small model has an output dimension of 1536. This means that the embeddings generated by this model will be vectors with 1536 dimensions.

Here is the relevant extract from the OpenAI documentation:

Model Description Output Dimension
text-embedding-3-small New Embedding V3 smallIncreased performance over 2nd generation ada embedding model 1,536

Source (https://platform.openai.com/docs/models)

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

lime field
#

@static mortar what is text-embedding-3-small priced at?

static mortarBOT
#

Let me check the knowledge base for you. One moment! :books:

#

The text-embedding-3-small model is priced per input token. The example given in the OpenAI documentation suggests that you can process approximately 62,500 pages of text per US dollar, assuming ~800 tokens per page.

Here is the relevant extract from the OpenAI documentation:

Model Rough pages per dollar Example performance on MTEB (https://github.com/embeddings-benchmark/mteb) eval
text-embedding-3-small 62,500 62.3%

Please note that this is an approximation and the actual cost may vary depending on the number of tokens in your input.

Source (https://platform.openai.com/docs/guides/embeddings)

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@lime field kapa.ai is still learning and improving, please let us know how it did by reacting below

lime field
#

@static mortar please tell me how much $ per 1000 tokens

static mortarBOT
#

Let me check the knowledge base for you. One moment! :books: