Hi @mortal cape. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
#Interaction Architecture
limpid swiftBOT
Based on the information provided in the knowledge sources, here are some thoughts on your proposed implementations:
Implementation #1: This approach seems straightforward and could work well if the information extracted from the images is sufficient for the problem-solving process. However, it's important to note that the quality of the assistant's responses will heavily depend on how well the information from the images is translated into text. If any crucial information is lost or misrepresented during this process, it could lead to incorrect or unhelpful responses from the assistant.
Implementation #2: This approach might not work as expected because, according to a forum post (https://community.openai.com/t/730338), the assistant API does not have the capability to interpret image attachments. The assistant can only process text inputs.
Implementation #3 and #4: These approaches seem to be more complex and might require more development effort. According to the same forum post (https://community.openai.com/t/730338), the assistant API can handle file IDs as parameters, but it's not clear how it would interpret or use these IDs. It's also not clear whether OpenAI provides a service for storing files or vectors.
In terms of designing your own implementation, it's important to keep in mind the principles of prompt engineering. According to an OpenAI help article (https://help.openai.com/en/articles/4936848-how-do-i-create-a-good-prompt-for-an-ai-model-like-gpt-4), the models often perform best when interacted with as if you are sending another human a request. So, providing clear and detailed instructions can increase the chance of success.
Finally, it's worth noting that the assistant API does not have web searching capabilities, as mentioned in another forum post (https://community.openai.com/t/779162). This means that the assistant cannot fetch or retrieve information from the web on its own. You would have to provide all the necessary information as part of the input.
Given these considerations, I would recommend starting with Implementation #1, as it seems to be the most straightforward and feasible approach based on the capabilities of the assistant API. However, it's important to ensure that the information extracted from the images is accurate and comprehensive.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
mortal cape
@limpid swift If i prepare a vector store in OpenAI and link it to an assistant made in openAI. When i prompt the assistant, i can not tell it to fetch some info from the vector store itself?
limpid swiftBOT
Let me check the knowledge base for you. One moment! :books:
Yes, you can link a vector store to an assistant in OpenAI. According to a forum post (https://community.openai.com/t/747575), OpenAI SDKs have been updated to send Assistants beta v2 headers, which means they use file_search as a method to enhance knowledge. This allows the assistant to retrieve data from the vector store.
However, it's important to note that there are some known issues with this approach. For example, another forum post (https://community.openai.com/t/783317) mentions that an assistant was able to retrieve data from the vector store via the playground, but not via the API call.
As a workaround, the user in the forum post mixed manual embedding with the assistant API, which resulted in excellent results and minimized token usage. The user's workflow involved creating an embedding of the pages via the embedding API, saving the embedding to a local database table with the text, and then carrying out a vector search on the local embedding store when a user asks a question.
So, while it's possible to prompt the assistant to fetch info from the vector store, there might be some limitations or issues that you need to be aware of.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
mortal cape
@limpid swift What is the difference between Vector Store in openAI, and Files?
limpid swiftBOT
Let me check the knowledge base for you. One moment! :books:
Based on the knowledge sources provided, OpenAI's Vector Stores and Files seem to serve different purposes and have different properties.
Vector Stores: According to the OpenAI API documentation (https://platform.openai.com/docs/api-reference), a Vector Store is a collection of processed files that can be used by the file_search tool. It contains properties such as an identifier, object type, creation timestamp, name, usage in bytes, file counts (in progress, completed, failed, cancelled, total), status (expired, in progress, completed), and an expiration policy. The Vector Store seems to be designed for storing and managing large collections of processed files, which can be used for tasks like file search.
Files: The documentation does not provide specific details about Files in OpenAI. However, based on the context, it seems that Files refer to individual files that are attached to a Vector Store. These files have properties such as an identifier, object type, usage in bytes, creation timestamp, the ID of the Vector Store they are attached to, status (in progress, completed, cancelled, failed), and potentially a last error and chunking strategy.
In summary, a Vector Store is a collection of processed files, while Files are the individual files within that collection. Each has its own set of properties and uses within the OpenAI ecosystem.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
mortal cape
@limpid swift If i put 10 files in a vectore store, and when i ask the model to retrieve something, will it retrieve it based on the information of all of the 10 files? Or i can specifically order it to fetch it from a specific file out of those 10 files?
limpid swiftBOT
Let me check the knowledge base for you. One moment! :books:
Based on the information provided in the knowledge sources, when you use the file_search tool with a Vector Store, the assistant retrieves chunks of data from the files in the Vector Store to find the relevant information. If multiple files are searched to find the relevant information, the tokens from all these files’ chunks are included in the input token count. This is mentioned in a forum post (https://community.openai.com/t/805817).
However, it's not explicitly stated whether you can direct the assistant to search within a specific file within the Vector Store. In another forum post (https://community.openai.com/t/802201), a user asked a similar question, but there was no clear answer provided.
Therefore, it seems that by default, the assistant may search across all the files in the Vector Store. If you need more control over which files are searched, you might need to create separate Vector Stores for each file or set of files, although this could potentially increase costs.
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
mortal cape
@limpid swift How will i be charged for the vector stores? Can i create as many vector store as i need? Or there is a cap on the count of active vector stores?
limpid swiftBOT
Let me check the knowledge base for you. One moment! :books:
Based on the information provided in the knowledge sources, the billing for Vector Stores is as follows:
- All Vector Stores created before the release of the
v2API (April 17, 2024) will be free to use until the end of 2024. This includes any Vector Stores that were created as a result of mapping yourv1data tov2before thev2launch. After the end of 2024, they’ll be billed at whatever the fees for Vector Stores are at that point. You can check the pricing page (https://openai.com/pricing) for the latest pricing information. - Any Vector Store that is created before the release of the
v2API (April 17, 2024) but not used in a single Run between that release date and the end of 2024 will be deleted. This is to avoid starting to bill you for something you created during the beta but never used. - Vector Stores created after the release of the
v2API will be billed at current rates as specified on the pricing page (https://openai.com/pricing).