#how can I insert node by node into a already created vector store using AzureOpenAI

1 messages · Page 1 of 1 (latest)

pastel whale
#

how can I insert node by node or document by document into a already created vector store using AzureOpenAI

sinful parcelBOT
#

Hi @pastel whale. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

To insert documents one by one into an already created vector store using Azure Cognitive Search and OpenAI, you can follow the steps outlined in the OpenAI Cookbook's example (https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/azuresearch/getting_started_with_azure_cognitive_search_and_openai.ipynb).

Here is a Python code snippet from the example that shows how to upload documents to your Azure Search index:

# Convert the 'id' and 'vector_id' columns to string so one of them can serve as our key field 
article_df['id'] = article_df['id'].astype(str) 
article_df['vector_id'] = article_df['vector_id'].astype(str) 

# Convert the DataFrame to a list of dictionaries 
documents = article_df.to_dict(orient='records') 

# Use SearchIndexingBufferedSender to upload the documents in batches optimized for indexing 
with SearchIndexingBufferedSender(search_service_endpoint, index_name, AzureKeyCredential(search_service_api_key)) as batch_client: 
 # Add upload actions for all documents 
 batch_client.upload_documents(documents=documents) 

print(f"Uploaded {len(documents)} documents in total") 

In this example, article_df is a DataFrame where each row represents a document. The 'id' and 'vector_id' columns are converted to strings to serve as key fields. The DataFrame is then converted to a list of dictionaries, which is uploaded to the Azure Search index using SearchIndexingBufferedSender.

If you want to upload documents one by one, you can modify the code to loop over the documents and call batch_client.upload_documents(documents=document) for each one, where document is a single document represented as a dictionary.

Remember to replace search_service_endpoint, index_name, and search_service_api_key with your actual Azure Search service endpoint, index name, and API key.

#

Please note that this code assumes that you have already created a vector store (i.e., an Azure Search index) and that your documents are represented as a DataFrame. If your documents are in a different format, you will need to adjust the code accordingly.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: