#how can I insert node by node into a already created vector store using AzureOpenAI
1 messages · Page 1 of 1 (latest)
Hi @pastel whale. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
To insert documents one by one into an already created vector store using Azure Cognitive Search and OpenAI, you can follow the steps outlined in the OpenAI Cookbook's example (https://github.com/openai/openai-cookbook/blob/main/examples/vector_databases/azuresearch/getting_started_with_azure_cognitive_search_and_openai.ipynb).
Here is a Python code snippet from the example that shows how to upload documents to your Azure Search index:
# Convert the 'id' and 'vector_id' columns to string so one of them can serve as our key field
article_df['id'] = article_df['id'].astype(str)
article_df['vector_id'] = article_df['vector_id'].astype(str)
# Convert the DataFrame to a list of dictionaries
documents = article_df.to_dict(orient='records')
# Use SearchIndexingBufferedSender to upload the documents in batches optimized for indexing
with SearchIndexingBufferedSender(search_service_endpoint, index_name, AzureKeyCredential(search_service_api_key)) as batch_client:
# Add upload actions for all documents
batch_client.upload_documents(documents=documents)
print(f"Uploaded {len(documents)} documents in total")
In this example, article_df is a DataFrame where each row represents a document. The 'id' and 'vector_id' columns are converted to strings to serve as key fields. The DataFrame is then converted to a list of dictionaries, which is uploaded to the Azure Search index using SearchIndexingBufferedSender.
If you want to upload documents one by one, you can modify the code to loop over the documents and call batch_client.upload_documents(documents=document) for each one, where document is a single document represented as a dictionary.
Remember to replace search_service_endpoint, index_name, and search_service_api_key with your actual Azure Search service endpoint, index name, and API key.