#Best memory option

1 messages · Page 1 of 1 (latest)

lost osprey
#

Hi, I was wondering if there was a good option on how to do a json file based option for a memory for a openai model in python

merry rootBOT
#

Hi @lost osprey. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

Yes, you can use Redis as a context store with OpenAI's Chat Completions. You can load data files into Redis as JSON objects with text and vector fields. Here is an example of how you can do this in Python:

import os
import openai

directory = './assets/'
model='text-embedding-3-small'
i = 1
for file in os.listdir(directory):
    with open(os.path.join(directory, file)) as f:
        content = f.read()
        vector = openai.Embedding.create(input = [content], model = model)['data'][0]['embedding']
        client.json().set(f'doc:{i}', '$', {'content': content, 'vector': vector})
    i += 1

In this example, the script reads files from a directory, creates embeddings for the content of each file using OpenAI's text-embedding-3-small model, and then stores the content and its corresponding vector in Redis as a JSON object.

You can also use RedisJSON for storing vectors as JSON with OpenAI. Here is an example of how you can load data into Redis as JSON objects:

client.json().set('doc:1', '$', doc_1)
client.json().set('doc:2', '$', doc_2)
client.json().set('doc:3', '$', doc_3)

In this example, doc_1, doc_2, and doc_3 are Python dictionaries that you want to store as JSON objects in Redis.

Please note that you need to have a Redis instance with the Redis Search and Redis JSON modules, and the Redis-py client library installed. You can install the necessary Python modules with the following command:

! pip install redis openai python-dotenv openai[datalib]

You also need to have an OpenAI API key. You can create a .env file and add your OpenAI key to it like this:

OPENAI_API_KEY=your_key