#OpenAI API - FileSearch Tool in Thread-A seems to have access to vector-store of Thread-B.

1 messages · Page 1 of 1 (latest)

sage plaza
#

Steps to reproduce:

  1. Use the API to create a Thread (with id thread-A)
  2. Use the API to create a Thread (with id thread-B)
  3. Add a message to thread-A using the API, attach a file (file-X) to it, this is a resume.
  4. Add a message to thread-B using the API, attach a file (file-Y) to it, this is a resume.

Expected result:

  1. When prompting in thread-A (with a ThreadRun), I must retrieve data that can solely be found in file-X (as opposed to file-Y)
  2. When prompting in thread-B (with a ThreadRun), I must retrieve data that can solely be found in file-Y (as opposed to file-X)

Actual result:

  1. In Thread-A I sometimes get data from file-Y, and Thread-B shows data from file-X. When prompting in a thread, the assistant should only be able to access the files in the vector-store of that specific thread.

Additional information

  1. The assistant does NOT have its own files/vector-store attached.
  2. Using an Assistant with FileSearch enabled, with model gpt-4o.
  3. These are not hallucinations, the data is too specific.
  4. These are not race-conditions in the application, I start a new process with each prompt. These processes are not running concurrently.
  5. When looking in the platform.openai.com user-interface, the File, VectorStore and Thread objects all have the correct relations/references, while the data accessed by the FileSearch Tool clearly had a broader scope than these relations/references.
  6. This behavior is very hard to reproduce. Sometimes it happens, usually it doesn't. The workaround that worked was deleting all vector-stores using the API, and running it prior to any prompt (which forces all I/O with OpenAI to be single-threaded, and rather destructive in nature).

MAYBE this is all a misunderstanding - is there a possible 'account / organization / assistant'-level vector-store? In the UI there doesn't seem to be a trace of it - on the Assistant-level there were no files attached.