Providing knowledge base to chatgpt API for customer support

[sage crater]

Hello, i want to make a customer support chatbot with openAi's api, and i need to upload a knowledge base for the chatbot to use. I tried using file-share provided in the assistant api, but it eats up so many tokens ( about 4000 tokens per query for only 3kb file!). So after some search i decided to use RAG with the api instead, can anyone give me any help or resources to start with?

[dim aspen]

assistants file-search is RAG and if you do it manually you will also pass the info to the context. you can limit file search with https://platform.openai.com/docs/assistants/tools/file-search/customizing-file-search-settings

[sage crater]

So there is not point of implemnting RAG myself? it will not reduce the token usage?

[dim aspen]

you would have a bit more control over it but essentially you would do the same what the file search already does. https://help.openai.com/en/articles/8868588-retrieval-augmented-generation-rag-and-semantic-search-for-gpts

[sage crater]

Well i saw this comment on reddit ( not an answer for me)

[sage crater]

what do u think about this?

[dim aspen]

what's the date on the post? the file search used to be garbage with only few options to tweak it but has improved since

[sage crater]

6 months ago

So you suggest i just tweak with file search a lil to reduce the usage?

[dim aspen]

that's the old one

[sage crater]

do u have any suggestions on how to tweak exactly?

[dim aspen]

yes try different values for best result/cost

limit number of chunks