how do you load documents or large text files with text-generation-webui?

[uncut rose]

is it possible? the ui does not have any sort of load capacity that I can see

[uncut rose]

please?

[unborn crag]

I'm also looking for a solution to this, but no time to implement and test right now until tomorrow. My findings to test are:

https://www.reddit.com/r/LocalLLaMA/comments/18dohlt/tutorial_use_real_books_wiki_pages_and_even/

And:

https://github.com/oobabooga/text-generation-webui/pull/3272

Something something, RAG with ChromaDB seems to be the solution. Just there's no direct extensions mentioned, and superbooga was recently taken off the extensions github page I assume due to the creator no longer working on it.

[dapper verge]

when I apply suberboogav2 I get the following warning: 2023-12-18 09:25:13 WARNING:Using embedded DuckDB without persistence: data will be transient

Is it normal?

[uncut rose]

i get this too. there must be an option somewhere to make it persistent

[uncut rose]

thanks , but it looks like a lot of work to get not even close to what h2ogpt can do