steep temple
I'm doing a chat. I hit a problem, too many tokens sent to openai.
OK, I want to lower the max context tokens.
Another problem, when I do this, this apparently stops meilisearch from doing its thing. When I mention names from the past, the model can't recall the details.
What I want to do specifically is limit the number of messages sent back by default, while leaving room for meilisearch to inject its results into the prompt.
Not sure what to do here.
hasty apex
I'm doing a chat. I hit a problem, too many tokens sent to openai.
OK, I want t...
Try to disable "artifacts" from the beta features section of the settings. That features uses a lot of tokens.
steep temple
?
Already off. The conversation is just too long.
Actually I worry now that Meilichat may well have not been doing anything at all, and the only reason it appeared to work on deepseek is because the old information was still within the massive context window.
hasty apex
meilisearch indexing is automatic, restarting LibreChat should retrigger it if necessary
steep temple
I know that it's on. I don't know whether or not it is doing thing I thought it was doing.
For example, in chatbox, there is this at the bottom there. Max message count in context. I am trying to find a similar setting here.
hasty apex
the meilisearch index is only for the search box, the model can't access these results. Meilisearch results are not injected into the prompt
steep temple
OK, so then now I have two problems.
First, I need a way to set the maximum message count in context.
Second, I need a way to inject "memories" into my prompts.
Sillytavern does this through Lorebooks, as pictured.
But chatgpt seems to have a way of "searching" previous messages for keywords and reinjecting key information into prompts way later into the conversation.
hasty apex
1 - You can set max context but cannot set max messages
2 -"Memories" have been suggested a couple of times, you could comment about that here: https://discord.com/channels/1086345563026489514/1342217637748408392
You might want to enable summarization if you like to have very long convos
steep temple
OK. How do I enable summarization?
hasty apex
for custom endpoints you do it in the librechat.yaml summarize: true
https://www.librechat.ai/docs/configuration/librechat_yaml/object_structure/custom_endpoint#summarize
here's a random (and outdated) example that shows you where it goes:
steep temple
OK. If I set max context tokens, will that include the summary?