Text stops generating in the middle of a response | LibreChat | Page 1

novel dirge Nov 15, 2023, 6:16 PM

#

I’ve had a number of users report this issue. Two users report that it only happens in longer conversations, leading me to believe that it might have to do with the context window handling.

floral orbit Nov 15, 2023, 7:08 PM

#

so this happens because the LLM will limit the response in proportion to the amount of prompt tokens. the only way around this is to make the context window "smaller"

fickle garden Nov 15, 2023, 7:21 PM

#

could enabling "message summarization" help in that situation?

#

since

it only happens in longer conversations

floral orbit Nov 15, 2023, 7:42 PM

#

yeah it does

novel dirge Nov 24, 2023, 1:33 AM

#

fickle garden could enabling "message summarization" help in that situation?

How do I enable message summarization? Is it in the .env?

#Text stops generating in the middle of a response