pure totem
The bot's responses are cut off, and sometimes the end of the response is gibberish. What is the token limit for responses? If there is a limit, what is it? I am not using an API on sites like 'char', but rather 'NextChat' or aichat in CLI. Is there any way to increase this limit?
Garbage fragment example:
rigid knoll
The bot's responses are cut off, and sometimes the end of the response is gibber...
Please provide all the details when you ask a question.
How can I answer you if I don't know which model you're using? What are the generation parameters you used?
wicked spruce
i think it's the context.. Not max response length
if there's something like Max Response Length or Max new token, set them on 4000 since it's the maximum. But with that picture, i think it's just indicating your context size (by how much message count will be used for context) and not your max response length
pure totem
calculation from example 'codechat' above:
word count: 273, token count: 524, tokens used: 995
Screenshots from NextChat, another example same question.
Im gonna check with Original API
word count: 313, token count: 578, tokens used: 1077
wicked spruce
seems good, this is original API?
