is there anything I can do to lower the input context size ? I'm using cerebras with 1mil daily limit and I was very shocked that it run out in a minutes becuase of the input context size
#lower input context
quiet marsh
rough seal
You can trigger the context condensing for a lower threshold.
primal needle
Yes, there is an option in Settings under Context where you can reduce the % until automatic condensing.
You can also click this button in the UI, or type the command /smol
quiet marsh
yea I know about the condense but I meant to mainly reduce the input token amount
condense will shrink it but soon later it'll grow back to same size
so how can we start with something already low !?
primal needle
@idq - are you asking about reducing the very first initial message? Or the size of potential individual items in the context? (file contents, etc)
quiet marsh
@idq - are you asking about reducing the very first initial message? Or the size...
the very initial one