I need higher token limit in order to generate a context thats comhesive.
Is there no way for me to generate around 8000 tokens per call?
#no way to increase the token limit for gpt finetuned models?
halcyon arch
wraith frigate
I'm a paying user. I also need an increase of tokens per prompt. Currently it's about 2400 words per prompt. Please increase this for paying users.
halcyon arch
Seconded. Increase the max token for paying customers
swift sequoia
You could try breaking your text into smaller blocks. Then you can get more of them into the 2000 token limit. Embedding let's you use 8000 tokens but that won't help you if you want to use whatever you find to ask a question of Davinci
halcyon arch
I need a cohesive coherent result
Breaking down to separate generation results in...
swift sequoia
We might have to wait until the next version of gpt. Ada embedding is 8000 tokens (no help though) maybe the next version of gpt will be 8k too (or maybe not)
daring obsidian
this is a model limitation
waxen tusk
^^^
Token generation complexity is quadratic with sequence length
generating 4096 tokens is already a pretty big feat in itself, generating 8000 would square the complexity, requiring a huge amount of scaling for processing power and etc
Meanwhile, like @swift sequoia said, you can break your text into blocks and use embeddings and indexing to achieve long-context synthesis