ChatGPT API will have to maintain session

[finite kestrel]

I'm thinking about what will change when the CharGPT API becomes available - which is coming "soon".
Somehow it will have to maintain session... Does anyone have info on how this will work?

[half sparrow]

nope no official info but the way its maintaining season im assuming it just keeping different context just as you would with the current available APIs

[finite kestrel]

So you're suggesting that there is nothing special about it and it's just that the whole conversation history is passed in... 🤔 that's a possibility

[gritty void]

Hi

[whole flax]

hlo

[half sparrow]

yep basically it doesnt actually have past memory or true session divider its just when ur sending in a new question it includes all the previous question and answers of the session u pick as context

[hot shard]

Hi

[austere fjord]

couldn't you just have a conversation id?

[trim oxide]

well only up to a point

the previous questions it can reference is limited, so for larger projects where you might want it to reference a lot of information you'd probably need to create a new data bank to feed it for reference

[half sparrow]

yes but that is just how it is both the API and ChatGPT has limitation on how much context it can go thru at once which IIRC is around 4000 tokens

so even if u sending in bit by bits to chatGPT once your context + prompt passes its limitation (4096 tokens?)it starts to "forget" the older context

https://github.com/jerryjliu/gpt_index

away to deal with it

[trim oxide]

I know, and hopefully we'll find more ways of dealing with that in the future since larger datasets are gonna be needed for many potential use cases.