#giving the bot ability to remember the previous comments of the users in discord conversations

13 messages · Page 1 of 1 (latest)

verbal pier
#

I have implemented davinci-003 model on a discord chatbot that plays a roles and it works fine but I’m being creative about how to manage him to have “memory” and remember the conversation he has with people

At the start every conversation was new for the bot and it couldn’t remember anyone

I connected it to a database that feeds in the request the last 3 comments of the user he is talking with so now at least can have a minimum conversation remembering the last messages

But it feels quite ineffective plus I have to request a lot of tokens for each comment

Is there any known solution for this?

drowsy shale
#

Hi, you can send the conversation history in the prompt to enable chatting

#

e.g, your first prompt would look like

# 
Human: Hello
AI:
#

Davinci would return a response for example like "Hey there!"

#

Then, if the user asked another question for example "What is your name", you make a new prompt as follows:

Human: Hello
AI: Hey there!
Human: What is your name
AI:
#

Does that make sense?

#

This will allow you to store the conversation history in a basic level

#

this will let you store the history up to 4097 tokens

#

A level up from that, would be using embeddings, you embed user prompts and GPT responses and search for the most relevant ones during answer time to find good context

#

There are libraries that do this nicely for you, such a gpt-index

verbal pier
# drowsy shale this will let you store the history up to 4097 tokens

Hi there, thanks for your reply

Yes that’s the method I have figured out, in fact I feed the bot with the last three comments of the users and I use a cache to reduce the use of tokens sent in the request however it feels quite ineffective since for replying a comment I need to feed the request with three of them all the time so I was wondering if there was a kind of established system that I was missing to create the maximum possible context by using the minimum amount possible of tokens to reduce the cost, because when the community is active it literally a lot of money

So I don’t know who is using 4000 tokens per request but that’s gonna be thousands of dollars for a few days

Unless there is a free model and I’m missing it 🤔