Chat model

[queen egret]

I am trying to use openAI API, and it's working fine, however the answers I am getting are far worse than from the browser. Maybe I need to use another model? Which one?

[daring plover]

I had this experience too initially when using the turbo model. Over a few days of testing it got better. Share your code and response, happy to take a look 🙂

[queen egret]

response = openai.ChatCompletion.create(
                model="gpt-3.5-turbo",
                messages=messages[id])

That is the only part that matters. messages[id] stores prompts and responses for each user (bot operates in DMs), but they reset every time I restart the bot.

I honestly don't see why I would need to store them anyway, it's not like it is important.

[limber wolf]

can u print the contents of the messages dictionary?

[queen egret]

messages[id].append({"role": "assistant", "content": response["choices"][0]["message"]["content"]})

It's filled with elements of this kind.

obviously, it is empty when the bot restarts.

[slim quiver]

You shouldn't. You can just pull message history on a per-channel basis, and each DM is its own channel.

I store chat history to a variable, but that var is lost when the process finishes.

Granted this way you'll use marginally more bandwidth but still

[white token]

I've noticed a wide variation in the quality of responses over days and hours, with it being much less capable for a period of time and then the opposite where sometimes you'll get a few hours where it will do things that are like magic. I have a personal custom chat client with fully adjustable everything, automated conversation summarizing etc etc and well honed system messages, so I know nothing on my end is changing.

[cloud adder]

could you share how you set up custom chat client? what's the difference between fine tuning? I'm stuck in that😧

[somber viper]

dude, if you want help you really need to be more specific and give people enough data to understand your problem...
is the messages variable filled with messages of more than one user at the same time? or do you have separate ones?
for your question, you need to store them simply forr context. it is a language model and it can only react to what you give to it. it cannot remember messages by itself... therefore if you always delete them, or even worse, mix messages of different users it will lose context and won't perform well

[queen egret]

Is doesn't matter. The same messages give worse results than on the website, that's the problem.

[somber viper]

well, one of us is extremely misunderstanding the other one^^
either you don't get that you are not providing the same amount of (good) context, or i really do not get your problem, as you seem to not communicate much of it... be more precise please...
do you always only send one message? without any messages that were sent/exchanged with the assistant before? then i would get the problem and would think you are just using different model settings than the chat-gpt version (did you already fiddle with temperature and top-p? this may change the experience for single messages a lot^^

[queen egret]

Well, if you send a single message to the web version, it answers really good, provides a lot of information, etc. But when I make requests to API, it's less informative

[somber viper]

okay, now i got it^^
well, then i really think it may be about your settings that you pass to the request^^ may you be so kind and paste the part of your code that defines the request parameters?

and also the system prompt you are using please

or is it really only the two lines you sent above?

[somber viper]

i think altering temperature and setting up a good system prompting will fix it for you @queen egret

[queen egret]

Yeah I was thinking about that too, but I can't find what settings are in the web version that make it so good

[somber viper]

well, the two that are important for it are temperature and top-p
what exactly they are doing can be read here:https://platform.openai.com/docs/api-reference/chat/create

but again, i think it has to do way more with your system prompt, as that is where the real magic happens in my opinion. would be nice if you could show me your system-prompt you are using, maybe we can work on it.

[queen egret]

I am not using any, as mine didn't help at all

[somber viper]

Ay, just saw you are using 3.5...unfortunately 3.5 does not care that much about systemprompts, there you are right... I thought you were using 4 already. Maybe @limber wolf does know the official settings? Or at least knows who could know them?
Sorry, i am not working with 3.5 as it was too clunky for me and worthless as i got access to 4😅

[queen egret]

Ah... ok. Thank you anyways

[somber viper]

No problem:) should you get access to 4 i will be there to help😁😊

[queen egret]

Can I just DM him or what

[somber viper]

Nah, pinged him, i think someone will come now😊

[white token]

The openai cook book is fantastic, it’s got everything you need and gpt4 will tell you everything you need to know to set it up

[cloud adder]

im new fish here, do you mean 'overview/documentation'？