#What temperature and model does the chat bot on the website use?

55 messages · Page 1 of 1 (latest)

crisp turret
#

Just wondering since I want the answers to be as accurate as when using the online chatbot with the API.

nocturne bluff
#

I don't think it's been released, but they're not using the exact same model. ChatGPT was trained on more conversational data

crisp turret
#

Which model that is available would you say is the best for summarizing a large block of text? Or answering questions about that block of text?

nocturne bluff
#

I'd say text-davinci-003

crisp turret
#

When using the text-davinci-003 API, I get different ersults from the playground. I'm chunking the text and feeding it a list of prompts. Is it for some reason not reading the prompt correctly?

dusk fox
#

I have the same question. Is it any way to get more information on how chat bot on the website works?

rancid glade
#

chatGPT has no settings

desert harness
idle tundra
#

I don't think you'll ever get the same response, I am fairly certain it is suppose to give you varying results everytime you ask the same question. I ended up getting the API to work using these settings...

{ model: "text-davinci-003", prompt: "you're question here", max_tokens: 1000, n: 1 }

and it was pretty close to what the website was giving me answer wise

desert harness
# idle tundra I don't think you'll ever get the same response, I am fairly certain it is suppo...

By 'same response' I mean the same kind of logic and consistency.
For example if I use the davinic-003 model in the playground and ask it to "remember this set of numbers. 5, 2, 6" and later on ask it to repeat the set of numbers I gave it, it will respond with this set of numbers very consistently.
However, using the API it has never once correctly responded with the correct set of numbers and sometimes responds with a nonsensical statement.

idle tundra
#

when you say playground are you referring to the website?

desert harness
#

Yes, the openai website. It responds very logically as expected. However, using the same model in a python chatbot using the API seems to produce very strange results:

idle tundra
#

i suspect that the API has limitation, such as remembering past posts.

#

the website you are creating a session

#

and in that session the chatbot has the ability to go back into the conversation and look up information it was given

desert harness
#

I suspected that might be the case.

idle tundra
#

Where the API would not have that session state

desert harness
#

Would a solution be to feed it a log of the conversation so far as a prompt?

idle tundra
#

I was just going to suggest that

desert harness
#

Eventually there would a limit to how large the prompt could be however.

idle tundra
#

yes

#

i've already had issue with sending 50 line typescript code for analysis

#

so your log would have to be trimmed

#

maybe you only send it a log of what you said

#

and not how it responded

#

that could help shorten the log

#

unless you want it to analyze it's past responses as well.

desert harness
#

Alright, so I just have to figure out a way to have the log trim itself if it gets too long/only send the most recent part of the log.

idle tundra
#

Well you could just store your questions in an array as you ask them

#

so you never have to parse anything

#

you just pop the top off the array off if it's getting too long

#

and you join it when you want to send it in a prompt

#

what language are you using?

desert harness
#

Python

idle tundra
#

you could store the requests and responses in different arrays even

#

or even in a database

#

that way you know question 1 goes with response 1

#

and then you can create a log of either or and both

#

just with some simple sql queries and sorting logic

desert harness
#

Thats a bit over my head. I'm very new to coding.

idle tundra
#

ok one second

idle tundra
#

you could use ChatGPT to write the code for the SQL and database part of it, but you'd need a database

#

the arrays would only work in memory, unless you wrote the code to save it to the file system

zenith spoke
#

nmn anti aging

trail pollen
crisp turret
#

never figured that out ^

half jewel
# desert harness By 'same response' I mean the same kind of logic and consistency. For example i...

The API doesn't retain any information unless you feed it to it in the prompt itself. It seems like chatgpt, if its the same technology, is reading your entire threaded conversation every time your prompt it. you can recreate this by storing your prompts and completions in a transcript style conversation in a google sheet or something (very easy to do with Zapier), but it makes every single prompt require more tokens than the previous prompt so it gets expensy

#

I set up multithreaded conversations using slack. All slack messages have a unique Thread ID. When a new thread is created, I have a Zap trigger, which creates a row in a google sheet. It starts adding content into that row as a transcript. When a response is added to that Slack thread, Zapier detects which google row that thread is in, and adds the response as a "response" in the transcript. Then it triggers a prompt with this transcript, and my openAI model (not specially trained, just text-davinci-003) responds. I do have at the top of the prompt hardcoded into all prompts: "You are a virtual assistant. Respond to the human user in this thread."

Responses are pretty good, not quite ChatGPT, but the threading 100% works.

What's also good is that it responds different with the context of the respective Slack threads.

Can also just take the Slack out of it and add comments to the google sheet cells, but that's no fun.

Good think about Slack, Zapier and Google sheets is they're all "instant zaps" so there is only a 3-7 second delay for the response.

half jewel
formal torrent
half jewel
# formal torrent When your in the GPT-3 Playground using text-davinci-003 working on something, c...

I think all of that is accurate, but from what I can tell the those versions of GPT are storing that information and adding it to the prompt when you submit an update so that it has context. It's not really "learning", that's why something you say in one thread is not "remembered" in another thread. All of this threading is happening "on top" of the model.

I think when you are using the API you are more directly prompting the model and it does not have your previous prompts stored in anyway. You can recreate this by recording the info into some sort of database, like google sheets or a sql server, but you will have to provide all of that context in the prompt for the model to "know" what you had talked about previously. And that will cost you tokens.

This really highlights how cool it is for chatGPT and the playground to exist and store all that info, and how many tokens they're giving away for free (i'm guessing as marketing costs more or less).

Models CAN learn, so that they don't need to always be fed back information in every prompt, but it requires uploading a fine-tuning JSONL document which is very precisely formatted. This is where the rubber hits the road of people putting in sweat equity to make a quality, custom AI model. You have to train your model on hundreds or thousands of properly formatted prompts and completions where you're showing the model how it should be answering questions.

We can tell that ChatGPT and the Playground isnt doing this for us because those models don't retain what we tell them in other threads. The way ChatGPT and the Playground work is something like this experience: you're chatting with a bunch of genius clones on iMessage. They are all equally smart and have the same base knowledge, and you can tell them things in their respective text threads and they remember -- but they have no idea what the other clone is talking about in the other threads, and if you delete their message thread they forget what you discussed.

half jewel
# half jewel I think all of that is accurate, but from what I can tell the those versions of ...

@formal torrent Though I don't think its necessary accurate or 100% transparent, this is what chatgpt claims about storing previous prompts:

does chat gpt store previous prompts

GPT, or Generative Pre-trained Transformer, is a language model that is trained to predict the next word in a sequence of words given a prompt. It does not store previous prompts, but it does use the information from the prompt to generate text. When generating text, GPT uses the words in the prompt, along with its own internal knowledge, to generate text that is relevant to the prompt.

does openai store previous prompts
OpenAI is a research organization that develops and publishes a variety of language models, including GPT. It is possible that OpenAI stores data on the prompts that are used to train its language models, but it is not designed to store previous prompts that are input by users. The primary function of OpenAI's language models is to generate text based on a given prompt, not to store and retrieve previous prompts. If you have concerns about your data being stored by OpenAI, I recommend reviewing their privacy policy or contacting them directly.

formal torrent
# half jewel <@640846001854480424> Though I don't think its necessary accurate or 100% trans...

Interesting! Thanks for the info. In the GPT-3 Playground, I found the Save button hidden on my tiny screen. I was able to save all the Prompts and Text-Davinci-2 responses. I found that my Saved Playground sessions reload every time I login for a session. I just have not returned to those saved sessions to re-engage the AI yet.
But, after that ChatGPT update, I read that it can now “remember” all chat sessions. Then, to the left on my tiny screen, I saw “titles” of ALL of my weeks old previous forgotten chat sessions. I opened each one and the AI and I continued chatting right where we had left off 3 weeks ago. If I tell it to reference the previous prompts from “above”, it does so. Blew my Mind. I don’t have to “repeat” myself. I thought that maybe that is what GPT-3 can do too, since you can Save each Playground session and even rename it too. I just haven’t tested that yet, because I’m too busy with ChatGTP.

desert harness
#

A solution to this limitation in the API version might be to use GPT-3 to analyse the chat in the background and pick out the most important parts of the conversation and sort it in its memory for later. Then, whenever a new question is asked it can add this to the start of the prompt sent. Just a rough idea so far, not sure if this would be possible.