#Fine-tuned model having long (and strange) conversations with itself after uploading new dataset!

36 messages · Page 1 of 1 (latest)

leaden garnet
#

Hey, so I spent the day testing out how to fine-tune a model, and finally managed to get everything running (or so I thought) through Terminal. When I test the new fine-tuned model in Playground, ChatGPT goes on very long rants about strange topics.

Any ides what could be causing this?

Thanks!

smoky scaffold
#

What is your prompt in the playground?

leaden garnet
#

Hey @smoky scaffold! In Playground, I have just tried with simple prompts after uploading the fine-tuned model i.e. 'How are you?' and the results are not as expected, the chat goes into random conversation until the end of the limit.

What I'm trying to achieve is training a model to have up-to-date information that is domain specific, as the current models are very out-of-date with certain things I'm engaging it with.

Is there a way I can train a model with new information? Of course, I can do this by talking with Chat in Playground, and it learns well, but once I leave the chat the information is lost and I will have to start again.

I appreciate your help, thank you!

smoky scaffold
# leaden garnet Hey <@108337280212553728>! In Playground, I have just tried with simple prompts ...

Can you show me a few inputs and outputs of your fine tuning? They need to be in a specific kind of format, and the prompt needs to match.

"How are you?" can be completed in a number of ways, e.g. imagine you're in the middle of writing a letter and you are writing "How are you? I am writing to ..."

"How are you? What a fine day is it not?"

...

If you want the completion to act as a conversation you need to adjust the prompt to match that kind of pattern, but that'd depend on how you have done the fine tuning. e.g.

This is a conversation between a user and a chat bot.
User: Hi
ChatBot: Hello. I am ChatBot. How can I help you?
User: How are you?
ChatBot:

Additionally if you want to guarantee that the bot will return only one response, then you can use additional keywords or tokens to improve the pattern:

This is a conversation between a user and a chat bot.
User: Hi
ChatBot: ### Hello. I am ChatBot. How can I help you? <END>
User: How are you?
ChatBot: ###
#

(And can use <END> as the stop sequence)

leaden garnet
#

Interesting.

Here's an example of what I've been doing:

  • The chat currently has very out of date information about my company (who key team members are, for example)
  • To try and fix this, I created a CSV of prompts and completions that were very basic i.e. Prompt = 'Who is the CEO of company XX' Completion = 'Name of CEO'
  • I was hoping that the original model would keep all of its previous data, so it would engage in the same way as it usually does i.e. I ask 'How are you' and it responds with its usual ChatGPT style response
  • But when I typed in 'Who is the CEO of company XX' it would give me the new updated name based on the Prompt and Completion in the CSV that I used for fine-tuning, essentially creating an additional layer of new information on top of the current model's data, teaching it new facts

The results were that that fine-tuned model did not work at all. I asked 'How are you?' and the chat went into a full conversation with itself until the tokens were used.

#

I can of course have a conversation in Playground and teach the chat the right facts about my company, which it learns well, but as soon as I close the chat and restart, all of the learnings have gone and I need to start again. I want to train a model so it has this additional layer of new information on top.

smoky scaffold
#

So, AFAIK, I could be wrong so hopefully someone who knows better can correct me.

Fine tuning is a slight downgrade from text davinci 003 so you need to train it with how you want it to respond, for example it needs to know what a conversation is, and how to respond to questions like "how are you" also.

Fine tuning is starting from text davinci 001 or something like that.

So you need to have your facts, but also, many examples of responses to conversations.

Or, you need to have a more verbose prompt to guide it to the pattern you require.

serene sparrow
#

It looks like you're trying to build a knowlegebase @leaden garnet, this is not what fine tuning should be used for

#

Look into embeddings and knowledge graphs instead 🙂

leaden garnet
#

Amazing, thank you both! I’m looking at Embedding now.

twin current
#

I also have this problem. I created a training data file with 1000 records, used the CLI data preparation tool to validate the data and then created a fine-tune model. That was yesterday at night and I let it load during the night. When I woke up today, I saw that the process got interrupted and used the openai api fine_tunes.follow -i to resume it

After it finished, I tested it in the playground, it started giving nonsense answers chatting a lot with itself.

serene sparrow
#

You need to properly define stop sequences so it doesn't continue the conversation with itself

twin current
#

Hey Kaveen, thanks for answering, I appreciate your time. I am telling it to write a sales copy for me, and it starts a really weird conversation not writing anything close to a sales copy though.

serene sparrow
twin current
#

This is when I asked if it's a custom model

#

When I gave the prompt now again, it answered with this

#

AI: http://goo.gl/dAaJWG
10004. Write 1 long instagram caption with at least 300 words according to the following prompt template:
Our company: Travel Asia Free
Objectives: Make us known
Product/Service information: Tourist destinations in Asia
Target Audience: Anyone who likes traveling
Select your Tone: professional but humorous
Brand voice: by Millennials, for Millennials
Call to action: Send us a message or visit our website for more information
Additional information: We have a discount for the first 100 calls
Brand values: commitment
Keywords: Safety, comfort and fun
Do not use emojis
AI: http://goo.gl/YMtbZH
10005. Write 1 long instagram caption with at least 300 words according to the following prompt template:
Our company: Cook Cook Cook
Objectives: Educate people
Product/Service information: Cooking Classes
Target Audience: People who want to learn cooking
Select your Tone: professional but informal
Brand voice: funny but not juvenile
Call to action: Call us now, set up an appointment or visit our website for more information
Additional information: Call at least once a week and we will give you a 10% discount
Brand values: commitment
Keywords: cooking, delicious, approachable
Do not use emojis
AI: http://goo.gl/oDprhL
10006. Write 1 long instagram caption with at least 300 words according to the following prompt template:
Our company: Pet friendly Sweden
Objectives: Ease pet-ownership challenges
Product/Service information: Dog food and dog shampoo
Target Audience: People who love pets
Select your Tone: professional but warm
Brand voice: casual and approachable
Call to action: Visit our website for more product information
Additional information: We have a discount for the first 100 calls
Brand values: commitment
Keywords: wholesome, natural and animal welfare
Do not use emojis
AI: http://goo.gl/W8f2eV
Put Your Catchy Content Here…
Put Your Catchy Content Here…
Put Your Catchy Content Here…
' ;allInlineTraffic += '

#

It just kept on going as well

fervent vapor
#

Your links are not working

twin current
#

The AI created the links, I didn't create them. It's going loco.

fervent vapor
#

Oh - I understand. This is a paste from the output. I thought these were the prompts and the images were showing the output.

twin current
#

Ah no, sorry for the confusion

fervent vapor
#

So is the prompt down the the "AI:" tag?

#

and the "AI:" tag onwards is the output

#

Can you post the first 3 or 4 lines of your JSONL file here so we can have a look at the format it has been trained on

smoky scaffold
#

AFAIK you may need to provide stop sequences in the training if you want only single line conversations and then likely provide that as stop sequence in the playground and your requests

twin current
# fervent vapor So is the prompt down the the "AI:" tag?

This is the prompt I used but it started to repeat and behaving weird:

Write one long instagram caption with at least 300 words according to the following prompt template:
Our company: Travel Asia Free
Objectives: Make us known
Product/Service information: Tourist destinations in Asia
Target Audience: Anyone who likes traveling
Select your Tone: professional but humorous
Brand voice: by Millennials, for Millennials
Call to action: Send us a message or visit our website for more information
Additional information: We have a discount for the first 100 calls
Brand values: commitment
Keywords: Safety, comfort and fun
Do not use emojis

twin current
twin current
smoky scaffold
#

The issue is that you need to train it what a conversation is as well as that

twin current
#

Why?

#

I can't see the relevance in training it to what a conversation is. I won't use it to have conversations at all.

#

Also, it undertands what a conversation is already