Quick question I am using the chat

[unkempt heath]

e.g. https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb here:

    {
        "role": "system",
        "content": "You are a helpful, pattern-following assistant that translates corporate jargon into plain English.",
    },
    {
        "role": "system",
        "name": "example_user",
        "content": "New synergies will help drive top-line growth.",
    },
    {
        "role": "system",
        "name": "example_assistant",
        "content": "Things working well together will increase revenue.",
    },
    {
        "role": "system",
        "name": "example_user",
        "content": "Let's circle back when we have more bandwidth to touch base on opportunities for increased leverage.",
    },
    {
        "role": "system",
        "name": "example_assistant",
        "content": "Let's talk later when we're less busy about how to do better.",
    },
    {
        "role": "user",
        "content": "This late pivot means we don't have time to boil the ocean for the client deliverable.",
    },
]

[tranquil idol]

Look at the documentation on how to count tokens (the code part). It shows you how the name is formatted when sent to the AI

From what I read it will replace the "user/assistant" label

[unkempt heath]

I see, thats great

I can't believe I missed it, 😮‍💨

[tranquil idol]

You can use the name property with other roles as well

I assume it doesn't really matter what the role is if it does override the default label

[unkempt heath]

thats exactly what I wanted 👍

[tranquil idol]

The only issue is that every reply is primed with <|start|>assistant<|message|> so the AI can struggle to stay in character

would be nice if you could override it to be primed with a different name instead

[unkempt heath]

hmm, quite, any guess to where assistant takes it's name from?

the default system prompt perhaps?

[tranquil idol]

What do you mean?

I think it's from the training

[unkempt heath]

Got it

[tranquil idol]

It's probably trained with user/assistant pairs

[unkempt heath]

Makes sense

[tranquil idol]

I've found some decent ways to make it stay in character

You just can't rely on the way OpenAI intended you to use the messages array lol

[unkempt heath]

I have some good algorithms which are working so far, history compression primed with a system prompt

[tranquil idol]

TL;DR I construct a chat log of sorts, ending it with the custom name/label, all in one system message, and that prompts the AI to continue generating the character's reply

95% of the time it avoids the assistant training from kicking in

[unkempt heath]

🙂 that is similar to this test I have

[tranquil idol]

yeah basically that

works nicely haha

[unkempt heath]

I was very happy that I can use the system message for this

Thanks for the spitball @tranquil idol