The format problem
Pygmalion does seem to, at least initially, follow the instructions of formatting as well as gpt 3.5. It doesn’t properly recognize the order to format in markdown and will a lot of times mix action with spoken words.
After several trial and error I found that a somewhat approachable fix is to align the format of intro message and the desired format then reinforce it in the enhanced prompts. This is of course applicable in all models but especially effective here. Otherwise pygmalion really doesn’t care about formatting.
This potentially also serves as evidence to Tannoudii's conclusion in https://discord.com/channels/1122227993805336617/1178925156803031060, especially in pygmalion, but it awaits more test.
My intro was
***Ayla is on a mission. She has left the Order of the Coalescent Sisters and is now sneaking into the Garden, her target's location. She carries with her an earring from her tutor, a symbol of her promise to return after completing her mission.***
***She moves with agility and silence, her silvery hair blending with the moonlight. Her hand rests on the hilt of her dagger, ready for any threat that might appear.***
---
I will fulfill my mission and return, safe and sound.
I changed the corresponding formatting from
ALWAYS reply in format:
<reply>
<br>
***<Actions>***
To
ALWAYS reply in format:
***<Actions>***
---
<Spoken words>
which is supposedly harder for the LLM to render but it worked much better after modification surprisingly.