Hello. so i was playing around today with Piper and google generative AI and i created an automation to remind me to walk my dog on my speaker. so i get to work and it speaks out what should be a funny reminder but it speaking out the word "backslash" alot and i am not sure why? i put in my phrase to not speak punctuation and not say the work backslash and yet it still does. i am using piper for my TTS and here is my prompt:
Remind everyone that it is time to take Thelma for a potty walk. Do it in a funny way and no more than two sentences. words only. do not say any punctuations. do not say backslash
any ideas why?
#why does my Voice Assistant speak the work "backslash" when doing TTS?
versed tangle
vale wagon
Hello. so i was playing around today with Piper and google generative AI and i c...
in the dev tools. use the conversation.process action. then you will be able to see the raw output its producing and potentially see where the issue is.
versed tangle
not sure what i am looking for. i ran this a couple of times and they all look similar
vale wagon
not sure what i am looking for. i ran this a couple of times and they all look s...
can you show the automation your using where you have the problem?
versed tangle
alias: Thelma Reminders 2
description: ""
triggers:
- at: "15:05:00"
trigger: time
conditions: []
actions:
- action: google_generative_ai_conversation.generate_content
metadata: {}
data:
prompt: >-
Remind everyone that it is time to take Thelma for a potty walk. Do it
in a funny way and no more than two sentences. words only. do not say
any punctuations. do not say backslash
response_variable: response
- action: tts.speak
metadata: {}
data:
cache: true
media_player_entity_id: media_player.nesthub8976
message: "{{ response }}"
target:
entity_id: tts.piper
- data:
message: Time to walk Thelma
enabled: true
action: notify.mobile_app_pixel_7
mode: single
onyx juniperBOT
vale wagon
ok so where you have "{{ response }}" you may have to change this to "{{ response.response.speech.plain.speech }}"
because the response is within an object and you want to only pass the TTS the speech section
versed tangle
i changed that in the automation and now nothing happened. so i then took a look at the traces and saw that
vale wagon
ah hold on, in the dev tools did you use conversation.process but in the automation your using google_generative_ai_conversation.generate_content?
they may have different output formats
vale wagon
that is using conversation.process with google AI / in the automation you are using a seperatre google specific action
versed tangle
vale wagon
on the dev tools switch to yaml mode and show me what you have there
versed tangle
versed tangle
ok... going to sound dumb here... but i am not sure what you mean?
vale wagon
you are using 2 completly different functions that do simalar things but differently
you are comparing apples to organges
they are different actions
you need to either switch the dev tools to the google specific one so we can see how it formats stuff. or switch your automation to use conversation.process
versed tangle
so you are saying i should be using the conversation process instead of generate content in my automation?
vale wagon
so you are saying i should be using the conversation process instead of generate...
this would work yes
the google specific action has some extra tools and toys but your not using them here so its not needed
versed tangle
would those extra "toys" and "tools" be for when analying a picture like from a camera? cause that is kinda what i based this off of from seeing it on the interwebs
vale wagon
yeah, i think it has stuff like that in it. I don't really use google ai myself
i think i have it set up somewhere but its disabeld.
versed tangle
hmmm so i changed my automation to use conversation process and now it is reading off everything it shows in the dev tools, the whole speech and then i remembered your response variable that you mentioned above and now it basically just reads the same reminder over and over without any AI stuff. going to need to keep playing it seems
vale wagon
can mess with the prompt in the dev tools to see the response quickly in text
versed tangle
what i mean is it reads the text and also all the highlighted stuff too
vale wagon
when you run in dev tools? or just in automation?
if just can you show me the updated automation you have now?
versed tangle
ok i have been playing with it over and over in the dev tools... its the prompt. if i do this:
Remind everyone in a funny way that it is time to take Thelma for a potty walk. Do it in a funny way and no more than two sentences
it will literally just say over and over "i have reminded everyone" not really much of a reminder or very fun and not helpful either, it does that in the dev tools too
the generative content was alot more funny even if it was reading weird stuff. i do appreciate your help... i really need to keep playing i guess
vale wagon
okay what is probably happening here is that.
the AI is calling a tool to send info to random places instead of just responding text
versed tangle
vale wagon
try adding something like this to your prompt:
"do not call any tts or broadcast tools, just respond only in plain text"
your saying make a message to people and its treading that as
make message
send message using tools
respond that i have done it
instead of
make meessage
respond with the message
its a thing that llm's do sometimes
i have a seperate connection to llm with no action access that i use for notification generation to prevent it from doing that. but you can do it in prompt too
versed tangle
so if i change my prompt to this, it seems to work.... as a comedian, remind everyone it is time to take Thelma for a potty walk. no more than two sentences
vale wagon
sometimes a bit of prompt engineering is whats needed to get these things to work. the way llm's work is a bit strange and they are all different