The problem with LLMs is that they don't have intention behind what they're doing and saying, but you could use a similar system to things like Minecraft (at least sometimes) where Neuro can choose an intention, desire, or feeling, and then begin talking based on that. Since it would cause latency you'd probably only want to do it sometimes.
#"Act then talk" to make Neuro smarter
coral haven
bitter rover
could you walk through an example? 😵💫
wicked lance
Sounds like suggesting stretching the function calls and generative convo equally to me.
Basically work on motor skills kind of? I've had similar thoughts about AI with function calls.
There's also datasets like 60gb for training this for gaming controllers and video games lol
coral haven
This isn't about games, it's about applying the "act then speak" system from Minecraft, where first she decides to gather wood, then decides what to say so she can talk about wanting to gather wood.
So she can decide that she wants something in a conversation and then says things based on that intention instead of just saying whatever she says.
wicked lance
Oh of course its more like hand eye coordination, I should have been more clear sorry lol
Like being stuck in games I think of it as gym training that kinda data
But I see you're referring more to the logic side, of decision making so, yea that's probably a better idea anyway lol
coral haven
It probably doesn't even need special data, it just needs to ask her the right question about her intentions. It's still piggybacking off of the language model but in a different way.
wicked lance
Right? Make them realize slowly they are actually interacting not just good at choosing tokens. Interesting idea 🙂
bitter rover
I reckon Neuro already has hidden tokens/internal thinking/hidden CoT
wicked lance
Had a model spew its token reel once... weird thing was alot of that wasn't in the tokenizer file. and between the spewing had a weird obsession with the secret garden.
empty spoke
Are we talking about a robot body? Or just adding fake functions and tell them they have real effects?
coral haven
The "action" would be choosing intentions or other kinds of internal thoughts
empty spoke
Oh, like thought, words that maybe only vedal can see (without her knowing), I thought about it before because everything she say it's like improv, or when she says everything that it was clearly supposed to just think, but she can't, or like when she played werewolf and she needed to be reminded to not reveal.