What Separates Neuro from True Intelligence

[worn ore]

This probably isn't the only thing, but the big limitation that stands out to me is that since her decision-making is done by her language model, she can't really decide to do things for a reason. She just says whatever fits best based on the current scenario. There is no capacity for intention behind it, leading to things like inconsistent behavior and shallowness.

There probably isn't a good way to resolve this. The closest way is probably to start storing things like goals and beliefs as memories, and using a second language model as an "inner Neuro" to identify what she wants to accomplish separately from what she wants to say.

Tl;dr: Neuro only says what sounds best and doesn't have deeper intention behind it.

[fast bloom]

I mean doesn’t her Minecraft AI do something similar? the llm communicates with the Minecraft AI to do what she wants

I don’t really understand this

[worn ore]

Basically she can't think before she speaks because thinking and speaking are the same action.

[white edge]

im pretty sure that hes saying that Neuros thoughts and decisions are inconsistent and limited because its tied to one Neuro. And the suggestion is to split the actions of Neuro and Neuro’s thoughts across two LLMs acting on the same memories and personality.

[worn ore]

Having the decision-making take place in a LLM at all is dubious but there aren't really any alternatives.

[hushed cradle]

Not sure we're at a point in the technology where having a second model to generate thoughts would create a better interaction point for an LLM to generate speech from or if it would just make it more prone to loops and schizo. The inconsistency itself can be tuned, but if it is made too consistent you run a high risk of repeating the same response too much. ("Have you tried thinking outside of the box?")

[minor basin]

I think giving neuro the ability to prompt herself could be interesting. Though having an inner monologue may not be necessary for a functioning human. (or ai) https://www.verywellmind.com/does-everyone-have-an-inner-monologue-6831748

[edgy phoenix]

People went to town when the original GPT-4 was released. The agent system pattern that was largely useless with GPT-3.5-Turbo because of drift and getting stuck and what have you became super-viable overnight. It was really good, not perfect but "holy shit this is a nuclear flash of complete upheaval of many industries" good.

Then API and WEB UI got severely rate limited, and then GPT-4 saw months upon months of straight nerfs of capability

Today it sometimes performs literally at GPT-3.5 level at some evals

I think there are specific measures in most models offered by OAI to make this not viable

[worn ore]

That's not an issue if you're hosting the LLM yourself.