Does neuro actually know when she’s singing?

[lost ledge]

Like is there an actual singing state for neuro or is she just muted while a sound file is played?

[small kernel]

i cant be 100% certain, but judging by the dodge charger incident and a general understanding on how the singing seems to work. there probably is a singing state but the audio itself is generated in advance

[lost ledge]

Yeah ik the audio is generated in advance, but there’s been instances of neuro talking during song, or even just talking with no audio only captions (happened in money machine), which would imply that she’s just usually just muted while the audio is just routed from elsewhere. But during karaoke she announces the song and even talks about it afterwards sometimes or says thanks for the applause, which implies the existence of a state where neuro is aware that she’s singing. But such a state would make the aforementioned instances of her talking during a song impossible, no?

Therefore, we’ve observed situations where neuro clearly has no idea she is singing, implying a mute. But there’s also times where she knows she just sung, meaning there should be a singing state. I just noticed this while listening and was just wondering about it lol

[little gull]

there is a singing state but the singing is a different AI so she's not aware that she just sang unless there is some communication between the two states but it doesn't seem to be the case. Her sometimes acknowledging afterwards could just be a coincidence (triggered by chat spamming clap emotes)

[heady rune]

I hope Vedal can make Neuro notice when she's singing. She doesn't need to know the melody; only the length, when she began and ended, and song information such as lyric, title, song writer, artist, and release date.

[jaunty haven]

That could be displayed as subtitles that appear when neuro usually talks

[heady rune]

I didn't mean for that to be displayed on the screen. When Neuro is singing, she should "remember" and "know" what she is doing. Right now she doesn't know when she is singing. Neuro doesn't even know that she can sing.

[zinc moat]

Its a chat GPT, so nobody can guarantee she even remember the last sentence of her own speech.

I think, its working something like this:

1. We get some solid info (prewritten, like: Neuro is you)
2. Some past conversation logs (3-40 latest turns) and take something about... 100-1000 symbols from it randomly (grouped by words or sentences).
3. Mix it together and trying to guess, what people will say in this situations (based on 600k+ real examples, used when she was trained), they say it.

So lets say, in the situation, when you have "Hi", most examples will contain greetings variants, so she will say one of them.

Chat GPT is not really have a memory, it mostly just trying to guess "your next words" (next dialog), based on examples.
You can try and put inside hard-written song name (it will bring the lyrics too, if trained), tho the fact she committed singing - you better not. Nobody really telling the vtubers "you singed", chats just spam emotes, like with Neuro, and they react to it. Try to force it, and you get something like... "What? No I'm not!".
I think, mb you can try put the hard-written
*sings "[song name]"* ??? So she kinda will be thinking like committed roleplay and will manage to continue it then.

[next badge]

@zinc moat has leveled up! (1 ➜ 2)

[heady rune]

Neuro is powered by GPT, not ChatGPT. Similarly, BingAI runs on GPT rather than chatGPT.

ChatGPT is an app; GPT is the brain behind that app.
iirc, Vedal said Neuro has both long-term and short-term memory. There are some GPT-memory plugins already, so it's not new.

[zinc moat]

I see. So it just GPT. But sadly, it appears that I just misspoke regarding the terminology.
For me, she didn't differ in appearance from AIDungeon GPT or any prepackaged 4chan analogs. She has a history of misinterpreting and switching genders within two turns to the same person, inadequately switching action points (i.e., switching from "you doing [x]" to "me doing [the same x]" in the middle of a conversation), and answering questions from the perspective of the interviewer rather than the interviewee. She will also change anything in her anshwer if the same question is asked twice or more. You can simply "roll" her till you acquire what you need, just as in AID. She cannot tell the same age of hers twice, the date of birth, or does she washed today or not. Within some ten minutes you can be her fiance, friend and enemy, then again fiance. She frequently didn't realize who she was speaking to at the time, she even called her collab partners Vedal, many times. To me, it doesn't appear to be a good memory, it just as any GPTs.
I suppose the hard-coded prompts can be thought of as long-term memory ("WI" in AID). And I suppose short-term memory can be defined as "remembering" some of the most recent conversation points ("context"). However, it is still far from ideal and very unrealistic, just like in ChatGPT or others GPT. I highly doubt she has some storage to "ask" it or something, like people or normal programs do; more likely, she just gets new material to overlearn every time post-stream.

Although she is still the most sophisticated AIchat companion I've seen, I sincerely hope she will occasionally have a true memory of sort. She will experience true human consistency when it happens.

By the way, in the real brain, repetition and current emotional state are linked to long-term memory. In comparison to the weak memory, a sharp, high-energy moment of creation or the repetition of a stable signal allows the strong memory to be created. (Or rather, it solidifies the neural connections and simplify the after-usage). Instincts and conditioned reflexes are also related to this idea because c.reflexes require repeated signals to be generated, whereas instincts are fundamentally long-established repetitive signals (billions of years+).
I.e., the sudden death of a mother will surely be remembered of its child, who saw it. Or, when you hear some name thousands of times, it's easy to remember this name. Also, like with c.reflexes, dissaperence of a stable signal repetition will slowly kill the link (a memory). As with the "power" of emotion, created a memory - the "spending" of its power will slowly (or not) kill the link; so basically, whenever you reliving it in your memory - you lose its power (if you do not "charge" it in the process with more emotions).
Can it be used?

[heady rune]

That sounds like you used Bing Compose for that. It's too long and sounds like an essay.

Neuro is running an earlier version of GPT, possibly older than ChatGPT. Again, everything I said was based on my memory of Vedal, but that could be a false memory because it's usual for humans to have incorrect memories. Memory is unreliable, but it can be overcome by keeping an external record, such as a notepad, diary, camera, voice recorder, etc. So I could "query a look-up" related chatlog to see whether it's true. It's not that deep tbh.

[zinc moat]

I see. That's kinda sad he used the old ones, Impressive results tho. I'm not a professional, but guess the new GPTs is just more trained or something, more computing power behind it, more neurons. And probably the true way of improving it at the end, is not in this, but in the new mathematic formulas. But that is just my opinion.

And about compose, yes. Not that specific one I'm using, but I don't really know this language (only learned it from watching films, books, youtube, speaking to people etc), so for the big texts I'm using correction. My normal text sounds like "what I'm hear", most of the times, and there is no chance I can explicit a scientific or specific info in it - since you don't hear it that often, in the internet or in the conversations.
Il try without, this time, say if it comes out any better this way.

Memory can be solid without external storage, there is some people out there, who can remember a whole book in the details, by reading it just once. And do remember this for a very long time (Anatoliy Vasserman i.e. once remembered the exact wordings of a book he read 20+ years ago, by the request of just page and line numbers). Its an anomaly, mostly seen out of autistic people with savant syndrome, but shows the possibilities.
Just, our brain usually didn't has its dedicated storage for a memory - we store the memory in the same places, where we compute sensory information (long-term memory too). That's basically mean, that you kinda overwrite it (-partially) every time you perceive something or think. Brain can restore info from the very crippled states tho. (Brain is all about being optimal and max. cheap, like every living thing, so its very limited. Nobody force us to use the same limitations in the virtual neural networks. Also, we have our personality and conscience written on extremely solid, dedicated places in the brain, in oppose to memory.)

[heady rune]

The limitation is the resource required to store permanent memory in a neural network. Memory should not be "solid." It will continue to pile up and overwhelm other functions, and it will have "solified" the network, which means it will be unable to store additional memory, a.k.a Anterograde amnesia.

About Photographic Memory, there is no evidence of photographic memory: https://artofmemory.com/blog/eidetic-memory/
The neural network is an interesting but complex topic. I hope you continue to research it. There is a reason why the existing neural network continues to hallucinate, it's an ongoing problem that they're trying to solve.

[zinc moat]

Cool, never though of limitations in that way.
I have some questions about technical aspects tho, if you can answer please do.
Can we create the text-to-speech neural model? Is there a way to incode sound into integer? Can we use TensorFlow to teach a NN through some random movie-series with subtitles, automatically, without by-hand gathered and organized material? Subtitles kinda has build-in timecodes and everything.

[heady rune]

Yes, that is called Artificial General Intelligence (AGI), the next step of AI.

[zinc moat]

No, I mean, how to do it actually.

Some articles mb, or examples. Preferably, on the Python and TF.

[young flume]

I actually built a Vtuberbot, ill correct the premises on memory. You can give the bot long-term memory using a vector database (I use https://www.pinecone.io/), and this works regardless of if you use OpenAi or not, the LangChain framework (https://python.langchain.com/en/latest/index.html) actually lets you implement this easily. You can use vectordb as chat history, such as its easy to insert past conversations that never happened on chat or at least the viewer will not be aware of. So yes you can insert a memory of what she sang, including the lyrics, or even describe how the song felt and it will be incorporated into the LLM responses. You can also insert an entire book or manual into memory and it will have the same effect. I discussed recently how memory affects the LLM response especially if it encounters a question for the first time and how the initial response will be solidified as part of the bot personality https://youtu.be/4ao5Fl_18Aw?t=8770

[jaunty haven]

What would be cool if there was one model that does image recognition, voice recognition, image generation and voice generation (with emotions)

[zinc moat]

It probably will be too big and will take a lot of resources.
Human brain has been formed the same way they do multineurals - one zone doing one thing, like one neural network doing one thing (chat, image recognition). Brain zones was created in the process of evolution, one after another, after new life conditions and demands || because of what, its whole structure is nonsense, in example visual information have to go from the eyes to the almost very opposite end of the skull || . Like Neuro got new modules.
It just... cheaper that way. Both for the nature and humans.
If Vedal can link modules together, it counts the same as one model I think. And is manageable, theoretically.

I dont sure he can tho. Like, if he can link minecraft nn to the chat one. It seems pretty hard task even within the same specified neural networks:
https://www.youtube.com/watch?v=MQ0XWJyreLg
Sorry I only got Russian one (its nn + machine learning), but its kinda obvious what's going on there.

My wild bet - you just need an another module, that is trained specifically to link other modules together.