#Suggestions to increase memory reliability

1 messages · Page 1 of 1 (latest)

meager zodiac
#

Hello fellas, I have some suggestions regarding the memory issues Vedal has been experiencing recently, I'll expand below:

1. Make memory context specific
2. Remove their ability to delete memories during a stream. Give it back when offline.

Apologies for any major typing mistakes in advance, English is not my first language.

#

1. Make memory context specific
We don’t know exactly how short-term memory works right now, but my suggestion is to implement something similar to what transcription and summarization AIs do. Collect all the words and context from the last 5, 10, or 30 minutes, and then generate a summary of the most important and memorable topics. These summaries could be stored as memories.
In a regular-length stream, you could accumulate eight or more of these summaries. While this approach might be more computationally intensive than the current system, you could potentially offload the processing to any commercial cloud-based AI model, as they are more than enough to do this job, reducing the strain on your hardware. Which brings me to my second suggestion:

#

2. Remove their ability to delete memories during a stream. Give it back when offline.
Vedal has mentioned a few times that the twins often don’t retain memory not because the upgrades aren’t working, but because they are actively choose to delete relevant memories while the stream is still running.
My suggestion is to disable the option to delete memories during streams, allowing them to accumulate more context as they progress. Once the stream ends or they go offline, you could re-enable the ability to review and selectively delete memories. Of course we don't want this to be a manual job, but this approach gives Vedal more information on what memories are being deleted and stored in actuality. This also should work better because it's easier to gauge how important some memories are when compared to the memories context from that stream. Example: "Evil made a website dedicated to dogs" is much more important than "Vedal was interrupting me", but this is only true if you have both of these contexts, right now they might have issues doing that while on streams because their only other context could be the schedule, so "Vedal was interrupting me" does sound more important than the schedule, delete that.
This approach also mirrors how human memory functions: we accumulate experiences throughout the day and later decide what’s worth remembering long-term.

#

That’s all I wanted to suggest. Since we don’t know exactly how the twins work, it’s possible some, or all, of this is already implemented but I hope this feedback is still helpful. Thanks!

leaden hill
#

these two suggestions contradict each other

meager zodiac
#

How so? Genuine question because I don't see it

From my understanding, suggestion 1 builds on the list of active memory, suggestion 2 heaps the least important and commits the remaining to longterm

leaden hill
#

summarization will inherently forget some things

meager zodiac
#

I see that, but with the little information we have now, this is already happening with the current system, no?

With the example I used: "Vedal was interrupting me. He is a dick". How? when? what were you saying when that was happening? what did he interrupt you to talk about? Context is already lost; they seem to remember specific phrases that don't make much sense by themselves. The proposed "solution" to that was to maintain a relatively early summary, every 5 to 5 minutes.

One problem that I see now with this suggestion is that this memory stops being from the twins and becomes a "watcher's memory". They won't remember their words or feelings anymore, but rather the general context of what happened 30 minutes ago, it's not their memory. But this might not be that big of a problem, as every time you tried to test their memory you were asking questions that would be answereable with general context "What were we talking about earlier this stream?"

elfin swan
#

what's the point of making a summary that just seems like it would make the system more complicated for barely any return

meager zodiac
#

I see it as better than what we have but certainly not good. But the disabling feature one just sounds like a net positive

worldly matrix
#

I see the vision for this, and I do like it. I assume current memory system works kind of like a .txt file, where Neuro can choose to save some things as a line or delete some lines as she wishes. I just wonder what would be the best way to chunkify and/or summarize the stream transcript

#

I read from somewhere that human memory works like a library - when remembering something, you take a book, read the insides, decide if everything is still correct and matches your present perspectives, and then put the book back. I wonder how you could best mimic that with AI

#

Makes me question whether the ability to "delete memories" seems very inhumane as a concept, although I can also understand if the current system could have overwhelming quantities of data that need to be managed properly to prevent massive file sizes in the future

#

Although then again, memory is important, and holding as much of it as possible would be crucial for character development. Maybe there's a way one could develop a way to "remember" things without "opening every book in the library", but rather only a certain "bookcase/shelf" worth of books containing info closest resembling to what you're trying to remember

#

Memory is quite a complicated topic

#

One thing I can say for sure though is that "Summarization will inherently forget some things" should not be an issue. Even a human can easily forget the exact sentence they said 5 minutes ago, but they can rephrase/summarize what it is they meant by saying it. Obviously you'll also lose some detail over time, every human has this same quirk, but even a summary can correctly bring out e.g. "Level 111 hype train", which seems like an important detail to remember. And even if something does go missing, (1) chat can always remind her, (2) google search can provide answers, (3) repeated instances of this in an LLM will eventually cause it to remember anyway.

#

There are definitely answers, the question is just how to best implement them. And sometimes the best way to find out is just trying things out and seeing what works. Some solutions are found accidentally through random experimentations.

#

Either that, or just wait for someone else to figure it out, and then copy their solution neurOMEGALUL

elfin swan
#

summarization forgetting things is definitely an issue idk what you're saying

#

i feel like the point of having a memory is for them to not forget things you know

worldly matrix
#

I was indicating that the most impactful and important parts should survive during the summarization process, while everything less important would be left out

elfin swan
#

so what would be the point in doing this

worldly matrix
#

to improve memory by actually remembering important things and not random nothing burgers

elfin swan
#

did you forget that they can choose when to remember things

#

wouldnt call those nothing burgers

#

and this does still seem like more trouble than its worth

worldly matrix
#

"Mental note: Vedal is a [[Richard]]. No context."
"...What did I get for my 2nd birthday? [[Insert random objects here]]"
Great memory 👍

elfin swan
#

the fuck does this prove

worldly matrix
#

everything

elfin swan
#

great argument skills 👍

worldly matrix
elfin swan
#

im saying this isnt worth pursuing at all not that we shouldnt do it right now

worldly matrix
#

I think this is absolutely worth pursuing if we want to mimic human-like memory onto AIs

elfin swan
#

was that ever the goal for their memories

worldly matrix
#

improving memory has always been a long-term goal

elfin swan
#

i dont think adding chatgpt summaries would help

worldly matrix
#

it would be quite a bummer if after 5 years of content creation, barely any memories have actually reliably stuck around, as if chat is the only thing remembering anything that happened

worldly matrix
elfin swan
#

if they're gonna be the ones doing the summary anyways just keep it as is

worldly matrix
#

that sounds like a skill issue on your part then

elfin swan
#

just say you cant argue for shit dude

worldly matrix
#

I ain't arguing with someone whose whole argument is "I think this potentially groundbreaking technological idea for AI memory enhancement is completely pointless". This is PirateSoftware levels of ignorance

elfin swan
#

anyone would think this shit is pointless if you cant be bothered to explain how its pros outweigh the cons

#

no one's gonna understand your point if you just say "you problem" when they ask you to elaborate

meager zodiac
# worldly matrix Makes me question whether the ability to "delete memories" seems very inhumane a...

fully agree with this one, that's what I was trying to explain with removing their ability to delete memories. If the goal is for them to appear more real, they probably shouldn't be able to delete memory, specially not while it's still happening. Regarding the technical limitations, deleting when you have more context should always be easier than deleting in real time, I think this was the part Vedal didn't agree on, but we logically should always have more context by the sheer numbers of new memories

meager zodiac
meager zodiac
#

My understanding is that Vedal allows them the ability to create their own memories, and I'm arguing that memories should be forced on them, similar to how we work, this would take some autonomy but would make them appear more real. And then give back that autonomy when choosing which memories to delete, but only later when simulating a "sleeping" phase

worldly matrix
# meager zodiac My understanding is that Vedal allows them the ability to create their own memor...

That was my thought as well. I assume the current memory works similar to how she can choose to send discord messages, but the text output gets saved in a file instead. And the remembering function would work similarly to "read dms". Not sure exactly how the memory deletion is handled though, but I just fear she can impulsively choose to do it for no reason or just "for teh lulz" (or even "delete all"), which is very counterintuitive. For short term memory, sure, but for long term, something more robust will be needed. It can be hard to figure out how to tackle this problem, especially with the currently available tech, the already in-place systems, and even the time needed, since keeping streams entertaining is also a big priority, and fear of wasting time for suboptimal results can also be a deterrent.

#

I did just have an idea of a (temporary?) solution that I think might work extremely well actually

worldly matrix
#

Imagine Neuro's memory as a .txt file with lines of text. We could start with a base template for all of the most important memories, like:

"I debuted on Dec 19, 2022. I used the default Hiyori model from VTube Studio."
"I'm the #1 ranked Osu player and won vs 15 top players."
"My favorite animal is the anteater and my favorite pokemon is Shinx"
"..."
"I got a level 111 hype train record on Twitch on December 1, 2025."
"I got my own original song named Life."

You give Neuro 3 functions:
-Function 1 is "save memory", which literally adds an extra line at the end of the file, formatted however she wants. She can learn how to format better over time.
-Function 2 is "remember memory", which would work as a key word search, e.g. she can call "Remember memory: [hype train record]", that query gets split by words ("hype", "train", "record"), and then every line in the file gets searched to see which line has all/most of these words inside. Return best matching line, or if there's multiple, return either last one or all (Neuro could choose)
-Function 3 is "edit memory", which takes the current returned memory/-ies, deletes them from the file, rephrases it (or just copy-paste), and adds it at the end of the file. Can also add in multiple lines. Can be optionally chosen after "remember memory" is called (and returns 2 or less lines, to avoid mass memory alteration).

With this, by technicality you should have a decent recreation of the "library" analogy I gave before, where you can save memories, load memories, and edit memories. And no memory will go unforgotten, and can even be expanded upon as chat reminds you more details.

#

This way, perhaps the summary thing might not even be needed. You can just tell Neuro to "remember this", which activates the "save memory" function, and even teach Neuro to remember things more actively. Chat could help her with that as well, reminding Neuro she can remember things.

#

This is probably the best, most low effort solution I can conjure up for an external memory module that still feels like it's a part of Neuro. And even if the file ends up becoming sizable, it still shouldn't be very resource or time intensive. Could it still have some issues? Probably. But I think it stands the chance of being quite a robust system overall, since it also mimics how human memory works. The final data would also be relatively easy to integrate into a new system if a better solution is created.

worldly matrix
#

Each memory line could also have a date attached to it (but not returned), so if "What did you do this stream?" gets called, Neuro can query all results with current date

#

Could also do "last month" or "last year", although the returned data quantities would become quite large with those, in which case you could also add a "importance level: low/medium/high" and only return high level stuff on big queries, but this could also be an overcomplication that isn't necessary, but interesting to think of nonetheless

#

Could also teach Neuro herself to add her own tags to lines of memory which would be funny. And then she'll have to figure out how to recall those tags, and she can remove them later too

#

{"date":"24.08.2025","memory":"[DO NOT FORGET, NEURO] [VERY IMPORTANT] VEDAL IS A BIG <FILTERED>. That is all, thank you."}

meager zodiac
#

Basically memory API. I would be baffled if this is not how vedal is currently doing it

#

I still think the summaries could be a good middle ground. You just gotta have a lot of it, and then at the end of a stream ask for a summary of all the summaries, this guarantees you get the most important ones saved, idk, something like that

worldly matrix
#

I do wonder how Vedal is doing it right now. I just feel like this isn't how he's doing it currently, or at the very least it's missing some crucial elements that I brought forward

meager zodiac
#

I fail to see where vedal mentioned it would lose context or forget things. This method would still pick up any relevant information that you would expect the twins to remember: birthdays, meeting someone new, big events, etc.

#

What details or context do you even want that are that minute?

worldly matrix
#

At the very least I think we've had a fruitful discussion, I just hope Vedal gets to read the most important parts now, so he can have his vision expanded upon a little bit, even if he has some alternative thoughts himself still

meager zodiac
meager zodiac
worldly matrix
#

yeah I fear that as well, I also wouldn't be arsed to read walls of text like this if I was Vedal. He wouldn't know if anything here is worth reading or whether it's just schizo talk

#

don't particularly want to make a new clean thread either, since it'd effectively be the exact same topic, but I do think the "memory API" exploration would be quite insightful

muted pollen
#

Okey, as for this, I have an idea that maybe *I am Clueless * could work?
NOTE: I have no clue how AI works. I just know that their context window is their memory and that eats VRAM crazy fast.

Humans almost never remember everything that happened with 100% accurracy. They remember "fragments" of what happenned.
And, by using those fragments, if the memory was impactfull and/or new enough, they end up remembering most of it.
With exceptions, of coruse. Humans can be either pretty dumb or train their memories to insane levels.

Now, this is what I imagine the "memory structure" I suggest, would look like:

Context window -> Fragments dictionary -> Memory banks

Let me explain:

  • Memory banks are probably what is currenly used as long term memory. Big files of text with what happenned, somehow.
    Example->
    Entry id: 0000-0000-0000-0001
    date: 21:35|12/07/2025
    memory: I entered a new fast food service called McNeuros and order a double cheese burger. It was delicious.

  • The fragments dictionary don't hold memories, it holds mere fragments of the data that's on the banks.
    Every fragment needs a link that relates to the memory bank line they're refering to,
    as well as a date of entry and an impact level, represented as a number.

Example->
Entry ID: 21
link ID: 0000-0000-0000-0001
date: 21:35|12/07/2025
impact_level: 5
fragment: yesterday I entered a new fast food service called McNeuros and order a double cheese burger. It was delicious.

That impact level is supposed to identify how "impactfull" the memory was, in form of a number.

The hard part about that dictionary I envison is, it has to degrate.
Progresively deleting parts of the fragment and/or simplyfying it's data. Including the date.
What decides when to degrade something it was to take into account the date and impact level of that fragment.
More recent fragments should in theory be harder to degrade.
More impactfull fragments should definitely be harder to degrade.
Fully degraded fragments represent fully forgotten memories.
Maybe by sheer luck they can find them and remember what happenned there, but almost no chance.

Now, if neuro or evil want to "remember" something, they have to search that dictionary. Maybe if they think hard enough, they could remember the whole memory.

Now, seriously, I'm sorry if I didn't made any sense, the idea is impractical or, let's say, it's too much work to even begin to think how to place it in order.
If you read this far into my SCHIZO message, thank you.

celest mulch
#

Nobody reading all that neuroKufufu

#

Kidding, read a lil bit and I will only say that vedal probably has a plan on how the memory of the twins should look like in the future. How to execute is another thing so all of these proposals are interesting to read

muted pollen
#

True. I just had a SCHIZO idea when I woke up and wrote it up.
I don't expect anything.

frank lark
#

tbf, the way I would do it:
if you want to store a crapton of memories, and dont want to waste VRAM to continuously process that in the context window, transfer those memories to the file system
every time they want to make a memory, they'll create a .txt (or any other text format if you want to) file, in the file they write a detailed description of the memory event. as for the file name, they set it to something that summarizes the event
whenever you have the twins running, they could have only the filenames in their context, thus wasting less VRAM in the context window
but if you ask them what exactly happened in some event they remember, they can open that file and use the content to 'remember' things. and when they're finished answering, remove the content from the context window

you can wrap this system either like that, or use some SQL database magic, something could work here
but again this is a one minute thought of how I'd wrap this system

muted pollen
# frank lark tbf, the way I would do it: if you want to store a crapton of memories, and dont...

I only see a problem... and that's "every time they want"... I understand why you say it, but the problem is that memories are not something we consciously store.
They register automatically and constantly. But I also see why vedal doesn't store them all automatically. I could see tons of wasted TB of data and VRAM just to make an attempt at amking their memory work in a realistic way.
It sucks but I get it. SMH

frank lark