LLMs and Model Corruption

[fading trout]

Quick question:

What happens if an LLM gets corrupted or poisoned due to bad data or some funky processing?

Just saw a clip of Evil bugging out (with Vedal trying to fix her) and that made me wonder what could realistically be done in cases like those. Would regular backups be necessary? Could the model still be salvaged after a full corruption?

Figured that it would be more interesting to ask here rather than to go through the good ol' google rabbit-hole.

[median copper]

you could probably ask this in #programming and someone might pop up to answer

but regardless of this issue vedal should be having backups of the twins anyways

-# how much does it cost

[fading trout]

I see! Ty!

[wanton grotto]

I hope Evil is ok. Watching her basically have an AI equivalent of a stroke was horrible.

Vedal's gotten this far. I'm sure he will solve it.

[worn monolith]

Is Evil gonna be OK?

[silk lodge]

She’s dead chat

[lost ginkgo]

You don't normally have an LLM like a physical object that you constantly make changes to. The way it usually works is that you have "snapshots" of a specific model in its different stages of training (fine-tuning). You pick which snapshot you wish to use whenever you like.
I don't know the details on what exactly Neuro and Evil use as their (main) model but, knowing Vedal, he should have either snapshots or something equivalent when it comes to that.

This Evil bug was interesting, however 🤔 We see the twins going full "schizo" almost every single stream for like a single response and then return back to normal with one time even being where Evil apologized in the next response before moving forward. This time, Vedal was almost in a panic state trying to "fix her" which is Very unusual 😅
He mentioned "She lost blood flow to her brain just give her a minute.". Maybe something broke which made her unable to use her "main" model for a bit there... 🤔 I usually have a decent idea on what could've broken when something goes wrong but this one was super strange to me... 😅

[fading trout]

In the comments of the few clips I saw, people kept saying that it was maybe an overfitting issue where her model would focus too much on specific tokens. As a result, they speculated that this is what caused her to forcefully include terms like "Location" and "Username" in every single responses (even when it was clearly out of place).

No idea if that's what actually happened though. I'm curious if this perhaps has something to do with the parts of her code that are in charge of handling commands (Like move x at y location), the ai spamming those in the background and it (for one reason or another) finding its way back into the model as tokens with crazy high frequencies.

[lost ginkgo]

That's very possible too! I've had LLMs go a bit insane in similar manners before due to overfitting now that I think about it.

If the model was trained on tool calls including "location" then I guess you could cause something like that in theory but my gut feeling says that that's probably not it 🤔 Why would he train the model on something that specific, basically.
Unless he's doing some sort of experiment

[lofty vault]

There are a few different things that could have caused it. Could have been that vedals mumbling sounded like a glitch token(a rare word such as a username that is in training data but isn't properly trained for). The vision data combined with what Vedal was saying could have made things go crazy. Hopefully Evil wrote something in her memories to give him a hint!

[fading trout]

Tbh, my assumption was that if something like that had happened, it wouldn't have been an intentional decision and more of a mistake caused by some unintentional piece of code (Since Vedal mentioned that he was trying some new upgrades).

Also, since the Twins have so many different external components where they can store data (memory file) and make calls to, I thought that maybe some funky interaction could have happened where the wrong calls were being recorded into files that went back into their training data. Though, I also have no idea if Vedal even uses some of their Output as training data.

[fading trout]

Interesting! I didn't even consider the vision as a culprit

[median copper]

If this (tool calling portion) ends up being true I'd like to know why asap

[lost ginkgo]

Well, unfortunately even if that will be the case, I doubt Vedal will be sharing anything with us as it's extremely rare for him to share technical details regarding Neuro

[median copper]

well damn it