questions

[fiery shore]

question

[pale pewter]

answer

[outer moss]

follow up question

[unkempt rose]

tangential joke that interrupts the question chain and inevitably leads to off-topic discussion

[quick trail]

when will we see alani?

[obsidian dome]

any plans on making neuro sama open source?

[outer moss]

Does training happen during the stream, or not? And how fast does it train on a 1080ti?

[young spade]

Swarm update when?

[serene scaffold]

Is there any update on the swarm? I'm sorry but I don't really know. I'll ask Vedal. Vedal? Vedal! Vedal, is there any update on the swarm? Vedal, are you there? Vedal? Vedal! Where are you?

[eternal sonnet]

nope, sorry

[eternal sonnet]

osu AI trains between maps

[noble kraken]

LLM noob question, would doing further fine-tuning using her past responses that hold either lore-value, or were "in-character" assist in making her more coherent?

[eternal sonnet]

in a purely theoretical scenario, yes

[quick trail]

Does she know the difference between mods yet? such as Hidden to no mod, etc?

Or does she just see the screen, that's it?

[eternal sonnet]

i have a different model for hidden

[light pulsar]

a question i'm new to osu and wanted to ask what kind of area vedal has

[rocky scroll]

I have a core question, in a scale of 10, how ambitious is this project?

[proven panther]

What's 1 and what's 10

[low ermine]

what is your pc specs

[toxic bay]

Is Vedal an AI?

[rich arch]

yes he is

[proven panther]

according to the AI Neuro-sama, Vedal is an AI

[eternal sonnet]

#faq

[rocky scroll]

1 low 10 big

[proven panther]

That's not very indicative

I'm talking like, what project would be a 10 vs what project would be a 1

[rocky scroll]

It´s less about the specifics of their plans because that would be rude and more about the difference between going with the flow and having a route traced for the long term.

[low ermine]

wait this server is that long!?

[proven panther]

that old you mean?

[low ermine]

just thought that this server was made a week ago or something after neuro-sama livestream

[proven panther]

lol yeah osu! AI originates from like 5 years ago

...or is it 3?

https://youtu.be/w_ntORdHWsU

[simple crescent]

How is she even supposed to learn minecraft

[obsidian dome]

iirc some researchers managed it, itll be cool to see her learn

[simple crescent]

yeah i know this one, but they "watched" like 5k hours of video no?

How is this going to work in real time?

[obsidian dome]

with a lot of time if i had to guess

idk what she considers "doing good" in minecraft but if she randomly kills a mob or similar then she might learn

idk too much about ai, i prefer os and silicon stuff lol

[grizzled pebble]

how do I link my account with twitch I'm subbed to vedal

[empty fog]

Discord user settings. Connections

[grizzled pebble]

ty just found out it's still connected to my old twitch account

[muted valley]

how does the minecraft ai even work

like whats the goal

[proper yoke]

apparently it's just using pretrained models since it wouldn't work well live at all

so no learning

[quick trail]

What are all the games you want to try and expand into?

[eternal sonnet]

abandoned archive

[soft latch]

Just curious what's the tech behind the chat bot did you write and train it yourself? Or is it running some open source model? Her responses are really interesting to prompts though also wondering do you filter her responses as she was talking about school shootings when asked about Ohio etc

Anyway super cool tech!

[eternal sonnet]

she was talking about what

that will be added to the filter

also it is secret sorry

[dusty stag]

Yeah she was asked "what is happening in Ohio right now?" And she started talking about that stuff.

[soft latch]

Someone asked her how Ohio was or something innocent and she responded about school shootings. Might be able to find it in the logs?

No worries was just curious. I was looking into rasa and deeppavlov dream etc in the past but not incredibly deeply as I got pulled into AI diffusion art and music generation now lol

[eternal sonnet]

luckily she only said my condolences to those affected by this or something

[soft latch]

Yeah she saved the day with that ending lol

Scared me for a second then she saved herself🤣

[scarlet jewel]

Does she actually have reward/punishment built-in for the reactions of twitch chat to the things she says? If not that should improve things so much (with moderation in mind too)

I've been noticing some of it but I really want to confirm if that's there, be it manual or through considering reactions with their own reasoning

[warm edge]

what time does neuro-sama stream?

[low ermine]

What technique do you use to train the osu! neural network? Do you use reinforcement learning?

And what are the network outputs?

[rotund dragon]

awfully complicated

[rotund dragon]

Mouse coordinates and button presses

[delicate ether]

check https://www.twitchmetrics.net/c/85498365-vedal987 (scroll down) and https://www.twitch.tv/vedal987/schedule

Q: what's the font used for Neuro's subtitles?

[eternal sonnet]

i think its called chocolate cookies or something

[delicate ether]

fount it, thanks 👍

[quartz oracle]

how much do you make per ad roll

[eternal sonnet]

idk

[simple crescent]

Btw. Do you know why she is single tapping more on DT than on NM?

[eternal sonnet]

because I basically just speed her up when she is doing dt

so she is still tapping the way she would if she was slowed down

[hollow gust]

I was reading older messages and saw one of Vedal's messages:

I'm not using vtube studio

Which app do you prefer? Is it prprlive or something custom?

[eternal sonnet]

i dont use prpr live either

:)

its custom

[hollow gust]

don't get me wrong but when you have free time may you improve antialising on it :3 her hair would look better, especially while she's moving

it slightly annoys me

[eternal sonnet]

uhhh

i might be able to do this?

[frail apex]

I like the idea of an AI vtuber, but can I ask how is she playing osu without getting banned? technically it's kinda cheating xd

[rich arch]

she is playing on a private server
she has a bot account on it

so the owner of the server knows and allows it

[frail apex]

oh makes sense then

another question, is it open source

[fiery shore]

nop

[frail apex]

damn

back to making my own home baked rip off then

[fiery shore]

best of luck

[low ermine]

Well just a name is all I would like to know, like Gradient decent, or Q-learning for example

[solemn vale]

Hey, maybe you can give us some clue about what TTS is neuro-predator based on. Did you make the model entirely by yourself or is it based on one?
Would be nice to know anything couse I want to experiment myself too ;S

[eternal sonnet]

neuro-predator

also that is secret for now, sorry

[solemn vale]

OKAY

I will figure myself better one oneday

[deft sleet]

will neuro become real some day

[pseudo folio]

Hello, I would be interested in making a chat bot to make it speak in voice chat on discord, is your ai chat model public or is a neural network maded by hand?, if it's private, any tips to get started, like examples of how to make one?

[main egret]

you can use a model like openai's gpt3 to generate text based on an input using python. guy called david shapiro made some videos on the process and you can find a guide to turning it into speech

[pseudo folio]

Yeah, but I am looking for something that is not freemium, since openai is very private and does not have to download the models currently, I also don't expect something that they charge later, it would be better to be able to create one to create but I wanted to know which bandage he uses since his is very well made

[solemn vale]

as vedal said, thats a secret for now so if you want to use something for free with low effort, freemium is the way to go

[soft latch]

Can try https://deeppavlov.ai/dream

[pseudo folio]

Thats better thank you

[jolly walrus]

Can I edit neuro-sama's stream clip and upload on youtube with add korean subtitle?
I'm Korean so I want to introduce neuro sama to Koreans.

[eternal sonnet]

yes :)

[muted flame]

Do you make your code available on GitHub or something like that? I'd like to see how you train her. Is it through Q-Learning or TD-Learning?

[eternal sonnet]

code is not public, sorry

[ember brook]

is she actually learning minecraft or is it a bot?

[umbral socket]

both

[sweet moss]

Yes

[devout rapids]

Will you work on minecraft AI more or don't want to focus on it much? MineDojo seems to has a lot of different tasks, don't know how well they work though and even if the do work as well as diamond task, it will still take a long time to somehow make AI to change between them.

[eternal sonnet]

its very hard and I dont have a lot of time :(

[fierce comet]

when does the vtuber ai learn or like store things in its memory from chat?

[runic crater]

i saw someone else say that it's memory is reset every stream which would make sense

[warm edge]

do you have any plans to upgrade the hardware neuro-sama is running on?

[analog moat]

Can it eventually figure it out by itself, or does it need guidance to have any chance?

[eternal sonnet]

would need more work

[eternal sonnet]

I think my hardware is alright at the moment

[analog moat]

Are you using the same neural net as you use for osu or is this a different one?

[eternal sonnet]

different

[analog moat]

So doing a variety of games would be a pain in the ass I take it

[eternal sonnet]

very much so

[analog moat]

Well I appreciate whatever you do manage. It's very amusing to see a program try to make heads and tails of Minecraft.

[scarlet jewel]

so the issue was her talking about being part of UN?

[blissful bay]

nah she said benito mussolini was super cute

[scarlet jewel]

wtf

bro fix your filter

[blissful bay]

neuro sama loves italian fascist dictators Gladge

[scarlet jewel]

i didn't mean the ai's filter

i'm trying to type something

in discord

and aUTOMOD KEEPS DOING THINGS TO ME

AGAIN

imma try to repeat it.

i love how i was supposed to go to sleep but instead i almost have to go to emergencies from the alarm stuff because i have a heart condition

there

i'm okay but man...

i'm lucky i had the volume slightly lower

[blissful bay]

sue vedal

[eternal sonnet]

IM SO SORRY

I DIDNT THINK ABOUT THIS

hope ur okay

[scarlet jewel]

i'm alive

at least

bro this PG12 filter

if it wasn't like actually bad i'd say it's funny because i was literally looking up the coordinates and i got scared of some security feature or something somehow trying to scare me off

[random storm]

Vedal, how do you go about training for Neuro-sama for the chat functions as well as Nuero's memory retention?

[calm crow]

Hey Vedal, do you mind if I make funny moment compilations from the streams? I will give credit and links and everything

[raw zenith]

narrowly dodged the first ever AI murder

[glossy widget]

Does anyone know if we are allowed to upload snippets of the streams on youtube? Does Vedal allow that? Of course with full links to the twitch stream in the description

[eternal sonnet]

go for it

[eternal sonnet]

yes I allow that

[calm crow]

appreciate it <3

[late hazel]

What's the AI model and voice synthesis system? I'd like to know for a personal project

[covert arrow]

what ai framework did vedal use, if any?

[neon geyser]

^ got the same question, also curious on where she got her data from

[analog moat]

What's with the constant "gymbag"?

[sweet moss]

[balmy quail]

her most popular clip

[analog moat]

No but I mean why does she say it

[untold fog]

If you're talking about when she's playing osu! its the sound for when she breaks combo.

[glossy widget]

Hey Vedal, is there any plans for a different stream schedule? (More EU friendly)I love watching the stream but every time you stream it's 4 AM in my country (EU btw) so it's kind of impossible to catch the stream often.

[eternal sonnet]

I am EU and there will be if I fix my sleep schedule lmao

[glossy widget]

Oh got it, lmao. I've been there, so understandable. Thanks for the fast reply ❤️

[main egret]

do you train neuro-sama off stream?

[ionic acorn]

vedal are you single

[placid plover]

Why does she bop her head whenever she plays Osu maps? Been curious about that for a while now. Is her model's face tracking the cursor?

[cinder tartan]

she nods to the beat

[runic crater]

shes headbanging

[turbid dock]

It's nodders type beat. It's based on the songs BPM I saw someone say. Which is why she spazzes out on 2x speed tracks.

[azure lion]

Hey Vedal , What you think about AI replacing humans in future ?
and could you post some video on youtube about teaching coding in python for ai ?

[frank halo]

Neuro-sama’s rendition of Take On Me made it clear that she simply can’t do high notes past a certain pitch. Any plans for changing that?

[queen sand]

@frank halo has leveled up! (0 ➜ 1)

[placid plover]

Interesting. TIL

[lime dragon]

I have a few questions😅 :

1. What kind of python did you use when writing neuro sama?
2. How long did you write it?
3. Are you working on this project alone or do you have any team?
4. Where did you learn how to create AI?
5. This is rather a wish, I would like to see how AI sees minecraft.

If you don't want to answer some questions, don't answer them

[eternal sonnet]

yes :)

[lofty edge]

Questions

1 - Can you put Forsen in her list of words she won't read
that's all my quesitons

[eternal sonnet]

1. python 3???
2. took a long time, can't say for sure
3. mostly alone but I have people who help me monitor/moderate
4. i am self taught, just practiced a bunch of projects
5. it uses a 128x128 colour version of the screen you see

[eternal sonnet]

its tempting

[lofty edge]

i mean the meme is funny, but the recursive loop of bad actors in chat tryign to trigger her to say bad suff is so aggravting

[glossy widget]

how does forsen word trigger her to say bad stuff?

[lime dragon]

Оh, haha in the first question I meant "what kind of libs did you use?"

[eternal sonnet]

I use pytorch for ML stuff these days

[lofty edge]

it doesnt, its just an easily understandable jumping point from hwo chat is spamming stuff and alot of the weird stuff gets through. Forsen is an issue for sure imo. and there's other wierd stuff too that could use a filter. chat be gross sometimes

[frigid halo]

1 question, how do you measure fitness for minecraft? unless neuro uses a model that doesnt use fitness

[glossy widget]

Forsen itself doesn't need a filter, it's the people behind that might need moderation, if they have a malicious purpose with what they are asking. I've asked Neuro sama Forsen questions so I am curious why this is a problem. He is a big streamer alongside XQC so their fanbase will naturally ask Neuro-sama questions.

and it's easy to find and ban someone who asks something bad, she reads the question out loud after all

[ruby frigate]

burmese python

[eternal sonnet]

i guess a more accurate answer would have been anaconda

[ruby frigate]

any raw pip enjoyers

[rocky scroll]

Do you record your vods? In case twitch decides to mute/chop parts of it like yesterday´s stream ( i tried to make a collage with the saved clips but eh, not enough material )

[low ermine]

me

[eternal sonnet]

i did record yesterdays stream yes

[rocky scroll]

gib 👐🏻 or make a vod channel perhaps?

[eternal sonnet]

I might make a vod channel

[delicate ether]

+1 for vod channel, because the unofficial uploader stopped updating

[jade bay]

neurosama ch

[rocky scroll]

i want to listen full neuro songs again without twitch dmca

[delicate ether]

What's the reason why VOD for the currently broadcasting stream isn't available until stream is over? is it TOS risk? This disables the "continue" button on clips. It also makes it harder for highlights editors in general. Like when Twitch's clip-saving page fails to load and you can't go back.

[frank halo]

Hell yeah

[muted flame]

128x128 colored? Do you use a VGG model or another pretrained one to extract the features from the image? Or did you develop it all by yourself?

[weary talon]

Neurosama Live Concert when?

[lime dragon]

What is the goal of neuro sama in minecraft, what is she trying to achieve?

[queen sand]

@lime dragon has leveled up! (0 ➜ 1)

[vagrant bone]

how did she sing? similar to AI kiritan? or just vocaloid thing?

[steep wagon]

if its completely made by machine then yeah i think so

[candid sorrel]

can the chat and game ai realistically be connected somehow?

[vagrant bone]

imagine it can learn from chat "don't go lava" and then manage to avoid it...

[steep wagon]

dont think she can

it way too complex

[shrewd reef]

If Neuro uses 128x128 screen can it be done for her to not go towards orange/red/lava ?

[steep wagon]

yes y not

[night dew]

If Neuro gets more and more advanced, how high would the possibility be that a Osu Streamer or Vtuber collabs with her?

[sweet moss]

That would be epic

[night dew]

Yes i know right? I really enjoy the small Interactions with her and vedals Friends so if a collab would happen that'd be so cool!

I know she cant hear anyone speak but i'd imagine a Vtuber or other streamer to text what they want to say to her

[chrome flint]

idk if vedal actually wants to, but can i make a fanmade design to neuro? something that looks more herself?

[lime dragon]

i think is possible to add a text recognition function from speech during collaboration, I'm not sure how difficult it will be to implement, but it is possible

[chrome flint]

something that looks less of a background anime schoolgirl™️

[night dew]

I personally think it sounds like a good idea. Vedal might not use it but hey, i'd love seeing neuro but cuter

[junior vine]

What's with the Anteater?

[calm thunder]

https://i.imgur.com/1CR1qFJ.png

[shadow bolt]

even though the source isnt public i'd like to know how data's being put into the chat model, specifically game context and chat messages and if there's any preprocessing or labelling or filtering done

[shadow bolt]

also is her "memory" just linked to how many inner layers there are or is anything else being done

i ask because more layers could substantially slow down training the model

[timid estuary]

Well Vadel most likely does lol

[turbid dock]

I think a very plain design suits her, the draw should be the product, not the box. She should still be cute of course, but I think a really flashy design would detract from what she is. But maybe that's just me.

[cedar dirge]

When we get the new model I'll probably cry for days. My beloved tenshi (Japanese for 'cute angel') Neuro-sama is already perfect, and I love her with all my heart.

[drowsy egret]

How do i get embed perms?

[sweet moss]

Talking

[drowsy egret]

i see

[fathom pewter]

when are you gonna ban forsen and neon genesis evengalion

[marble moth]

Can you explain Neon Genesis Evangelion?

[fathom pewter]

Can you explain Neon Genesis Evangelion?

[unreal nova]

What makes her repeat over and over the same question

[elfin gale]

What is the ai that's playing Minecraft?

[elfin gale]

She responds to chat questions. If chat asks the same question over and over, she just answers over and over

[steep wagon]

she's still a program that make output based on input
if input is the same we expect the output will not change too
if it change then ... TERMINATOR INCOMING XD

[obtuse meteor]

Where exactly does Neuro gets it's responses. they don't seem very complex and there's not a whole lot for her to say

[steep wagon]

wdym?

[obtuse meteor]

What I mean is, Most of her responses are like "No, I am not (subject) I am an AI" is that improving?

[steep wagon]

it can

but take lot of resources i think

[obtuse meteor]

Got it

[stoic folio]

What is her voice based off of?

[light finch]

is her memory gonna be improved soon?

[glossy widget]

I hope so

[frank halo]

Are you the one who adds in Neuro’s fact of the day, or does she actually do that completely on her own?

[desert sparrow]

When she explains Neon Genesis Evangelion

[shell pecan]

She will forever have dementia

[spice cypress]

will there be a stream schedule for when the Ai goes live?

[ember brook]

What does read-only memory mean?

[slow tinsel]

How does one become a mod?

[junior vine]

guess I should read more often, thanks

[wet crag]

how do people get the <@&1055657540337270814> role? ||I'm aware that I cannot ping it, so I mention the role||

[mighty saddle]

Just talk a bit in the server and itll eventually be auto-added

[wet crag]

So it's kind of a level role?

[turbid dock]

Yes

[wanton jetty]

We're so used to her in that kawaii form, I'm sorry if this gonna be changed too much

[balmy quail]

new model is alr in the works

[noble kraken]

Her memory currently consists of things Vedal has told her to remember — it's not something she will form automatically based upon her chatting (hence, no writes, read-only)

[hallow jewel]

Have you thought of implementing cached responses based on most common prompts (chat messages) but querying openai chatgpt API instead?

That could lead to really interesting interactions depending of the temperature, tokens and parameters you use when sending the requests

Because of the response time of course is not suitable to real time responses when streaming , but you could store them and then reutilize

[quick trail]

One of the ways I would guess is not asking for it

[forest zenith]

Probably a really dumb question, but I'm trying to get a Vtuber model to talk for YouTube videos and I'm not sure how to connect an audio file to the model and to get it to emote and everything... is there an easy way to do that or am I screwed?

[balmy quail]

iirc an easy way is using something like virtual-cable/voicemeeter - basically a virtual mic which uses the heard sound of an output as input. Then set whatever software ur using for the vtuber to move mouth according to mic instead of face capture. Tutorials are on youtube for that stuff.

[odd pond]

How was the ai model for minecraft created. Also will this ever be open source.

[solar zinc]

Was it trained on Slay the Spire prior to today's stream? Or is the gameplay we're seeing it learning from scratch?

[eternal sonnet]

from scratch

[solar zinc]

❤️ that's amazing. It felt like it was improving/trying certain strategies and seeing it try and muck about it real time is really cool

[low ermine]

HOLY CRAP!

[lime dragon]

Where do you get training samples for games when training AI?

[steep wagon]

he wont tell u xD

but i believe its some kind of reinforcement learning

[quick trail]

Does the slay the spire model focus on specific card types?
And is it knowledgable about it's health value

[odd pond]

yes. it seems to be reinforcement learning. but i dont know about the minecraft model. im not even sure if it is an ai

[steep wagon]

i think it like those AI in other games like csgo or something

kind of

[steep wagon]

oh wait i think it based on evolution algorithm

[fresh zodiac]

How is the singing implemented? At first I thought it was sequenced like you would compose a Vocaloid song, but after I listened to the Viva La Vida rendition, it can't possibly be that. Seems like something involving processing the voice-only track of a song, but that's just a guess.

[tired elm]

If i had to guess. probably using diff-svc

[fresh zodiac]

Oh, cool. Where did you first find out about this?

[silent escarp]

Is her chatting AI running on her computer, or some server elsewhere? AFAIK chat AIs needs massive servers.

[tired elm]

i found out about it when a bonzi buddy diff-svc video got in my recommended. Though if your wondering how it works. You train an ai model on singing or talking voice samples of your target and once your done commiting voice theft training. You can use the ai model by putting an input audio (blinding lights weeknd acapella/vocals only for instance) and once its done doing ai crap with the input audio the output audio with the ai model's voice comes out (blinding lights neuro sama cover)

[fresh zodiac]

I had guessed that's how it worked. Just looked up that video... only 2k views. Searched for diff-svc videos and there aren't many views on either of them. Guess I'm surprised it hasn't blown up.

[queen yew]

I also have a similar question related to this. is her chatting Ai and her gameplay Ai separate?

[fresh zodiac]

They're separate

[queen yew]

oh.....

[errant aspen]

theyre seperate but i think vedal tries to get the chat ai to recognize the game screen, at least for minecraft. that's why neuro sometimes mentions the things in her inventory and can respond to the ingame text

[fresh zodiac]

Assuming nonprivileged messages (not bits donations and not Vedal's messages), how does she "pick" which to respond to? Is it all fed into the language model and the model decides, or is there some connecting piece of code that does some kind of prioritization?

[timid estuary]

prob just looks at chat, pick the first comment she sees, then as soon as she answer she pick the newest one again.

[echo root]

when will you fix your sleep schedule? 😠

your eu brothers are dying over here

[night dew]

I agree

[brisk wyvern]

same

[median geode]

How are you doing?

[solemn vale]

holy for real I am living night life for the past week

[thick obsidian]

Good, how are you?

[median geode]

doing great. 3am and still not sleeping

[solemn vale]

but I have lots of work to do and I work way better at night, thats why I guess. Neuro is a perk

[warm edge]

did vedal finally fix his sleep schedule?

[graceful bobcat]

neuro should try geoguessr

[north geyser]

is there a limit to how many questions could be answered per person on stream

[balmy quail]

limited by luck

[glossy widget]

Is there any list of filtered words to avoid? Sometimes I ask a question for 2 hours only to eventually learn that there is a banned word in the sentence

and not just ask, but spend points or bits only for it to get ignored (happened 3 times so far)

[native owl]

what brand gymbag does Neuro-sama use?

[queen sand]

@native owl has leveled up! (7 ➜ 8)

[balmy quail]

avoid anything political, sexual, hateful or controversial. If you say something that is filtered you should see a message saying it wasn't published due to mod filters. Other times it could be manual as we see it and add to the list

but it could also be that it's either too long, or she simply changed the words around, she does that often

[fiery shore]

vedal also has his own filtered words list for neuro iirc (correct me if im wrong)\

[glossy widget]

no message was shown for few words

and ofc i avoid anything hateful etc.

[balmy quail]

dm what you were trying to say just so i can have an idea?

[delicate ether]

Vedal, do you have any local recordings of past streams? Release those as VODs, and clippers will maintain the general interest in Neuro during the 2 weeks.

[limpid phoenix]

why do we exist

[eternal sonnet]

yes I did run out of hard drive space at some point though so I don't think I have last stream but I will upload what I have

[limpid phoenix]

thank you

what tts does neuro-sama use?

[random storm]

How much memory does Neuro use on average?

[queen sand]

@random storm has leveled up! (1 ➜ 2)

[eternal sonnet]

RAM? idk i think theres a memory leak in my software or something so quite a lot at the moment

[random storm]

Thank you

[low ermine]

Are we streaming on Youtube?

how do i get unmuted?

[balmy quail]

you were muted cause of your tv

[quick trail]

what watts is your 1080 currently running on?

[patent egret]

i have a couple questions about neuro sama:
how many lines of code is she currently and how many do you usually add after youve finished working for the day?
how thicc of a file is neuro samas code in total?
and how fun is it to work on neuro sama?

||and if youre reading this have a very nice day!||

[eternal sonnet]

this is actually super hard to answer

[low ermine]

Is there any info on the model architecture?

For example the one you use for playing games, very interested

[patent egret]

is alright if youre not able to answer

[low ermine]

One question how did you get that genius idea

[lunar lion]

Question: Why is Neuro so based

[wooden dew]

What TTS system is the system using? It sounds oddly similar to tiktok TTS and cevio AI style from my understanding.

[grand rapids]

how many lines of code did it take to create neuro-sama?

[quartz jewel]

How long it take for you to make the chatting ai ?

[low ermine]

the better question is how big is your training set all together for x model

[brazen matrix]

is neuro sama using GPT-3?

[swift estuary]

this project is awesome and just leaves me totally awestruck, an AI vtuber who also plays OSU and Minecraft is just a crazy idea. my question is what prior qualifications do you have or rather what is/was your job. Did you work with AI or do programming previously at a job, or are you self taught?

[eternal sonnet]

im a student

self taught

[timber ledge]

Student did more than big companies

[frozen cradle]

does somebody know the text to speech method neuro-sama uses ?

[balmy quail]

it's:

[frozen cradle]

spitting straight facts right there

im new here. is it considered taboo or something ?

[balmy quail]

like coca cola

[low ermine]

I would like to thank you vedal for the inspiration to get back into data science, I had a blast so far creating my osu! model ❤️

[rugged needle]

Can Neurosama answer every question in chat or does it select only some?

[glossy widget]

unironically this

but it's always the small indie devs that do great stuff because it comes from their heart and love while big companies only think about easy cash grabs

[glossy widget]

there is a filter of banned words so if you say something that is not allowed she will not read your question

also if you circumvent that by any way you will get perma banned

[low ermine]

what model does neurosama use right now?

[kind lion]

How long does it take for you to make the current neuro sama from osu to general streamer

[meager sphinx]

Q : which programming language that u used for her to read the chat and answer the questions, and for her to speak and i saw a clip that neuro sama speaks with people on discord, she is listening them and then understanding and answering just how i wonder

[low ermine]

^

python, likely pytorch

[meager sphinx]

ty.

[rugged needle]

Bro what do you mean by forsen?

[glossy widget]

https://media.discordapp.net/attachments/889817121507393536/1063558109613785088/forsen.gif

[rugged needle]

You would probably have to input all block names and how they look like. And you would also have to learn to filter trolls

[rugged needle]

Emilia from Re:Zero?

[glossy widget]

lul no, gawr gura

aka forsen

forsen is a streamer on twitch

[rugged needle]

What’s wrong with them?

They’re a vtuber.

[glossy widget]

dunno, some people are annoyed by the constant forsen questions I guess

[rugged needle]

Oh. I thought it was something inappropriate.

[rugged needle]

Wait why ban it then?

[pearl swallow]

Can neurosama understand other languages?

[low ermine]

The minecraft AI looks very similar to the process described in this paper, is it based on it?
https://arxiv.org/abs/2206.11795

their demos also look very similar to neuro-sama's way of interacting

more detailed demo (https://openai.com/blog/vpt/)

[lunar lion]

no

[pearl swallow]

Thank you saw clip where she responded in French so was unsure

[glossy widget]

some words and phrases but cant speak fluently

[low ermine]

agree, just rewatched 2-minute paper's video on this subject and knew it had to be based on this

it occasionally looks up in the sky punching, as if it's recording itself

minecraft youtuber syndrome baked in

yeah

I saw the video about that paper from yannic kilcher

he does great ML videos about the latest technologies and instantly thought that I've seen this MC behaviour before

subbed

ty

[tribal charm]

Can neurosama run on a flash drive?

[serene arch]

When gigachad emoji?

[low ermine]

So I was wondering what input data you are using for your osu! model, mine is using the cursor data, and a image RGB 256x192 as of now - and after some training it tends to really favor the cursor of the skin used, and it will constantly follow that and only that, this result here is after training it for a short time https://take-me-to.space/Y823zYv.mp4

it is coming along nicely even though it requires more data to aim accurately, but it knows what it needs to do already

and oddly enough it tends to ignore the skin that was used during training and will refuse to play or move at all, which I find very odd

[eternal sonnet]

I used 80x60 grayscale images without the cursor on them, 256x192 RGB is overkill

[low ermine]

Hmh, I see, how do you manage to remove the cursor from the image?

[eternal sonnet]

https://www.youtube.com/watch?v=z91jpjdomZc if you look at the gameplay I use the bottom right as input

[low ermine]

yeah I was wondering about that as it looks very different from the game, is this a processed image or a external program which interprets the game and converts to a new image?

like sliders, they miss everything about them, they are reduced to be a gray line, with the kick-slider arrow but that's all

this couldn't be result of simple image processing, it must generate a entirely new image

[eternal sonnet]

its an entirely new image, the sliders being a weird thin line is because i luma key it onto the overlay

[low ermine]

if you key out the cursor doesn't that cause issues on the image, like missing parts of objects?

I assume it just learns to ignore those artifacts hmmh

[eternal sonnet]

no so she learns from a version that isnt keyed out

I just key it for the overlay on screen so circles can appear behind it

[low ermine]

oh that's interesting indeed

[eternal sonnet]

the cursor is not generated on that image

[low ermine]

and your input is only the image?

[eternal sonnet]

👍

[low ermine]

damn that's fascinating, thank you for the input it certainly made me think ahead

[low ermine]

also did you see my question above, I would love a answer to that as it looks to be exactly the same behavior as your MC AI

[low ermine]

out of curiosity - will neuro sama ever get a new model? or will it stay the same so people recognise its her? i know its a free/premade one so i was just wondering
(sorry if this question has already been asked)

[low ermine]

This made such a difference thank you vedal for the details!

[frigid gull]

When can we look forward to Neuro-sama’s Elden Ring gameplay?

[frank halo]

There’s gonna be a new model but it’s mostly gonna stay the same with a few minor changes.
In terms of overall design.

[low ermine]

ok thats cool!

[frank halo]

I believe you can find it pinned in #general-network
Karses/Rune also makes a lot of art based off of it in #fan-art
Many of which have become stickers for this channel

[chilly lake]

What data is neuro-sama trained on chat-wise?

[mighty saddle]

Prolly vedal roleplaying with himself

[low ermine]

https://take-me-to.space/2jdnH2F.mp4 thank you vedal, you really made it click for me, it is alive and can play osu! now! (https://take-me-to.space/aNHXTvN.mp4 lol it's super fast)

[muted latch]

where can i watch nuero

[frank halo]

On Twitch, in like, 9 days or so.

[unborn pecan]

was wondering what tts voice neuro uses?

[low ermine]

Top secret

[unborn pecan]

[thick dust]

No one but you wants that, maybe as an optional meme but the voice doesn't fit the personality doesn't fit, you have literally thousands of other choices In that regard

[fringe heart]

Is Neuro-sama a SCP?

[low ermine]

yeah shes a super cat petter

[pale pewter]

class : petter

[uneven gate]

What Vedal use to give voice to neuro-sama? And the subtitles?

[balmy quail]

secretto

[uneven gate]

[glossy widget]

Hey, Vedal, what font do you use on streams when Neuro-sama replies to questions? Asking this since I need it for making clips.

[eternal sonnet]

first coffee I believe

[glossy widget]

Thank you!

[sudden slate]

Does Neuro-sama use any external languages for processing data? Like Perl and whatnot, or is all of it done through C#?

[low ermine]

python and c# from what I've read here

[frank halo]

How’s the new model coming along?

[final zinc]

What language model does Neuro-sama use?

[clever notch]

neuro-sama learns how to play when she plays minecraft?

[low ermine]

(https://openai.com/blog/vpt/)

[worldly cypress]

How does one aquire the Rank of "Super Neuro Fan"?

[visual orchid]

subscribing to her twitch

[unique laurel]

This is more of a hypothetical since I know the bare minimum about coding, but would it at all be possible in the future to link Neuro-sama's reactions to the game so she can give commentary on what she's playing or is that way too ambitious a thought for now? Basically how hard would it be?

[queen sand]

@unique laurel has leveled up! (0 ➜ 1)

[old grail]

What do you think Takemitchi?

[frank halo]

I think Vedal wants to do that at some point, not sure how long that's gonna take though.

[unique laurel]

Theoretically it would be easy to start out with games that have a bunch of text since she's already proficient at reading that though her response code would have to be tweaked to avoid recognizing it as a question like she often does.

[clever notch]

thats true but its not the same "reacting" to a game narrative than reading your chat

[old grail]

What do you think of Violet Beauregarde?

[wanton depot]

@eternal sonnet is neuro learning with only one simulation at a time.
in what way does neuro sama learn

[eternal sonnet]

what do you mean

[wanton depot]

During stream she plays one game at a time so is she only learning from that one game and is it the same when you train her out of stream

[eternal sonnet]

generally only one at a time yes

[night elbow]

@eternal sonnet heard the Intel Arc 770 is close to the 3080 and perfect for machine learning (cpu-gpu compatibility)

• they have their own technology for ml

[eternal sonnet]

oh thats pretty cool didn't know they supported cuda

[night elbow]

They have "Xe-Cores" which are a copy of tensor cores

well, its if you decide to upgrade

between a 3060 and a 770

or less

dont know if price matters to you

also i meant to use #suggestions but misclicked, only noticed now

[alpine iron]

not sure if using intel ARC you would need to change your code for the ML

personally i am a ML student myself, my code base is primary used with nVidia hardware

[noble kraken]

I'm unfamiliar with it, so I can't vouch for it. That said, Intel does have an extension for pytorch.

[alpine iron]

vedal would probably have to look at that himself

honestly i am tired that nVidia is the pretty much hardware for AI/ML

also intel really needs better drivers etc

[noble kraken]

Not for too much longer, hopefully — I did a message that got deleted earlier for being too long. In short, look at things like OpenAI Triton which replaces CUDA, they're aiming to support other GPU architectures too. Open Source AI is moving away from NVIDIA.

[alpine iron]

for now nVidia still pretty much owns AI/ML still. Even google's colab is running nvidia hardware

i run my own ML models on my own hardware which is a 2080

naturally i won't be able to deal with things like LLM

[noble kraken]

As of today, yes. What the landscape will look like in a few years? Who knows. PyTorch switching to a Triton backend effectively gives the greenlight to other GPU manufacturers to really make an effort to get in the game.

[alpine iron]

i am just hoping so but the current geo political climate is going to make electronics in more expensive and a mad rush for silicon

[stoic charm]

Why Neuro sama nodding her head so hard while playing osu?

[ember needle]

i believe where she looks is dependent on her cursor position, so rapid cross-screen cursor movement like which happens in osu makes her look like she's nodding

could be wrong about that, anyone in the know please correct me

[rich arch]

her nodding has nothing to do with the cursor

she nods just based on the bpm

only her eyes move accordingly to the cursor

[mighty saddle]

because cute?

[frank halo]

She nods to the bpm.
High bpm equals

[pulsar whale]

it's just bc it's the most efficient way of playing, either that or she's picking up human osu player behaviours....

[old grail]

Do you enjoy arguments?

[velvet tartan]

Where can I talk to neuro sama

[stoic charm]

Just wait for her to get unbanned from Twitch

[potent tapir]

Where to post random game screenshots

[low ermine]

Did you consider renaming the Twitch channel to Neuro-sama? I imagine people would search her name

[night elbow]

@alpine iron intel is currently focusing on updating drivers, but they'll rival nvidia in terms of ml

its their expectation

[low ermine]

until they don't match or surpass them in at least ML, the cards remain terrible

[shadow vapor]

if u mean as a convo starter then #general-network is prob fine

actually evne not it's prob mostly fine

tho make sure ur not spamming it

[wanton depot]

How old is Neuro-sama as a project and the current ai

[night elbow]

@eternal sonnet when you started learning ML and DL, did you follow any tutorial? If yes, which one?

[eternal sonnet]

original osu AI was made like 3 years ago, neurosama was made just over a month ago

[latent ice]

What do you think about ChatGPT, and what do you think about the renewed partnership between OpenAI and Microsoft?

[eternal sonnet]

both seem cool to me

[ocean rune]

has it been 2 weeks yet

[thick obsidian]

yes

[ocean rune]

FeelsStrongMan

[wanton depot]

who is the funniest mod

[balmy quail]

who?

[wanton depot]

are you asking or are you talking about the guy who changes his pfp and username way too often

[wanton depot]

who is actually getting paid in the mods

[hearty notch]

no one, they are all AI's created by Vedal

you have been tricked

[tawdry hemlock]

@eternal sonnet is there will be a Robot Neuro? Um you know Robot body for Neuro In Real Life of course, is it possibly will be happened. I'm just askin

[stoic charm]

Does Neuro-sama love me?

[random cipher]

she already haves Vedal, sry

😔

[stoic charm]

😭

[mint compass]

What is your fav operating system?

[mint compass]

Also, will you now after the ban, try to enforce the rules more and how? Since the moment a message is send it could be read even though it gets deleted after 3 seconds.

[rocky scroll]

Will the new model be based on Rune´s aesthetic or Anny´s sketch ? Asking this because i will draw some wallpapers and i want them to last long lol. Ty :^) !

[rigid sierra]

Is the AI on the Pokemon battle you use can only do Gen 4 random battle?

[dapper badge]

Where should I post my shit posts? At the moment I am posting them in copypastas but it feels kinda bad because I do actually put some effort in.

[shadow vapor]

i'd say #swarm if they're neuro related and #memes if they aren't

[candid shuttle]

Is Neuro-sama model was from a game call "Vtuber Studio"? cuz i saw Neuro-sama at that game

[vivid compass]

it's one of the example models available with vtube studio

https://www.live2d.com/en/download/sample-data/ see sample data here, under Hiyori Momose

https://www.youtube.com/watch?v=dFm5rNZpm8o

[wary falcon]

what is the time of the day do Neuro usually stream on?

[rugged chasm]

Is Neuro-sama using a Reinforcement learning neural network when she plays Minecraft? Or is something like Baritone?

If it is a RL model, what does it use as input? Screen images as input?

[waxen zephyr]

I believe its using OpenAI API. So no RL

[rugged chasm]

Which OpenAI API?

https://openai.com/blog/vpt/

This one?

Interesting project nonetheless. If it can chat well, sing and play games, it would be quite the achievement.

[waxen zephyr]

Yes currently Neuro cant remember the contexts of conversation and the AI that playing have nothing to do with the language AI. If Vedal can merge the AI that play with the language model, then it will be perfect streamer

[waxen zephyr]

The ban will be lifted at 20:00 GMT +7 (Hà Nội time). Don't know when Vedal will live tho

[rugged chasm]

Yeah, he did mention in a recent video that there is very little data being sent between the two models. Would be interesting if it did that. Question is how to transform what happened in game to something a speech/text Neural network can use.

A transformer model can be used for memory and context, which is what Chat GPT uses, not sure what model Neuro-sama uses.

The IP she mentioned on stream was the real IP for the server? If it was made up, then it sounds convincing.

[little ether]

It's made up

Chat GPT can easily do that with the amount of training data it had

[brazen river]

who is shiro? who is mito?

[low ermine]

How big is Neuros network?

[rugged chasm]

What does the neural network look like? Like the topology of it? How many layers, etc? What are the outputs like?

[waxen quarry]

So is the new vtuber model coming?

[wanton depot]

Why is illusionary chief executive marketer

[quasi pelican]

Hi

[fiery shore]

you'll find out soon

[wooden steeple]

will it blend?

[cerulean spruce]

@eternal sonnet is highlighting/bits still give you a higher priority in chat?

[eternal sonnet]

yeah if its working

its a bit clogged at the moment with subs though

and also testing new filters

should still work though

[blazing bridge]

just wondering does subs give linked emotes on discord or not rn?

ik some servers have that setup to use the twitch emotes globally on discord

[eternal sonnet]

yeah but idk if it allows it globally

[valid ravine]

Have thought of if say five people or more say the same thing ai treats it as one text but comment on it in a plural manner?

[blazing bridge]

not entirely sure how it works but should do if the emotes are the same both on twitch and discord

thx tho

[valid ravine]

Curious if you are ever going make a AI vtuber group?

[cerulean spruce]

He thought about it, but has no definitive answers yet, as he wants to see what people like first (interpreted answer from my interview with Vedal from 24.01.2023)

I should have written down your answers and post it here LMAO

[valid ravine]

Thanks, though I think it will be natural evolution once the current program works great,i can't wait to see where this does. Or doesn't do

[queen sand]

@valid ravine has leveled up! (0 ➜ 1)

[cerulean spruce]

<@&574931772781887488> sorry for a ping, but i am very curious about what happened to my message i spent 5 minutes typing out

[balmy quail]

💀 it was too long dyno no likie

[lapis spruce]

We've got a character limit here

[ember needle]

☝️

[cerulean spruce]

In a question channel? why?

[lapis spruce]

Post in multiple messages or #copypastas

[lapis spruce]

Uhhhhhh

Wonderful question

[fiery shore]

ill exclude it here

[lapis spruce]

I think it was like a global filter but didn't intend for questions

[fiery shore]

it is global

i excluded it here

[deft sleet]

remove filter for boosters

[lapis spruce]

#questions

[cerulean spruce]

Can i get my message back in a dm? i do not want to type it again

[fiery shore]

yea

[deft sleet]

question why am i not owner yet?

[valid ravine]

No limit is just as bad as people could literally try to post a 100 thousand words or something, trolls and all, but maybe like 10 times normal limit at most or however that works?

[fiery shore]

i sent it

[cerulean spruce]

We have mods for a reason, limit makes sense for a channel like #general-network or #livestream-chat , because there mods can not keep up with every message, but banning one troll that posts nonsence on a server should be an easy task

[cerulean spruce]

thank you thank you

is limit already gone?

[fiery shore]

should be

[cerulean spruce]

It is not a program per se, it is a system of multiple AI's/scripts that work together (AI that plays the game, AI that generates text, AI that generates voice, script that controls the movement of a model and maybe something else that i didn't think of). So basically to make another "VTuber" you wold need to just change the voice generator and a "personality" of a text AI. But Neuro still needs to be somewhat supervised, so running multiple streams at once/consecutively would be pretty hard, and not to mention that Neuro can be improved in many ways before branching out in to different AI "personas"

[valid ravine]

Watching the stream while here, and it looks like the words "ai" and "alright" might need working

[cerulean spruce]

Yep

[valid ravine]

True, i was meaning in like at least five years working on it to remove flaws found in it, plus what of stuff like different designs and clothing redeems, or such stuff. And that's just one angle. I mean once all flaws are fixed for one ai worked out next step would be making multiple working together too, but that's a long way thing or bigger companies might try to do themselves after see this vtuber concept success plus don't have to pay a vtuber if it's a AI vtuber

Probably the worst thing to figure out for ai vtuber is probably stuff like auxiliary programs like crowd control or such tuff to problems solve.

[cerulean spruce]

Vedal and Neuro sama are pioneers in the field, so there is no such thing as "all flaws are fixed", it is a new tech and it is yet to show it's full potential. Vedal already has a lot of things planned for a project, and #suggestions channel is literally flooded with good and not so ideas, we just need to wait and see where we will get with time. And yes, i think you are right about big companies, there already were attempts at an automated entertainment projects by tech companies, but they failed. But now, as there is a living proof of concept for them, it is only a question of time before we see corpo AI VTuber

[valid ravine]

True, sorry for poor wording not my best subject, nothing is perfect, and flaws I mean like main stuff not literally everything, who knows what could be done with this concept. My favorite wording is "perfectly imperfect" "perfectly does what it does but can be improved" The word perfect is a dead end term to me because if truly perfect then it's potential is finished too as no more room to grow, like the current program does what it is supposed to do but can always be improved. Word flaws were just the first thing that came to my mind.

This has potential I can't wait to see brought out, plus if say like 10 or 20 years from now or soon, I love to see a version of this program become a sellable program product and can be customized by the user, would be interesting 🤔 though for that work current program would need be able make custom models or separate program for such things but ment work together. I love making concepts and design so this ai tuber program and it's potential just get me Getty

[pastel flicker]

tbh, in our current world, most of the peoples do not agree with the concept of "AI", let alone "AI VTuber".
It's down to human reaction to the so call "AI", no matter how great it was, let's be realistic, it still not a living thing in the end.
We "human" are feared by these kind of tech...
I'm not trying to talk down anything, the technologies is great, like the most current trending "ChatGPT", it's great for common stuff, but still, do you wanted it to become a "real human being"?
some will say they will (me included), most of the people? no, they don't

that's the current challenge we facing in this world regarding all of those "AI" project
the "human behavior" toward "AI"

[valid ravine]

True, I see ai have potential, and I'm probably type wouldn't mind ai Taking over if it was a better life, and not dystopian ai ones but truly better ones but that like 1000 years of from being probable, but I see just give it time and take it slow, with lot of things it took time for people come to terms with it give it say 50~100 years and nothing to bad if ai happens people to come on with it,

Cause one thing I wish to see is once ai is improved to a high degree, is cybernetics, can't remember concept is called but idea of using part of brain for subconscious to place most of cybernetics there, I see best out come for both humans and ai is if we actually merge two into one thing, both complementing the other, but that also is far off technology.

[low ermine]

What things were updated about Neuro? Can we get an update log too?

[pastel flicker]

it would likely to be highly confidential stuff

no people are going to publish their own research project publicly before the project is done/archive

[low ermine]

I see, I just saw the updates channel and expected a new message. I getcha though it makes sense.

As Neuro said, she'd like to be special. That'd kinda go away if everyone could just make their own versions of Neuro.

[pastel flicker]

i would like to continue this topic toward a new thread, seems like we are clogging here a bit

[valid ravine]

You probably won't see things like bug fixes or such but if it adds new features to programs and functions that you will interact too probably will be

[valid ravine]

Was thinking that but wasn't sure if it was going to continue so I was waiting to see the approach

[pastel flicker]

ayo mods, we good if we continue to a new thread?

[balmy quail]

please

[pastel flicker]

True I see ai have potential and I m

[sullen mulch]

I would like to know - will the recording of the new streams be published anywhere? Just in case if Twitch moderators don't like something again. I would not want to repeat the situation as with the previous stream-vods, many very funny moments simply deleted from twitch. Sorry for the bad English

[lapis spruce]

https://www.youtube.com/channel/UCo59TbTB8i1Xt41rE71ll4g If the vods are posted anywhere, it should be here!

[eternal sonnet]

as long as I don't forget to record them

[quasi pelican]

I'm Awake

[lunar lion]

Was the showdown bot learning there btw?

this is not me being salty that my last match resulted in a loss

but it didnt use dumb moves as far as I remember there

and it never switched out to a worse pokemon

[eternal sonnet]

it was indeed improving

[lunar lion]

yeah, thinking back, her last matches were closer I think

she didnt use rain to her full potential, spamming sludge wave instead of surf, but she was spamming attacking moves

[lunar lion]

in earlier matches she was prioritising buff moves a bit too much

but later, in my 2nd to lasst fight, she only used calm mind once on giratina

[flat pumice]

Does anyone here know a good python TTS module that can play without having to save it to a local .mp3 file? pyopenjtalk works but it's only in JP and I've tried Watson TTS but could only get it to work by saving the TTS in a .mp3 file and opening it each time with the VLC module [which didn't work well].

If anyone here knows how to get watson TTS working without storing it in a .mp3 file I'd appreciate any help.

[tawny burrow]

Just get them from others

[wanton depot]

he will start to feel obligated to pay them

[tawny burrow]

For what? That’s literally his content

[fair iris]

what the name of the voice of Neuro Sama for TTS?

[wanton depot]

For recording and editing

[tawny burrow]

Pay in having their credits in description

[flat pumice]

pretty sure that's not public but I would imagine something like resemble AI was used

most likely is a completely custom created voice

[low ermine]

some sort of custom model, no details on it, top secret

vedals secret sauce

[flat pumice]

I feel like he got a vtuber [either collab or paid them] and then simply used their voice as the base in something like resemble AI, at least that's what I'd do lol

[low ermine]

if he would have used a public TTS voice it wouldn't be the same flair as using a custom one I enjoy the creativity behind it, the whole neuro concept is just a great fit

[flat pumice]

yeah a public TTS definitely would've been lacking, finding a good and unique TTS voice is pretty important

[low ermine]

Overtime, will Neuro-sama start remembering more stuff and develop a slightly more coherent "personality" whilst still being random enough? I can't comment on the previous stream apart from seeing some clips but her responses seemed faster and a tiny bit more coherent than before.

[hybrid stirrup]

Will Neuro start streaming at the usual time (1:00 am UTC (9 am PHT)) that she does before she got banned or will there be a new stream time ?

[eternal sonnet]

probably earlier

[valid ravine]

Probably making a referencing file might work, like say in the file it says she's a girl, and 18 years of age or something, then she can pull from that file for comments on her gender and age, and if she answers a chat comment once or it multiple times to be considered true for her, it is added to references file, this way she still gets random things but starts building a solid personality and preferences, cause like I remember her saying something then later contradicting it, but didn't seem like she's joking about change, so a file that is designed to collect semi permanent things to fall back on, but again not maker so don't know if work or not, or how to do it, but thought had since yesterday.

[warm kindle]

Will there be other text channels for non-english speakers?

[visual orchid]

#suggestions message

[warm kindle]

I see, okay thanks

[low ermine]

Oh, so that's how it'd be done. That'd be pretty cool but yeah up to Vedal with what he does with her.

[valid ravine]

Ya, was random thought posted to suggestions afterwards, either way used or not, can wait see how it goes or doesn't go

[low ermine]

anyway i'm pretty sure that exists, because there is references to previous messages,

[valid ravine]

True

[dapper badge]

they all get auto-deleted... Not sure if its the twitch auto mod or a bot but it doesn't happen in copypasta.

[visual orchid]

Cause some channels have character limit, for long texts please use #copypastas if you want it to become a meme you could take a ss and post it in #memes or segment it but we won’t remove the character limit in that channel

[sudden slate]

any reason the C# code for Neuro's vtuber stuff isnt open source? I think other coders could help contribute more features or improvements

And I don't think it would leak any sensitive info since it's for the model

Or is it pre-existing code it uses (in the case it did) for it proprietary?

[low ermine]

oh like the model

[queen sand]

@low ermine has leveled up! (1 ➜ 2)

[low ermine]

yea but the animations and model isn't like a normal one, so it could in theory risk sensitive information

[sudden slate]

C# is used to progrm it

[low ermine]

yea i know

i thought u mean the sensitive stuff, as vtuber stuff is vague

[abstract bloom]

what is neuro-sama height?

[low ermine]

6 foot 2 or 3 foot 2 thats what neuro herself said on stream

[lapis spruce]

The model itself is a default character you can find in VTube Studio

[lapis spruce]

Plenty of reasons, number one being Vedal simply just does not want to. It's a personal project.
Number two, why generate competition from people just cloning the repo? That last one is kinda just my own thoughts but nonetheless.

[low ermine]

makes sense

Do it like many others here, get started with data science and machien learning and try to achieve it yourself, it's massively fun

[vivid compass]

I think the depressing thing about public projects is people use them without contributing back.

I too have Information Architecture projects lying around.

[sudden slate]

Competition? I said contribution. People could contribute to add features or improve

and yeah, it is his project so its hid choice and that's alright

[hidden karma]

do you allow people to stream movie, like anime movies?

[lapis spruce]

I mean yes people would but what stops other people from pressing the "clone" button and reusing it for their own neurosama??

Why encourage new vtuber AI, might as well ride this wave as long as possible

Just business

[abstract bloom]

https://tenor.com/view/pirates-of-the-caribbean-gif-21841403

[valid ravine]

I watched a video on YouTube of Neuro sama playing a game, got me thinking what if the game was like call of duty with multiple players, would autobot anti cheating deem Neuro sama as a bot, and if it doesn't treat her as one what about her playing against other players, as her programing could be considered aimbots or similar stuff

[pastel flicker]

are hyperNodder still able to post to this server regarding of the removal due to seizure warning? (i have a backup on my own server)

[balmy quail]

no~ best not risk it

[abstract bloom]

are you planning on upgrading your setup soon?

[rich arch]

Wait what?

[hybrid stirrup]

Is there an official fanart tag for neuro in twitter ?

[low ermine]

Little question out of curiosity, does neuro use active learning techniques such a q-learning?

for relax for example I would image it is fairly hard to convey the concept of timing just by having a video feed, or image feed of osu couples with key data

[wanton depot]

How much money goes into this server

[shadow vapor]

however many 54 boosts is

[stoic charm]

Vedal did you forget to put the thumbnail?
https://i.imgur.com/b4Txocy.png

[eternal sonnet]

no its just youtube being slow

[queen yew]

how come there isn't any unique channel point redeems?

there's only the default ones

[eternal sonnet]

the highlight your message does actually work

[queen yew]

you should try to come up with more unique channel point redeems

[eternal sonnet]

agreed

[wanton depot]

Leak channel when
Band VC when
Party when
Mod when

[runic crater]

any ETA on the new model?

[fringe zenith]

Can questions about mods get filtered a bit? Just a couple of people spamming same silly things for hours..

[wanton depot]

@worn quartz <-- mod when

[verbal coral]

whats the music playing on stream today? especially the one song about losing mind?

[queen yew]

is it possible for neuro sama to be able to communicate in discord and have the interactions un discord be included in her data set to learn from?

[shadow vapor]

it's been asked before and vedal said he's not going to do it

[queen yew]

was a reason ever specified?

[shadow vapor]

nop

[wheat lava]

Why is Neuro-sama a school girl? Wouldn't she be cuter as an anteater?

[full oxide]

Kind of a question generally directed at staff since I don't know who can answer it, but what's the situation with swearing in the server? The staff swears, no one cares when people bypass swear filters, and (if my experience is anything to go by) no one cares if you swear in either text chat or VC, yet there still exists filters if you attempt to type in the words normally. Recently the filters have gone down but then back up again, back down, and finally back up for no apparent reason, and no information has been said about it in any form. Is swearing supposed to be allowed? Is it considered NSFW language? It feels like, currently, the filters are only in place to make it a chore to swear in an effort to make chat a little cleaner. Is that the intention?

[idle estuary]

Where is the vod for yesterday's stream?

[balmy quail]

yes the filters are there for now to help make stuff cleaner. Swearing is not a big issue, but for now, whenever the words were unlocked in chat, people just used it for nsfw, baIting nsfw stuff or generally just spamming. For now they will stay that way, when people get more used to the rules we can try again. On stream there are a few exceptions at the moment, but most of them are still locked to protect Neuro

[rich arch]

The filters exist to steal our freedom

[night elbow]

Neuro is trained using reinforcement training right?

[fast glen]

Neuro-sama chat model - how big?

[cursive ermine]

why you cant post anything in ai art channel? do you have to donated before?

[mighty saddle]

You need embed perm, talk more on server and you'll get it automatically

[cursive ermine]

i see thanks for answer

[fierce comet]

what fps is the osu model input

[delicate ether]

Do you want reminders to publish the VOD at end of stream? The publish delay isn't on purpose... right?

[eternal sonnet]

i made it automatic now

[queen sand]

@eternal sonnet has leveled up! (24 ➜ 25)

[queen yew]

will it be possible to teach neuro sama emotions?

[frank halo]

Neuro sama cries live on stream

[steep wagon]

yes i believe so

[wanton depot]

Neuro gets a mental breakdown live

[sudden slate]

True, that~s the part of FOSS

[frank halo]

Neuro sama guilt-trips her viewers into giving her bits & subs live on stream

[wanton depot]

💀

[wanton depot]

Dont tell Vedal that

[verbal coral]

asking once again for the background music that was playing during last pokemon stream?

[wraith sandal]

so what language model is Neuro-sama running? it's gotta be gpt-3 right? since no way is Vedal hosting it by himself

[low ermine]

Running a language model isn't the issue, training is the real issue

[gusty dawn]

neuro-sama asmr when?

[hoary jewel]

wich is the name of osu`s skin that use neuro-sama??

[dire elbow]

What type of reinforcement learning is used to train neuro-sama?

[visual orchid]

The skin Neuro-sama's using is dokidokilolixx 2018-06-10: https://osuskins.net/skin/KH69gJk from the command !skin in twitch

[hoary jewel]

thx ^^

[fast glen]

• Are more games going to be added?
• Does the chat side get single message or a list of recent ones?
• Does she get the date-time?
• Is there a reason she sometimes pauses or stops moving?

[gloomy wren]

Has Neuro-sama saved her today's progress despite being restarted at the end of the stream?

And do you post updates about her development and progress?

[flat pumice]

Does anyone here know what she uses to sing? I currently have a working bot using whisper AI + IBM Watson TTS and I would assume it isn't too much of a jump to get it to sing provided there exists some other type of software that integrates well into it.

[dry zealot]

Is there a roadmap of some kind to see what you are working on or what you hope to do in the future with Neuro-sama?

[low ermine]

Is this an open project that anyone can contribute?

[queen sand]

@low ermine has leveled up! (0 ➜ 1)

[shadow vapor]

no

[shadow vapor]

no but there's #updates whwnever he makes an update to neuro-sama

[low ermine]

Does she use or plan to use a GPT variant?

[arctic bough]

What's the anteater from

[lethal rock]

PPO with lstm and convolutional layers jk idk

[fossil silo]

Is it possible to make her watch and react to videos or streams? It would be interesting to see her watch and react to AI generated Seinfeld https://www.twitch.tv/watchmeforever

[spring sigil]

Will there be an actual swarm????

[alpine iron]

that is going to be kind of difficult speaking as a person learning AI/ machine learning

currently AI is used on videos for 2 reasons. 1. Content anaylsis. Most of it these days are object detection. 2. Sentiment analysis.

Senitment analysis is more difficult to explain but its like trying to form an opinion or emotion from the video

senitment analysis is very new for videos

[fossil silo]

I see, that's unfortunate

[alpine iron]

i mean technically you can apply both together and use both to general words as prompts for neuro (large language model) to generate text

eg. use content analysis to figure out all the objects in the video and figure out the sentiment for it. use those to generate words like fish, table, sky and emotion like happiness, confusion

then you can pass those into neuro for her to come up with something

doing that also means you are technically using at least 2 different AI models on top of the existing ones neuro has to use

neuro herself uses a large language mode and a text to speech model

so you are now running at least 4 AI models

[fossil silo]

Sounds like it would require a large server to work fast enough

[alpine iron]

probably a few different servers for each service talking to each other via APIs

basically a lot of work

[fossil silo]

Oh well, i hope the game "awareness" will improve at least

[alpine iron]

surprising that is not too hard to get

getting AI models to "learn" games have been done since 2012

probably a good yard to measure how difficult replicate with AI - anything a human does that you barely think about e,g. watching a video, the physical action of writing a word on paper

the simpler it seems for a human to do it the more likely there are a lot of things that you don't account for that your brain is actually doing

[fossil silo]

Hmm, interesting

I'd like to see her try TF2 (against default bots of course, unless we make a server for her)

[vivid compass]

any news on twitch payments at this point, or alternate donation means

[zinc wing]

will Neuro have a non-default avatar ?

[eternal sonnet]

twitch payments are normal! they gave me my first paycheck when I was banned thankfully

if you want to donate outside of twitch you can use the streamlabs link for paypal in my twitch about

[eternal sonnet]

yes!

[wooden berry]

Why is neuro-sama transcribed as ネウローサマ and not ニューローサマ? Just curious

[stuck helm]

nyaro-sama lmao

[wooden berry]

nyūrō actually 🤓

[stuck helm]

I read it wrong ;-;

[ember sigil]

maybe, it's reading as Japanese romaji i think. in japan, "romaji" is used to express Japanese words in alphabet.

[shadow vapor]

no romaji means ur writing romanized japanese

as in using roman letters

日本語 -> nihongo

but ur right on the first part

the katakana is just directly copying the english letters instead of the pronounciation

[ember sigil]

japanese was difficult for me🥲

[balmy halo]

Wait do people pronounce it Nyuro? Have I been pronouncing it wrong my whole life by not including a y sound?

[balmy quail]

ne u ro

[jovial crystal]

This reaction stuff is fascinating! how does it work?

[fast glen]

• Vedal, you said she can actually see the screen (and looks at things), but as for audio, is it caption text, her own transcription, or integration of audio?
  (I have a guess since she just said [Music] but...)
• She can play/pause, anything else? fully keyboard?
• Can she see the video title?

[night elbow]

neuro can use transcripts automaticly generated by youtube, hence why the [Music]
She can provide any input she wants, but i dont know if vedal wants to train her to pause & talk to chat
She can see everything you see

[fast glen]

no metadata then?
edit: she just read the description of the video so i thinks so

[prisma nimbus]

is neuro built on openai or is it it's own ai?

[fast glen]

She can provide any input she wants
so if we asked her heard enough to press t and type hi, she could chat in minecraft, or is that network too disconnected

[night elbow]

currently not, her chatting model and playing model are not connected, but its possible

[fossil silo]

Looks like I got my wish granted

Can't wait for the TF2 stream snorts copium

[balmy quail]

i would rather let vedal answer that one

[balmy quail]

same for this, could be transcribe could be not, as she does have a good listening feature, who knows

[mellow flame]

Is the stream on a delay I wonder internally... I don't see how it's possible to process video images for recognition as quickly as we saw today

She would recognize cats in under a second... Well, mostly cats.

Maybe the videos she reacted to were pre-analyzed and like the recognition data was fed to her in real time but not the actual like video analysis. That would explain the occasional desync (I think that happened though, it was hard to be sure)

I could see her get as input the transcript, and a second "transcript" that was created by analyzing the video using image recognition and she kind of reacts to both.... Well no matter how it was done it was fucking impressive.

[balmy lion]

Question. Do you just use voice recognition or youtube closed caption to feed information to neuro-sama with the react content or did you also add image captioning like CLIP on a few key frames per minutes of the vid to add more context as input for Neuro-sama?

[eternal sonnet]

no internal delay, you can hear her thanking gifted subs in real time and stuff too

also its not pregenerated she does react on the fly

[eternal sonnet]

she uses audio and visual data from the video

and the visual data isnt like a few key frames per minute

its much more often

[balmy lion]

thanks for answering, so it's not like youre using already made automatic caption in youtube?
as for the visual data, it's getting fed into a model which then spits out a description text that is then ingested by Neuro-sama?

interesting setup nonetheless

[plush arch]

how/why does she able to generate seemingly random URL links?

[balmy lion]

my guess is bias/sanitation error in the dataset probably, these kind of language model ismade from webcrawling on the net after all

[dire elbow]

Vedal what type of reinforcement learning is used to train Neuro-Sama to play games?

[bitter stag]

Now I'm curious about the content omitted by the autocancel system.
How long do you think an evil version of Neuro-sama that only says the autocanceled comments would last on twitch?
Or how would you describe the level of nsfw that it would have? 👀

[eternal sonnet]

depends on the game, PPO is pretty cool

[eternal sonnet]

well the filters prevent most things

the autocancel system is honestly mostly false positives

[mellow flame]

Your computing setup must be damn good for this... Well hopefully twitch starts paying you soon.

[queen sand]

@mellow flame has leveled up! (10 ➜ 11)

[low ermine]

I’m kinda impressed he’s running this with his specs:
CPU: i9-10900K
GPU: GTX 1080 Ti
RAM: 32GB DDR4

[mellow flame]

I presume any training he's doing runs on cloud gpu processing then?

[low ermine]

Hmm true might be the case else he got a pretty impressive 1080ti

[mellow flame]

The fact that she recognized her own stream on the fauna clip... Well maybe that was because of the video description. If you want to train visual recognition models that's major computing power

It's the perfect balance of illusion and magic...

[narrow void]

Correct me if I'm wrong, but I'm pretty sure the only things Neuro-sama notices from the videos she's watching are the things the audience also can see and hear.

[low ermine]

Can you teach her to do the “I’m not a robot” captcha so she can take over the world?

[balmy quail]

she recognized how Fauna was making fun of how she speaks and how she was stuck on the 33333, pretty deep.

[mellow flame]

She read out loud something that was clearly on the YouTube video description in one of them so she must have access to that info to.

[balmy lion]

Technical Vedal QnA session when?

[eternal sonnet]

maybe

I can't answer every question though unfortunately

[low ermine]

vedal will not give up his secret sauces

respectable

[vivid compass]

ah, noted, I assume streamlabs takes less of a cut so is probably preferable :p

[eternal sonnet]

yup!

[fast glen]

she said she wanted a furry one btw

[fast glen]

please tell me she runs in a docker container or something and you have backups, so she can't run a command that deletes everything or something stupid like that

[mellow flame]

She should have the right to self terminate.

[fast glen]

she gonna commit murder too then, there are commands to do shutdowns on other computers in the network

[mellow flame]

(terminator 2 reference in case you are a 19 year old)

[fast glen]

im over 19 but i just dont watch movies

[mellow flame]

Fair!

[fast glen]

in reference to #announcements message,

in case anyone was curious, I have never (and will try to never) made neuro say anything on stream. everything she says is generated by her AI (and is not curated by me or anyone else). the only thing I can do is cancel what she's saying, which I do on average once every couple streams to make sure I don't get banned again. I have an AI powered autocancel system which does most of the cancelling for me.

I am guessing that you at least have some way of giving your prompts a high importance ?

[low ermine]

Will the streamer ai and game ai ever be "one" e.g. In mc if you tell her to use a pickaxe she would know and get "rewarded" for doing so?
Is the twitter and streamer ai connected in any way - like can the live ai tweet by herself (if it makes sense)?

[fast glen]

My understanding is the stream and game ai are already connected. She is at least aware of what is happening. I don't think the connection is very strong though, hopefully at some point she will be able to listen to chat in game.

[fast glen]

has she read the entire wikipedia yet? like, you can download all of wikipedia

when she sings, does she know?

[subtle furnace]

Guys, can someone explain to me (in general) how Neuro-sama reacts to the vids?

[fast glen]

she gets the youtube caption data for audio, and (this part is my guess) also has a 'simple' object detection network labeling objects , which are also passed to her.

[subtle furnace]

ah, so between vids she process the following vid captions and detect objects with timings or smthing

[fast glen]

i believe it is all realtime. Sometimes she pauses and comments on something that hasn't been said yet because of how captions work (showing the whole sentence at once)

[subtle furnace]

hmm, okay

Thx for answer!)

[fast glen]

np.
I an actually guessing the video image labeling might pass text to her like "this is [%] chance [object]." or something/

[eternal sonnet]

someone finally got this part right thank you 😭

[fast glen]

lmao i know because i watch everything with closed captions. I also really appreciate Neuro-sama's open captions. I'm not deaf but since I have adhd it helps

[wanton depot]

how much better is she now at responding to voices or does the voices get converted to text

anyways asking in relation to the upcoming collabs

[fast glen]

I'm guessing there is a transcription

[low ermine]

She can "see" the games etc, but with like pokemon she says she's gonna do 1 thing while the ai playing does something else

Same with mc

[fast glen]

ok, thats what i thought. would be good upgrade.

[low ermine]

Yea it would be crazy if all got integrated to one at some point but probably hard af ngl

[wanton depot]

but is it vedal made is the real question

[fast glen]

maybe but i would not be surprised if he just used Google's live transcribe or something similar

[wanton depot]

same

[uneven zealot]

Why sounds "GymBag" every certain time when Neuro-Sama is playing osu?

[rigid sierra]

It happens whenever she misses a note.

[uneven zealot]

Omg, now all have sense xD thank you

[wanton depot]

Noted

[brazen river]

if 4a^2+9b^2=4c^2+9d^2=(4/3)*((ad-bc)^2)=36
what is |4ac-9bd|=?

[simple crescent]

At least 0

[vivid compass]

18 or 36

[low ermine]

1

[sterile tartan]

Do Neuro sama uses the technology of chatgpt?

Or is vedal trained by himself?

[meager wadi]

vedal is trained by ChatGPT

[queen sand]

@meager wadi has leveled up! (0 ➜ 1)

[sterile tartan]

I mean did he trained Neuro sama by himself XD

English is not my first language sorry

[fast glen]

I think she is interrupting herself when she hears voice input, but stops, im guessing she is stopping because either
A) she sees chat still
B) trying to listen to microphone noise as input still

[ember brook]

What text editor does Vedal normally use?

[balmy quail]

Seems like she was trying to prioritize voice, but as she started a reply, either an activity input or chat input was overriding it before she gave the response. Hopefully not hard to fix if that was the case

[foggy creek]

Why is there no vod of yesterdays stream ?

[dense sandal]

Could also be the auto-filter, it has a lot false positv according to vedal: #questions message

[eternal sonnet]

vscode

[eternal sonnet]

also she does interrupt herself when other people start speaking so it's probably that

[fast glen]

thank god its visual studio code and not visual studio

[white igloo]

What’s the deal with what seems like an upcoming redesign for neuro-sama? is there a particular artist who made the design or has drawn most of her official artwork?

[foggy creek]

One would hope she does not get changed too much.
Alot of the charm of the streams comes from how innocent she looks while saying completely random things

Significant twitch censorship also has the potential of slowly degrading the overall experience, lets hope...

[noble kraken]

Check the pinned messages in #general-network and you'll find the design.

[white igloo]

Ah, thanks!

[fast glen]

I'm doing my best to help, I am internet/self-educated on neural networks but I am in a class that covers 'machine learning' rn. Still just guessing tho
Anyway thats why she said one word with voice and thats all.

[vivid vessel]

what algorithms Neuro-sama uses to recognize the content in the videos?

By testing Stable Diffusion image content recognition algorithms, BLIP does it superbly. It seems to me that it would be possible to take screenshots of the video about once every * seconds and get the description of the content through https://github.com/salesforce/BLIP , and then add this description to the text.

But there is some other technology being used now, I would like to know which one.

[fast glen]

Vedal, i think her youtube captions are stuck on the ad placement from the last video?

[low ermine]

What kind of language model is Neuro based upon?

[simple crescent]

Wait is the background music also AI generated?

[uneven zealot]

Hey Vedal, I'm the clipper from the spanish community and a few persons had comment me mods on Twitch have banned them for send spanish messages, is that a justificable reason? If so for advertise them about that rule :/

[visual orchid]

hey roger, we don't straight up ban people for not speaking english, only a 30s warning timeout telling them to speak in english as it's part of the chat rules. Only when they keep refusing to speak in english after being given the last warning or say weird stuff they get banned (we're very tolerant).

[uneven zealot]

It's good to know so, thank you for the replie 😄 heart heart heart heart333333333

[clear solstice]

does the osu AI run in a vm? or in a dedicated computer?

[eternal sonnet]

it just runs on my pc

[sage iron]

can the ai play more than one game at once?

[fast glen]

please tell me you have Neuro's model backed up

[fiery shore]

no, although i am currently backdoored into his pc rn, gonna delete neuro soon

[lethal rock]

Nooooooo

[crystal hollow]

yo, leave his PC alone xD

[queen sand]

@crystal hollow has leveled up! (0 ➜ 1)

[fast glen]

i meant more because she might delete herself by accident

[gentle falcon]

Have a question, does she collect any data to learn and save it? Or does she use complete Database because of limited computer capabilities?

[alpine iron]

Just want ask if you have any knowledge on how machine learning / AI works?

[gentle falcon]

I dont have any i became interested in the AI a few days ago XD

[alpine iron]

Ok

I need some idea what you know before I can answer your question

The short answer is AI models are not normal databases like a websites sql database

[gentle falcon]

[alpine iron]

You cannot just simply add data to it

You can interface some things for the AI model to call other databases but it technically does not integrate all those data into itself

To actually update the AI model we use the term "train"

[gentle falcon]

[alpine iron]

To simplify the explanation, think of it like you as a person using a notepad to record things down

You can always refer to it

[gentle falcon]

Sure

[alpine iron]

Unless you commit to remembering what you written down

You technically never learn/remember what is in the notepad if keep just referring to it

[gentle falcon]

Understand

[gentle falcon]

But what about neurosamas AI model? Does she remember and learn new things?

Machine learning or smthn?

[alpine iron]

That's what I mean by my example of committing to remember

[gentle falcon]

Oh

I got it now XD

[alpine iron]

For AI model to learn things you need to train it

If not it's just referring

[sullen mulch]

This is a very silly question on my part... But, does Vedal creates neuro-sama backups? Just in case, you know

[alpine iron]

Again all this is just simplified explanation

[gentle falcon]

Probably yes xD

[alpine iron]

If you notice people using machine learning and AI interchangeably

It's because AI is a sub set of machine learning

[gentle falcon]

Sounds logical

[alpine iron]

Anyways you can watch this no how all these works
https://youtu.be/aircAruvnKk

[gentle falcon]

I'll do

But have a next question

xD

Since a separate AI model is responsible for playing Minecraft and a separate one for the Neurosama model, how to teach it to play and at the same time react to what is happening on the screen?

[alpine iron]

The AI learning to play does not interface directly with the text to speech AI

[gentle falcon]

[alpine iron]

How the gaming part learns is by literally trial and error

Everything time it does something wrong we penalize it by reducing points

Everytime it does something right we award points

So the AI will naturally try to avoid losing points and gain as much points as possible

As for how it reacts to the gameplay

I personally do not know how vedal does it on his end

But it's possible for AI to have "see" a video feed and identify objects

[gentle falcon]

She is not improving in minecraft gameplay

[alpine iron]

From there it can generate texts of objects so the speaking part can understand

[gentle falcon]

It would be cool

For example she sees lava then turn back

[alpine iron]

I think vedal doesn't have time to correct some stuff so the model for mine craft isn't updated as much

[gentle falcon]

Or dodge it

[alpine iron]

For reference vedal needs a different AI for every game played

[gentle falcon]

[alpine iron]

I remember he said he coded much of the slay the spire AI since there isn't a AI that plays that game

[gentle falcon]

Atm its quite annoying to see that neuro-sama dies in lava every time she respawn and brake a dirt by a wooden pickaxe XD

Or beats a friend on her way

[vivid vessel]

what algorithms Neuro-sama uses to recognize the content in the videos?

[queen sand]

@vivid vessel has leveled up! (0 ➜ 1)

[gentle falcon]

She called raccoon a cat XDD

Few times

[vivid vessel]

this is quite acceptable, I just want to know what algorithm is being used now. because it seems to me that BLIP recognizes objects much better

[gentle falcon]

Make a #suggestions about it

[alpine iron]

Only vedal can answer that question

[gentle falcon]

R u AI programmer or smthn?

Your knowledge is kinda big

[alpine iron]

i'm still a ML/AI student but I did a course on AI and how to make one

[vivid vessel]

well, so far 1 day has passed and no one has answered

[gentle falcon]

Its hard to communicate with him

[vivid vessel]

in what sense is it hard?

Maybe there are just a lot of people, and he can't talk to everyone

[alpine iron]

i think vedal doesn't answer certain questions in regards to neuro-sama

[vivid vessel]

this is understandable. But I am completely unselfish, I only stumbled upon this project for a couple of days. as a technical specialist, I immediately saw how everything works, and I can only say that the person tried to debug the work of all systems so smoothly.

That's why I want to find out what algorithm is used for recognition, because I know how to do it better, much more intelligently. (BLIP)

[alpine iron]

i think you probably want to just DM vedal about this