#[LOOKING FOR ARTISTS] ScammerSama (Scambaiting Neurosama implementation)

1 messages · Page 1 of 1 (latest)

opaque falcon
#

This thread discusses the implementation of the tools required for a Neuro/Evil scambaiting stream.

I have developed an API that allows the twins to call a scammer and essentially waste their time to prevent others from being scammed. Neurosama can raise awareness about scams while also providing for a hilarious stream! All the code is implemented already and you can check it out on the GitHub page here https://github.com/Ubuntufanboy/ScamSama

The API uses Twilio for calling scam centers with custom phone numbers and the audio from the calls is streamed via a discord bot which is put in the same VC as Neuro or Evil. 0 external integration required on Vedal's end in terms of software AFAIK.

Latency is under 1 second for transmitting/receiving audio (when tested on regional calls in my personal region). This is the same for speaking on the phone. The latency comes from the mobile carrier not from the code itself. That's a bottleneck outside of my control.

More info on the README.md

Hope you all enjoy!

Notice to Vedal,
If you wish to implement this in a stream, I would be more than glad to help you set this up. Shoot me a message and I will help set it up.

GitHub

Neuro Implementation for Scambaiting calls. Contribute to Ubuntufanboy/ScamSama development by creating an account on GitHub.

#

I will send video demo of the code working later. It's quite neat!

opaque falcon
#

Also it should be noted that I still have some things to set up.

Current TODO:

  • Fix weird race condition edge case
  • Set up trigger for Neuro to start a call (she should control it)
  • Include installer script
  • Maybe improve audio quality slightly
  • Give Neuro toggles for hanging up and muting herself (so she can speak to chat momentarily and not leak that she is a streamer to the scammer)
misty surge
eternal acorn
#

That’s pretty funny

vagrant hare
opaque falcon
#

The bot is essentially the middle man in communications

vagrant hare
#

some of the future toggles can be API sure, but not the bulk (ditto)

opaque falcon
#

Yeah the toggles I can take a look at it. Also the code isn't very long or complex rn. You should be able to make a PR pretty easily if you are interested. I can explain anythings you don't understand in the code too

misty surge
#

I haven't had enough experience with the API yet to do good prs, I don't think it should be that hard however, the API documentation is pretty good

opaque falcon
#

Yeah the API could be quite useful for toggles but I still gotta think about the best way to go about that

#

99% sure i'll use the API for muting, hanging up, and starting calls

misty surge
#

vedal is already afraid with security stuff even with the API and has yet to put into a good implementation for it, so I would recommend putting as much into API as possible

opaque falcon
#

The thing is, the bot would run in a private server with just neuro and Vedal.

trail jetty
#

converting this to use API will be very interesting to say the least

opaque falcon
#

So it should be completely fine without needing to add technical debt

trail jetty
vagrant hare
#

Maybe tie startup to API as well, IIRC there is a startup and shutdown being considered

trail jetty
#

what if you added a zip bomb with the build files?

opaque falcon
#

The code is like 150 ish lines. It's very simple and easy to audit yourself

trail jetty
#

your code is very short but bluecat was talking about in general

#

unless I am being completely stupid

#

and don't understand the convo

vagrant hare
#

I think bringing up general cases is not useful in this case and is just being a wrench to this clearly abnormal usecase

trail jetty
opaque falcon
#

I don't doubt it. If I have to use the API specifically it's not a massive deal but it would just be unnecessary complexity imo.

So long as this gets implemented I will be a very happy man

trail jetty
#

but I guess in this specific project it's not too bad

vagrant hare
#

I don't think the neuro API lives in vedal's stack to where voice can be grabbed

#

So no matter what some extra will have to be done

trail jetty
#

though someone did write a library to perform STT over discord calls and posted in this discord before... let me see if I can find it

vagrant hare
trail jetty
opaque falcon
#

as in hanging up?

trail jetty
#

Not even

#

hanging up can be a registered action

vagrant hare
#

Nah like shutdown itergration so killing the bot and stuff

trail jetty
#

startup is meant for initialising the connection properly such that previous actions are wiped

vagrant hare
#

hangup should be an action

trail jetty
#

the graceful shutdown should be fine, not sure about immediate though

opaque falcon
#

Frankly, I'm not awfully knowledgeable about the neuro api. Can the api do things like muting/unmuting in a call?

#

that seems somewhat complex

vagrant hare
trail jetty
opaque falcon
#

So the API doesn't come stock with it though

vagrant hare
opaque falcon
#

I mean, I am quite familiar with Neuro's technology and the discord api so I could just send a PR tbh

trail jetty
trail jetty
#

idk what you mean by action activition is though so I'll assume you mean LLM tool calling

#

right, time to get to work with PRing this

opaque falcon
#

yeah tool calling

#

I mean a tool for (un)muting, joining and leaving calls should all be trivial

#

I would be kinda shocked if it wasn't already implemented tbh

vagrant hare
#

When in doubt just alternate the mute and unmute actions

trail jetty
#

I would assume Vedal already has something like that for Neuro's current Discord integration

opaque falcon
#

Also there needs to be some command line arguments

trail jetty
#

why?

vagrant hare
#

I'd guess if you impl mute then you'll have to make sure she gets back on call or hangs up, might be a use for an action_force?

opaque falcon
#

cuz telephony uses different codecs for different countries. The USA uses mu-law telephony while Europe uses A-law

trail jetty
opaque falcon
#

and it will fuck up if the person runs it on the wrong one

trail jetty
opaque falcon
#

Ah right. Duh ofc

vagrant hare
opaque falcon
#

I'll fix that shortly

trail jetty
#

or yk, do it yourself

opaque falcon
#

Worst comes to worst... there is probably a CSV file somewhere

trail jetty
vagrant hare
trail jetty
#

hm

vagrant hare
#

yeah it should be context reminder

trail jetty
#

yea

#

action forces have been noted to not work well if multiple action forces come in

opaque falcon
#

also, does vedal have extra voice filters 💀 cuz her normal voice won't really work for this

#

so some kind of grandma neuro filter might be required

#

It would be hilarious to hear

trail jetty
#

so definitely yes

vagrant hare
#

If he is going after v3 then maybe so

opaque falcon
#

ig that's not something we worry about and leave it to him ig?

trail jetty
#

yea

vagrant hare
#

yep

trail jetty
#

I think also vedal might have the ability to hook to his API server such that when certain context comes in, then it fires an event which could change her voice

#

I can't confirm he has that though

opaque falcon
#

The voice would have to change on command anyways

trail jetty
#

because if he does have that ability then he hasn't shown it ever

opaque falcon
#

because during mutes, neuro should use her normal voice

vagrant hare
trail jetty
vagrant hare
#

luckly neuro's LLM is paused during tool calls so it isn't bad

opaque falcon
trail jetty
#

when she executes an action

#

*a result packet

opaque falcon
#

for the api prolly yeah

vagrant hare
#

yep, otherwise neuro won't continue until she gets it

trail jetty
#

^

#

that's how llm tool calling usually works

#

pretty sure

#

I think I'm living too much in abnormal API use cases

opaque falcon
#

Well it DEF wouldn't be async and it still needs the result of the tool call since llms are autoregressive anyways

vagrant hare
trail jetty
#

if its async, you NEED to return an action result ASAP (usually after validating parameters)

opaque falcon
#

I just went on a minor tangent mb

trail jetty
#

asynchronous action results need to be done with 1. a successful action result and 2. a context describing whether or not the action failed

#

I really should get around to publishing the unofficial Neuro API docs I wrote a while ago

#

anyways

opaque falcon
#

That's pretty cool

trail jetty
#

minor tangent aside

opaque falcon
#

So yeah do I need to implement the API or not?

#

I'm assuming so atp but idk

trail jetty
#

I just realised I cloned the repository directly instead of forking it first

vagrant hare
#

I'd say it isn't worse than whatever your doing for the bot, same endpoints just looks diffrent?

trail jetty
trail jetty
#

although maybe it could also be easier to just directly integrate with API?

vagrant hare
#

I see benifits of both methods, but I'm leaning to API with override

opaque falcon
#

Unless vedal says himself I will just maintain both versions

trail jetty
#

cool

#

time to integrate API

opaque falcon
#

I might need some slight help with the neuro api but i'll figure it out eventually

#

wait are u writing it rn?

trail jetty
opaque falcon
#

💀

trail jetty
opaque falcon
#

Oh thanks a lot. I'll shout you out on the readme

trail jetty
#

I cannot believe that you committed .env btw

vagrant hare
opaque falcon
trail jetty
opaque falcon
#

mb

#

are u going to PR that or should I fix it rn?

trail jetty
#

I'll PR that too

opaque falcon
#

alr thanks

#

Problem is I wanted it to be "noob friendly" so I tried to make it as simple as possible

trail jetty
#

well if we want to make it vedal-friendly the API would be more helpful, but I can see why

#

you did the current method

opaque falcon
#

yeah

trail jetty
#

what python version are you using btw

opaque falcon
#

3.13

trail jetty
#

i'm using 3.13 rn

#

oh cool

opaque falcon
#

I hate it but yeah

#

so many libraries don't support it yet

#

also I will push the installer helper script soon

#

single command to install

trail jetty
misty surge
# trail jetty It's probably better to control via a separate bot anyways

I do agree with seperate bot, part of keeping things secure for vedal is limiting the information you have access to, currently you're getting the bot token for I assume neuro, which is already a no-go

also you're connected to several other services (ngrok and twilio) that vedal cannot check the code for, this a security concern, vedal would have to run the program on another pc/vm to be able to keep the program secure and away from being able to read anything on vedals pc

I cannot see what ngrok is used for in the code, so if you were at least able to remove that, that would be nice, twilio is probably fine since its required to make calls, but if it could go though a discord bot as another user and just send audio from the call to be able to talk with the scammer that would be best, with the current api that's the best you could do

trail jetty
opaque falcon
#

Bot can be run on any PC, even in a sandboxed vm. ngrok isn't really needed depending on ur network config either

trail jetty
#

also pretty sure vedal would run this on a separate VM anyways, he has said he wants that for all neuro integrations

#

he just can't figure it out

opaque falcon
#

For this I could cook up a simple docker container

trail jetty
#

you don't even have to

opaque falcon
#

I just need to know what he wants

#

it's easy either way tbh

#

it's not a very complex program thankfully

trail jetty
#

I wrote a whole page on sandboxing NeuroPilot (the Visual Studio Code extension) with VirtualBox and I realised it can technically work with any integration

#

just depends on how tolerant he is on slightly increased l-word

opaque falcon
#

we are simply bottlenecked by telephony

#

and it's like 800ms

trail jetty
opaque falcon
#

I mean if Vedal single-handedly rewrites the telephone lines for neuro alone... I think i'd just be speechless

#

cuz we do like 0 processing basically

#

just old ass telephone lines being old

trail jetty
#

yeah it's just telephone distance and quality

#

anyways

#

are we keeping the discord bot for api integration or nah?

opaque falcon
#

yeah we will keep both implementations

trail jetty
#

no I meant

#

are we still middlemanning via discord bot in the neuro api mode

#

or are we getting rid of that entirely and just conecting straight to twilio

opaque falcon
#

both

trail jetty
#

no I meant like

opaque falcon
#

which ever vedal decides he prefers we will keep

trail jetty
#

when you start the program in neuro api mode

#

is the program going to connect to a discord bot, or straight to twilio

opaque falcon
#

oh, twilio

#

so long as she can still perform the actions like muting and hanging up

trail jetty
#

also, did I forget to install something

opaque falcon
#

prolly pynacl

#

oh hm lemme see

misty surge
#

oh also one bug report before I go to sleep your links on your readme aren't working

trail jetty
#

I'm going to fix them in the same PR

opaque falcon
#

smh

trail jetty
#

it was annoying the fuck out of me

opaque falcon
#

discord.py>=2.3.0
discord-ext-voice-recv>=0.2.1
twilio>=8.10.0
flask>=2.3.0
flask-sock>=0.6.0
python-dotenv>=1.0.0
rich>=13.7.0
pyfiglet>=1.0.2
requests>=2.31.0

#

In ur version there isn't the requirements.txt

#

cuz I didn't push it yet

trail jetty
#

some of those I can tell immediately you need to bump the version of

#

it's fine

opaque falcon
#

I just froze lol

#

maybe update then refreeze

#

my venv is pretty fucked

trail jetty
#

I'm just going to create a pyproject.toml and use uv

opaque falcon
#

dw about it 😂

#

alr alr

#

also... we are almost at the same number of messages are the Neuro SDK 👀

#

and I literally just made this thread 😬

opaque falcon
#

Understandably

#

It just works

trail jetty
#

game dev general is 2.4k messages

opaque falcon
#

surely

#

that's a somewhat old channel

vagrant hare
trail jetty
#

There's no sort by messages but I can tell that game dev general is the channel with the least messages that exceeds 2.4k

vagrant hare
#

it was recently because of a game jam, but that's about it

opaque falcon
#

also I fixed the race condition I think

#

I misplaced my phone so I can't fully test it rn

trail jetty
#

hell I'm pretty sure gdg is the thread with the least messages with >2k msgs

trail jetty
opaque falcon
#

blocked?

vagrant hare
opaque falcon
#

Most funny idea thread though 😎

trail jetty
vagrant hare
#

neurolings is 4th at just over 1k

opaque falcon
vagrant hare
#

No one is beating neurobot

trail jetty
#

no fucking chance

#

those guys are dedicated

#

@opaque falcon #1

#

pr made

opaque falcon
#

Lemme review

trail jetty
#

not the api unfortunately but I had to do that first

#

did I forget something?

opaque falcon
#

Nope

#

3.13 is probably too strict for requirement but it's okay for now

#

also it's merged now

#

I actually might just push my new changes rn

#

I think it's good

#

I'm pretty sure it works on 3.8+

#

but I don't feel like testing that rn

trail jetty
opaque falcon
#

Yeah works for me

trail jetty
#

won't be able to get the support done right now (I have a commitment to attend) but by saturday/sunday I should get it working in theory

opaque falcon
#

Sounds great. Thanks again

#

That works out cuz I'll probably finish my side of it before then so you can base your code on the fully working version

trail jetty
#

ok

trail jetty
vagrant hare
opaque falcon
#

TODO:

  • Fix PERSONAL_PHONE_NUMBER name
  • Add correct handling for if the scammer hangs up (Very common)
  • Add actions via API (Unmute, hang up, call)
  • Add handling if the scammer does not pick up
  • Make launcher less verbose (give -v flag)
  • Disconnect bot if nobody is in VC
  • Add auto-audio codec changing based on country
  • NeuroAPI integration just as a whole
  • Add relevant README image
  • Learn to program
  • Fix versioning
  • Get Vedal's opinion somehow someway (Probably not gunna happen)
trail jetty
opaque falcon
#

Oh @trail jetty is there a way to actually test these things with the Neuro API? I'm assuming not. It's not I can actually run neuro or that Neuro is constantly running a server so integration can't be verified until Vedal does it himself

opaque falcon
trail jetty
opaque falcon
#

Ah nice i'll give it a try and maybe even send a demo video here

opaque falcon
#

Hm yeah none of these include any of the audio components. No STT and no TTS. Seems the API is entirely meant for silent gameplay not speaker interactions. Which I suppose could work would be using the api for LLM triggered muting, joining, and disconnecting. But that's a lot of baggage to hold just for such a small feature.

opaque falcon
#

Also, while this not matter too much, it's probably better to use USA phone numbers for this because the USA uses mu-law which is much better for high pitched voices. It's much harder to find scam centers using US numbers but even international calls are okay because Neuro's voice would be encoded using mu-law (which is important for high-pitched speakers) while the scammer's voice is in A-law which doesn't play well with super high-pitched speakers but that won't matter since they will not have such a high voice

#

mulaw is the USA and alaw is the international standard. I think mulaw sounds a lot better

#

I know Vedal really doesn't make the L word though and international calls can have a delay of like 150ms so it really just depends if Neuro can be heard on Europian phone lines.

It's just something worth experimenting with

trail jetty
opaque falcon
#

Well I am using discord

trail jetty
#

well in discord bot mode you don't really need to care then

#

*sorta

#

with API though yeah you do need to do STT since we are bypassing discord entirely

opaque falcon
#

Hmm is using the API smart then?

#

Cuz that seems very janky to run a whole extra STT

trail jetty
#

so is running a discord bot that relays everything anyways

#

albeit with api neuro can mute/unmute if she "feels uncomfortable" or smth like that

#

or if she wants to talk to chat

#

basically, point is, both ways are jank, the question is do you want to give neuro autonomy in muting or not

trail jetty
opaque falcon
#

You're not wrong

#

I wish I knew what Vedal wanted cuz cuz now I have to juggle a few implementations

#

Although not all integrations used the API

trail jetty
#

wdym

#

if it doesn't use the API it's not an integration

#

unless you mean like twilio end or smth idk

opaque falcon
#

Lemme send the thread

#

oh nvm it does something different

#

but like still basically the api

#

The options I have here are:

  • Pure discord bot (She just kicks out the bot and reconnects it as a normal user). Very jank. Won't do unless the API simply doesn't work (which is probably does)

  • Discord bot for audio and api for muting/joining/kicking. Keeps it simple.

  • Pure API with additional STT. No discord bot just twililo api audio streaming audio via ffmpeg probably to whisper and voice detection models and API to stream text over API into context. Program rules for when to update neuro's context and how. Determine if Neuro will just speak or send text that we use a custom TTS to convert into audio that is directly streamed over ffmpeg to encode audio into correct telephony format back into twilio for to the call. I really don't want to have to do this

trail jetty
#

the last one I know for sure there was a repository for discord audio speech-to-text

#

here somewhere

opaque falcon
#

That's a bit more convenient

#

but like that kinda defeats the purpose

#

why would I do STT on discord audio if I could just use the discord audio by itself

trail jetty
#

neuro cannot hear it

opaque falcon
#

Doesn't Neuro invite her friends to vc?

trail jetty
trail jetty
opaque falcon
#

then Neuro automatically uses STT

#

so why would I implement that myself?

#

it would 100% be worse

trail jetty
#

idk, thought you were for some reason

#

also if you want to keep the discord bot method I would probably use the 2nd solution

opaque falcon
#

Idk if 2nd or 3rd is what Vedal wants

#

I mean it's more likely that 3rd is better for him but it's so much more of a pain

trail jetty
#

I mean 3rd gives Neuro access to phone calls directly, which probably would work weirdly

#

instead of proxying via discord which already has a solution

#

Man, I just wish Vedal checks the projects channel

opaque falcon
#

ikr

#

Also I think 3rd solution has more l||atency||

trail jetty
#

I don't really see why?

opaque falcon
#

Because realistically, Vedal probably optimized neuro's STT

#

for latency

#

and it's better tested than guessing how it works and making a new one

#

plus neuro just works better in calls than piping text into her context window

#

since it just suddenly appears and doesn't come with "speech metadata" like if the speaker is still speaking and whatnot

trail jetty
opaque falcon
#

Even with all this said... I don't doubt Vedal might still want this

#

so I think i'll have to prepare it anyways

opaque falcon
trail jetty
#

yea

opaque falcon
#

and since there is literally 0 documentation on how it works (Vedal doesn't want to give too many secrets away) it's guarenteed to be bad since it's shooting in the dark

trail jetty
#

sure then

opaque falcon
#

So I meannnnnnn I can still write it but it's going to be bad

#

hence writing both and letting Vedal choose

#

aghaghaghaghag I just wish Vedal checked projects threads

trail jetty
#

at the very least the ones tagged neuro integration for stream content vedalCorpa

opaque falcon
#

wait wdym

#

oh yeah

#

idk I feel like he's probably looked in here but hasn't had the time to write a reply yet

#

but that could just be coping

trail jetty
opaque falcon
#

Vedal needs a bigger team 😢

#

He is very clearly exhausted and can't manage all this even with more Neuro automation than before

#

surely there is an additional filter

trail jetty
opaque falcon
#

no way

#

But tbf he did actually implement a lot of these integrations

#

I think at least

#

I feel like he did the Attorney one

#

but idk if he did that himself

trail jetty
#

none of them, though balatro and outer wilds are planned supposedly

opaque falcon
#

oh...

trail jetty
#

this is the current list of active neuro integration threads

trail jetty
opaque falcon
#

Court stream

#

I could be mistaken

#

might be something different

trail jetty
#

court stream is custom

#

it doesn't use aa at all

opaque falcon
#

wow

trail jetty
#

in fact iirc it's entirely roleplay

opaque falcon
#

Looks like our odds are near 0 😭

trail jetty
#

yea

#

abnormal game API use cases don't get enough attention since they aren't games

#

you have to be (almost) very creative or targeting a use case mentioned on stream

opaque falcon
#

I mean I would argue that this could be more popular of a stream than playing a game like buckshot roulette or something. There is quite a lot of a crowd who love scambaiting videos/streams.

#

What do you reccomend doing for the best chances of getting this stream out?

trail jetty
#

also doesn't require knowledge of the game beforehand

#

and I think vedal has wanted to do a collab with kitboga before

trail jetty
#

but uh, definitely not 10% either

trail jetty
#

except keep this thread active with updates and stuff

opaque falcon
#

I might be able to pull some strings

trail jetty
#

Also just so you know, the neuro integration tag is meant for using the Game API

opaque falcon
#

Yeah we are still using it

trail jetty
#

tho since it's being implemented I guess you're alright now

opaque falcon
#

for muting allbiet but yeah

#

and joining and stuff

trail jetty
opaque falcon
#

well it's not implemented yet but ik what u mean

trail jetty
#

ok so

#

we're reintroducing the discord bot as the middlebot with the game api right

opaque falcon
#

both with discord bot and without

#

two versions. Version A will use discord bot for audio and API for actions

#

Version B is pure API and custom STT implementation

trail jetty
#

right so what I'm hearing is I need to make 2 entrypoints

#

for the API

#

I think

opaque falcon
#

Two repos

trail jetty
#

no not two repos

#

two entrypoints

opaque falcon
#

or more likely folders

#

sure yeah

trail jetty
opaque falcon
#

yes

#

exactly

trail jetty
#

I'll implement the neuro side of the API version, can you later implement the discord/twilio side

#

I have no expreience with twilio api

#

and don't want to f it

opaque falcon
#

yeah ofc

trail jetty
#

and I need to take a break from discord bot development

opaque falcon
#

yeah bots are annoying sometimes

#

I have 0 experience with neuro api so this works out

#

just let me know what format to expect to send/recieve data from

trail jetty
#

JSON

#

with parameter validation by jsonschema

opaque falcon
#

I mean I need audio

#

I can send text over JSON

#

from the STT

#

unless we roll our own TTS as well

trail jetty
#

I didn't think of that

opaque falcon
#

I mentioned ffmpeg streams

#

but this is actually non trivial

trail jetty
#

Neuro has to send and receive everything in text

opaque falcon
#

idrk how to do it 😂

trail jetty
#

so do with that what you will

opaque falcon
#

fuck. so we have to do our own TTS

#

which is super weird

trail jetty
#

unless we somehow get ahold of a new audio module that solves this for us

opaque falcon
#

because her movements won't line up

#

and she can speak two audios at a time

trail jetty
opaque falcon
#

that's annoying ahhhh

#

reason 99999 why pure api is pure pain

#

surely we can figure it out glueless

trail jetty
#

(@)vedal audio module when NeuroClueless

opaque falcon
#

Yeah idek if it's possible to do this purely over the api then

#

I mean it's possible

trail jetty
#

hence why just stick with 2

opaque falcon
#

but it would require a lot more work

trail jetty
opaque falcon
#

I can explain it but it's really rough

trail jetty
opaque falcon
#

We have to remove her model from the stream entirely and remove her speaking and make her basically a glorified text LLM that takes the text from STT and then we use our own TTS and speaking logic.

(We'd basically be making Neuro entirely from scratch)

#

Put like a still frame of Neuro staring

#

that's so creepy and bad

#

I hope Vedal can accept working with a discord bot...

trail jetty
opaque falcon
#

I mean a stream without Neuro's movement seems kinda spooky

trail jetty
#

also the model doesnt need to be removed, wtf?

opaque falcon
#

it does...so she doesn't start talking to chat while our TTS is running

#

double speaking

#

it's a weird edge case

trail jetty
#

it would happen either way

opaque falcon
#

not if we removed her entirely lol

trail jetty
#

???

#

oh I think I see the issue

opaque falcon
#

We only summon neuro once the STT is done and feed it into the context

#

it's really really really scuffed

#

like 🤢

#

she would legit have to walk into a room the camera can't see maybe with a sign on the door saying "DO NOT DISTURB. SCAMBAITING IN PROGRESS" play a sound of a door slamming and show the scene of the door and put the audio of our own TTS over

#

it's super weird

#

She starts stream as normal and talks about how she wants to mess with some scammers then she walks through the door and her model vanishes behind the door

trail jetty
#

ngl it sounds funny neurOMEGALUL

#

but wouldn't the problem persist either way?

opaque falcon
#

Not if we're smart about it. We would have to completely kill the neuro program running on Vedal's computer and turn it on with 0 context from the outside world and mute her speech and disable subtitles then have her text engine give her replies then we take the text from the STT model which is listening to the twilio ffmpeg audio stream and put it through our own TTS engine to play the audio to twilio and to the stream

#

if she recieves no stimulus she can't do anything

trail jetty
#

ok so basically beg vedal to give sysadmin access, got it

#

yeah no let's just go with discord bot if you don't like the taste of constant torture

opaque falcon
#

that's why it's a horrible idea but technically possible

#

Maybe I can think of a smarter idea later but that's all I can think of atm

#

surely a discord bot isn't too much to ask of Vedal

trail jetty
#

yea

opaque falcon
#

Well in that case i'm already done pretty much

#

the bot works

#

I just need to implement the muting and joining via api

#

Also... I had 0 clue kitboga was already working with Vedal 😭

#

I mighta wasted my entire time and they might already have this done

opaque falcon
#

Well at the very least it seems a lot less likely that our code would be used

#

if he decided he wants to do this stream he would work with kitboga since it would get him more viewers

#

I mean i'm still going to maintain the repo but that kinda stinks on our end

trail jetty
#

ig so

#

I was thinking you could also repurpose this into a scam toolkit but again, requires a bit of work

#

and also access to stuff we don't have

#

and will likely never have

trail jetty
opaque falcon
#

I think the best bet is try to pull some strings and get in touch with kitboga

#

I've worked with youtubers before

#

but it's very difficult to get in contact with them and have them actually talk to you

trail jetty
#

well, let me know of your progress

opaque falcon
trail jetty
#

you can connect the rest

opaque falcon
#

Yeah just get her input for commands so I can run the mute on the discord side

#

actually idk how u get her text that's hard

#

so basically if she wants to run an action to mute, unmute, join, or disconnect then she can

#

which idk if that's done over a custom protocal or text

#

u know the API better than me

trail jetty
#

do you know what llm tool calling is?

trail jetty
opaque falcon
#

Not all LLMs do it the same way

#

I assume Neuro uses an MCP server

#

but somethings like deepseek use tags

#

If you can write a callback function for each action then I can add code in that function to execute the actual actions neuro is trying to do

#

but if it's text (like <mute>) then you'd need to get and parse the text

#

hopefully it's MCP

#

actually it's def MCP cuz neuro's TTS would say the command if it wasn't

trail jetty
#

most mcp servers are http but neuro is websocket

opaque falcon
#

that's weird

trail jetty
#

which means neuro can call a command whenever she wants

#

basically

opaque falcon
#

so you just listen on a port and trigger a callback whenever she sends the packet

trail jetty
#

yes

opaque falcon
#

I assume you know how to do that

trail jetty
#

quite possibly

#

ykw I should probably comment the code so you know what you're looking at once I'm done

opaque falcon
trail jetty
#

I want to finish the neuro-sdk-docs repo so I can just point to the thing and say "look" but honestly there's a fair amount of ways to implement it in python that I can't be bothered to write them all

trail jetty
opaque falcon
#

that's perfect

#

just put some comments in the places where I write the code to actually do the command

#

but that code alone is probably good enough

trail jetty
#

There are a couple other nuances as well though

opaque falcon
#

hm

trail jetty
#

eh it's fine, I'll comment them in the actual code

opaque falcon
#

thanks

trail jetty
#

@opaque falcon finally decided to put this up as a website (both for you and because I need to put it somewhere at some point to see if people will hate me gauge interest), I have info on handling actions from Neuro on this page.
If you poke around the WebSocket protocol section, you should be able to understand more or less how the API works.

If you're wanting to jump to the Python SDK part, uhhh don't look quite yet :)

#

aww no embeds :(

opaque falcon
#

That's pretty cool

#

Although I am most certainly going to use the Python API so I'll wait for that to be completed

trail jetty
#

yeah well that wont be done for a while imo

opaque falcon
#

Alright

opaque falcon
#

I am away from home right now so I cannot work on this as much as I would like to. However I think I finally fixed the race condition for real this time glueless I'll push out an patch before I sleep. I'll maybe push a UI update too

#

discord threading is a bit strange but I think it should work now

#

I'm also just stupid because I was running ngrok as a subprocess instead of using the API which is just... bruh 🤦

#

I can fix the versions later by creating a new venv instead of using my very broken and outdated system env

#

also, I made a video but messed up the recording and recorded through my PC mic instead of desktop audio and I can't record here rn since I am on a business trip rn but when I get back I can record it again 👍

trail jetty
#

Oh, might have forgotton to make some updates to make the Neuro API shell code understandable...

opaque falcon
#

That's alright

opaque falcon
#

@trail jetty Sorry for the ping. Should I start working on the Neuro API Implementation? The backend has been done for a solid minute. I just don't have much else to work on

trail jetty
#

sure, give me a sec

#

I was gonna start pushing the shell code for today

#

but I ended up being at a CTF competition

#

(going on for another 8 hours + travel time)

#

I'm just quickly making some changes before committing and PRing the shell code

opaque falcon
#

woah that's crazy

#

Also wdym shell code? Did you make the implementation in bash or something? lol

trail jetty
opaque falcon
#

oh the shell of the code I see

#

thanks!

trail jetty
#

@opaque falcon I opened a PR to incorporate that shell code, I also put implementation notes in the PR description

opaque falcon
#

perfect I'll check it out in a sec

trail jetty
#

that implementation should hopefully be sufficient

opaque falcon
#

Do you know a way I can test it?

#

like with a local ollama model or sumthin?

trail jetty
#

It doesn't have any code

opaque falcon
#

no ik

#

but when I actually fill it in

trail jetty
#

oh also you don't need a local ollama model to test

opaque falcon
#

it would be cool in a video tho lol

trail jetty
opaque falcon
#

a GUI seems weird tho\

trail jetty
#

it's also able to be acted via API

#

(as in, it has an API)

#

also it's for testing to ensure your implementations actually work first

trail jetty
opaque falcon
#

How does that API work?

#

like what exactly am I sending/recieving

trail jetty
#

you're sending websocket packets from your program

#

which is then received/acted upon by the api server (tony, gary, jippity, randy, etc...)

opaque falcon
#

and ig tony just always uses the tool or something?

trail jetty
#

tony allows you to choose which specific tool you want to execute at any time

opaque falcon
#

oh so it's manual

trail jetty
#

with an option for auto-answering forced actions

trail jetty
opaque falcon
#

like by a human in the GUI

#

ohhh

trail jetty
#

yea

#

altho if you are able to use an AI locally, gary is almost entirely an upgrade

opaque falcon
#

yeah I got a pretty nice GPU

#

ig I just have to prompt it so that it uses the tool correctly

#

cuz idk if ollama can do native tool calls

#

idrk

trail jetty
#

yea

#

there's some tips on using AI models via gary on its readme

opaque falcon
#

oh perfect

#

also I'm reading the PR rn

trail jetty
#

👍

opaque falcon
#

Am I dumb or does this seem off

trail jetty
opaque falcon
#

3 weeks yeah lol

trail jetty
#

it was meant to add a way to spawn it in non-API or API modes

#

obviously we've changed it since then

#

so

opaque falcon
#

alr alr

#

woah this is really cool

#

way more built out than raw MCP 😂

trail jetty
#

tell me about it

#

the idea is that because Neuro can execute something at any time we needed something more than just plain MCP

#

and also it makes more sense for the games to be the websocket client instead of server

opaque falcon
#

right ofc

trail jetty
#

less configuration for same benefit

opaque falcon
#

alr merged

trail jetty
#

👍

opaque falcon
#

thanks again 🎉

trail jetty
#

🎉

opaque falcon
#

Oh do you know how to install abc?

opaque falcon
opaque falcon
#

@trail jetty I have the muting part working perfectly but I just don't know what ABC is and how to connect the code to the network to accept outside requests for commands. Like I don't know how to connect Gary to the code

#

and I also have no idea how what ABC is

trail jetty
#

currently in the middle of the CTF, you'll need to wait

opaque falcon
#

Oh good luck!

trail jetty
#

finished ctf

trail jetty
opaque falcon
#

Oh I had no idea

opaque falcon
#

Wait. I have a crazy idea. What if we set up Evil to be a scammer and Neuro was scambaiting/trolling evil? Neuro wouldn't even realize it would be Evil either since it's not like she can recognize based on the voice anyways. Chat would go WILD

opaque falcon
#

Native API integration is semi-working

opaque falcon
#

yeah it all seems to work just with some weird quirks

#

Like I am using manual command sending

#

over the websocket

#

and when it does "hangup" it just disconnects the bot which leaves the scammer sitting in the call with no audio 😂
Accidental but might keep that as a feature 💀

opaque falcon
#

Too big for discord but here is the log of what the AI model is doing

#

I know I'm parsing in a dumb way but for now this works

#

like if she says "join" just in her daily dialog it'll fuck everything up

#

And when she muted it got added to the context and she said to chat "Okay, the call is now muted. This prevents the victim from hearing my responses and ensures a more n..."

#

this is just a base ollama model though so ofc it doesn't sound like her

#

I just need to differentiate the scammer from somebody like Vedal so she doesn't get confused

#

When joining the call she said "I'm initiating a call to the target phone number. My objective is to establish a connection and begi..."

#

And when she unmuted she said "Alright, I’ve just unmuted the outbound audio. Now I can respond directly to the victim. I’m going t..."

opaque falcon
#

Tony integration seems to be broken...

opaque falcon
opaque falcon
#

seems everything is fully working finally. I just need to finally implement the shutdown mechanisms and make the installer a little simpler to run

opaque falcon
#

Okay it's a little scuffed because I am adding messages to the context about the action that was taken as if it was successful before the result comes back. Butttttttt surely it's fine cuz a failure is added to the context anyways glueless

opaque falcon
#

Just accidently deleted all of the files. I don't have a backup... 😁 fml

#

The code is lowkey kinda ass rn but I will fix it soon enough

#

It should work

#

it's running on hopes and prayers but I think it should work with a little tinkering

trail jetty
opaque falcon
#

Tony

trail jetty
#

you shouldn't need to integrate tony

#

you just need to use the neuro api package

#

for tony all you need to do is start it in a terminal

#

and then start scamsama

opaque falcon
#

I lowkey mighta implemented the api myself by accident

trail jetty
#

...

#

I'll go take a look at the code later

opaque falcon
#

I didn't use the package

trail jetty
#

well the thing is

#

tony is supposed to act as a server, not as a client

#

or in neuro api terms, tony is trying to be like neuro

#

and not the program that connects to neuro

#

so tony's api won't have anything to do with registering actions to another server

opaque falcon
#

ohh

trail jetty
#

because tony is supposed to be the server

opaque falcon
#

well isn't that what we want

trail jetty
#

that's why there's a package called neuro_api even tho tony is there

opaque falcon
#

the game registers the commands that neuro can use

#

tony acts like Neuro so that's good no?

trail jetty
opaque falcon
#

Tony isn't in the code really

trail jetty
#

actually wait we might not even be on the same page

opaque falcon
#

My code just binds to ws://localhost:8000

#

and Neuro acts as the server

trail jetty
#

oh, then that's good

#

you're not really integrating tony then

opaque falcon
#

I figured... phew

#

yeah nah Tony I didn't even touch

#

just stock config

trail jetty
#

wait

opaque falcon
#

Tony isn't even in the repo

#

just used for testing

trail jetty
#

so you implemented the api... yourself?

opaque falcon
#

I mean it was easy lol

#

just a websocket with a schema

trail jetty
#

LMFAOOOOOOOOOOOO

opaque falcon
#

had to read the spec

#

took 6 hours but wtv

trail jetty
#

yea but the point of me using the api lib was because it was easier to do that way

#

oh well

opaque falcon
#

I mean it works SMILE

trail jetty
#

if it works, it works

opaque falcon
#

at least on my machine 😬

trail jetty
#

tho when api v2 drops get ready to have to change some stuff

opaque falcon
#

I haven't fully stress tested it in different environments

opaque falcon
#

Surely that's in a longggg time

#

like a year later

#

The code was 950 lines at one point but I removed a lot of clutter and it's only like 650 or sumthin now

#

kinda forgot the API existed

#

I mean at least I understand it better now

trail jetty
opaque falcon
#

tbh I doubt my code works rn

#

cuz I deleted all my files by accidnet

#

and I kinda just guessed what things were

#

like I know for a fact that requirements.txt is completely broken and state.py is 90% probably broken too

#

I'll fix it up soon and maybe record a demo video somehow

#

Might play with an ollama model and some random crapy TTS and whisper to get a PoC and gain some more interest in the thread

trail jetty
#

hmm

opaque falcon
#

I can edit some 60 second clip that we can share around the server

trail jetty
#

I know for sure that:

  1. I'm not sending ws messages incorrectly
  2. if there's a problem it's 90% because I forgot to stringify my objects
  3. it's battle tested
opaque falcon
#

Counterpoint:

  1. I forgar
  2. Was funny
  3. Learned a bit
  4. The API part works I think
  5. I get better logs... probably
#

THe logging is pretty sick actually

trail jetty
#

also tony can log the networking parts quite easily

opaque falcon
#

tony's logs were pretty limited iirc

#

also it took AGES to figure out how to push things to context

#

I mean it was a skill issue on my end

#

like it was just a little confusing even after reading the spec

trail jetty
opaque falcon
#

I did

trail jetty
#

I thought you said you were using stock settings tho?

opaque falcon
#

Well also tbf my program was doign a lot more

opaque falcon
trail jetty
#

stock settings by default only logs info level

opaque falcon
#

yeah it was on info

#

but I changed to debug

trail jetty
#

huh

opaque falcon
#

cuz context sending was being annoying

trail jetty
#

I mean honestly SDK-level debugging isn't that bad of an idea

#

I'll toy with it for JS/TS SDK at least

#

though that will not be for a little bit since today's neuropilot stream was, uh, not perfect?

opaque falcon
#

It was quite the stream 😂

trail jetty
#

yea

opaque falcon
#

Also aren't you involved in basically every neuro project here 💀 ?

opaque falcon
#

Must be entertaining watching me struggle with the most simple things 😂

trail jetty
#

watching the development and progress is always interesting

trail jetty
#

though I was like that when I first started and there was no python sdk

opaque falcon
#

I enjoyed it too dw

#

I mean it was a little confusing at first since I didn't understand what the SDK was meant for at first

#

and I only first heard about neurosama a few months ago

opaque falcon
#

Okay I am back 😂

#

I honestly don't remember too well where I left off

#

but I think it was working

#

woah bluecat made a PR

Thank you @misty surge (Also super cool repo you made!) I will review it rn

misty surge
#

huh?

#

that's not me

#

my github is blueturbo47

opaque falcon
#

oh 💀 CoolCat

#

too many cats 💀 💀 💀

misty surge
#

yeah I haven't actually uploaded much code to github in a while

opaque falcon
#

how did he even find my repo hahaha

#

he isn't in this thread I don't think unless bros been lurkin

opaque falcon
misty surge
#

I have plans on making an archipelago for a game once the actual modding api for it comes out, but it'll probably be a bit

misty surge
opaque falcon
#

That's amazing

#

cuz I really didn't feel like learning the whole NeuroAPI...

#

but I implemented it myselfd

#

but the Python API is 999% better than my "implementation"

opaque falcon
#

@devout quarry I reviewed the PR

#

thanks for submitting it

devout quarry
#

@opaque falcon Just make responses for your review comments

devout quarry
opaque falcon
#

oh cuz you import voice somehwhere else I get it

opaque falcon
opaque falcon
#

@devout quarry How are you intending the user to provide the credentials to twilio and discord api?

trail jetty
devout quarry
opaque falcon
#

That explains the absolute versioning commit

devout quarry
#

Hopefully added fix in my pull request for multiple instances running at once issue

opaque falcon
#

Sweet! I'll test when I get back home. I'll see about pushing a test suite as well

#

Thanks!

devout quarry
#

Using new neuro_api.server module from recently released v2.1.0 might be helpful, but could also just do mocks for the write_websocket or whatever it was

opaque falcon
#

Still experiencing some very strange behavior

#

The code is failing to detect the discord token despite being in the .env file...

#

and ngrok occasionally just doesn't load...

#

trying to pinpoint the problem but it's tricky

#

because it isn't acting deterministically. Each run has a different error when nothing is changed

#

Error: Discord Bot Token Not Set. Please set DISCORD_BOT_TOKEN.

#

Meanwhile it is 100% inside the .env file

#

however, perhaps since code directories have shifted the program looks for .env in the local directory which works for launcher.py but not inside src/neuro_sama

#

still wouldn't explain the random NGROK errors tho

#

Tony may have worked though:

[16:22:09] [Info] Running version 1.6.2
[16:22:09] [Info] Starting websocket server on ws://localhost:8000.
[16:35:09] [Info] Started game "ScamSama"
[16:35:09] [Warning] Action schema is null: mute
[16:35:09] [Warning] Action schema is null: unmute
[16:35:09] [Warning] Action schema is null: join
[16:35:09] [Warning] Action schema is null: hangup
[16:35:09] [Info] Actions registered: mute, unmute, join, hangup
#

Not sure why the schema is null though

#

It was indeed the .env directory problem btw

#

the NGROK, null schema, and now a twilio error is still unaccounted for though

#
HTTP Error Your request was:

GET /Accounts/REDACTED/Calls/-1.json

Twilio returned the following information:

Unable to fetch record: The requested resource /2010-04-01/Accounts/REDACTED/Calls/-1.json was not found

More information may be available here:

https://www.twilio.com/docs/errors/20404

Traceback (most recent call last):
  File "/home/anon/lab/ScamSama/src/scam_sama/discord_bot.py", line 487, in monitor_call_status
    call = twilio_client.calls(state.call_sid).fetch()
  File "/home/anon/venv/lib/python3.13/site-packages/twilio/rest/api/v2010/account/call/__init__.py", line 407, in fetch
    payload = self._version.fetch(method="GET", uri=self._uri, headers=headers)
  File "/home/anon/venv/lib/python3.13/site-packages/twilio/base/version.py", line 141, in fetch
    return self._parse_fetch(method, uri, response)
           ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/anon/venv/lib/python3.13/site-packages/twilio/base/version.py", line 112, in _parse_fetch
    raise self.exception(method, uri, response, "Unable to fetch record")
twilio.base.exceptions.TwilioRestException: 
HTTP Error Your request was:

GET /Accounts/REDACTED/Calls/-1.json

Twilio returned the following information:

Unable to fetch record: The requested resource /2010-04-01/Accounts/REDACTED/Calls/-1.json was not found

More information may be available here:

https://www.twilio.com/docs/errors/REDACTED
#

^^^^^^^^^^^
Twilio error

#

Not sure why I am getting 404. I assume you didn't touch the twilio code

#

it's accessing the API in an incorrect way that differs from my original implementation

#

I'm thinking the NGROK error is a race condition since I've encounter it before

#

and it isn't fully reproducible or deterministic

trail jetty
#

hmm

opaque falcon
#

Sorry for inactivity. Uni life gets in the way quite a bit. I'll push some updates today and hopefully get this fully fleshed out. After that, it'll be mostly "Get Vedal to notice this" kind of thing

trail jetty
opaque falcon
#

Damn 😭

trail jetty
#

#1350968830230396938 took months for noticing I think, and even longer for actually being featured on stream
this is for something that doesn't necessarily require tons of configuration and safeguards to get going btw (we already have our own safeguards)

opaque falcon
#

Well, do you have any suggestions on what we can do with it then?

#

Maybe starting smaller scale

trail jetty
#

Not really. best you can do is make some attention in the general community and keep working on it.

#

which both spreads awareness and meaningfully bumps your project thread

opaque falcon
#

Hm yeah. I'll work on that

opaque falcon
#

I am thinking that making a community video could be a great usecase for this project. It's unlikely it will be used on stream but we can record a video of it and potentially work with community animators to get a video out. We can post the github link in the description as well to get more developers on it and more attention. I don't think it's impossible to have vedal react to it if an animation is out. And even if he doesn't, at least we can have something to show for our efforts

#

The code already works fine except for some very rare race conditions

#

so it is more than good enough for a video. As the video is being worked on, more bug fixes can be made

#

If anybody is an artist/animator and wants to get involved, I highly encourage it

#

as well as software devs as well!

tranquil plank
opaque falcon
#

That's a good idea

#

[LOOKING FOR ARTISTS] ScammerSama (Scambaiting Neurosama implementation)

opaque falcon
#

Yeah it's hard to keep active here if the project has basically already been finished 😭

#

We could make some art for it but I am not an artist so it's a bit tricky

#

I guess we could actually make a video ourselves to maybe get the idea out there? I don't know if Vedal would care if we used the Neuro voice in a fan-made video that we could post

#

I mean it is just Ashley's (Azure TTS) voice pitched up so...

#

and if we specify that it is a fan-clip, he'd probably even appreciate it

#

So if by some crazy chance that you 🫵 are an artist and would be interested in making a fan-video then PLEASE ping me and we can actually make something that the public can digest

true glacier
opaque falcon
#

Hello! I am still alive 🫠

The code is fully completed for this project! However that doesn't mean we are done! We need to showcase our work!

I am thinking we could try to make a video of this with a Neuro voice (With a disclaimer that this is not actually neuro) and a base LLM just to be a proof of concept. Once we have the video we could try to get clipper channels to cover it to get some attention with it and then Vedal could see about using our integration

#

Beats just sitting in silence

opaque falcon
#

Oh perfect

true glacier
#

for the record, atm that thread is about one fan made news website and a couple of fan news youtube channels.