#[WIP] Neuro's Desktop (An integration for letting neuro use a computer)
1 messages · Page 2 of 1
i know whats happening, its not attached at all
From the trace function, it was going from Neuro Backend <> Nakurity Backend <> Intermediary
The Nakurity Client was just left out
I also removed the debug trace function for the commits
fuck i wanted to save my changes but looks like i backed up wrong files
to enable it, there's these debug configs
add line to it
what is this
it wasnt here yesterday, i remember it was none
i need it to be assigned to actual client
or attached.. idk how to correctly word it
now it doesnt start
why did i pull
i cant just have new pulls every 30 minutes which cancel everything i learned 
it just makes new problems, i have to fix this and not pull until i actually get something useful
its like climbing the ladder and jumping down midway
all progress lost
how do i fix this
do i need to install some new packages
And also comment these, I forgot
uhmmm wait
gimme a moment
There was a miracle fix function that was being used
looks like this miracle lost another 20 minutes for me 
You can either scroll to the top of main.py
and remove NakurityRequirement
Or pull the new changes
I'm so sorry!
I should've responded faster
i pulled
Okay it should work now
noo we need to separately solve something and not just add infinite amount of pulls and traces
I hope
no pulls until something is achieved
The pulls were giving you the latest code update with those stuff resolved, I hope
Neuro Backend receives something now
with those code updates
I used the miracle function with OpenAI to fix it lol
recursive fixing
worked, apparently
But we have a new issue
uhmmmm
The tests pass
but the Neuro Relay needs to be tested with a more complicated neuro integration
for me neuro backend just receves x2 less messages on connection now
nakurity backend says something failed
since it is kinda hard to tell if jippity would actually work correctly
Screenshot of it all?
https://github.com/EnterpriseScratchDev/neuro-api-jippity
Received something and made it work
For Randy you would need to manually respond to that forced action
That's good
it just doesnt send the other part
Yeah that was just for context for neuro
Try using jippity
Uhm, jippity said something is received
tho I need to fix that invaild error now
now retry queue in intermediary doesnt run at all
where do i put my test messages
intermediary has no access to client and cannot send anything to neuro
we must fix that first
Are you sure...?
│ │ │ │ │ │ │ │ │ │ ╭▶ retry_queue() intermediary.py:302 [ 5.455s]
│ │ │ │ │ │ │ │ │ │ │ args: self=<src.dev.nakurity.intermediary.Intermediary object at 0x0...
│ │ │ │ │ │ │ │ │ │ :boom: StopIteration: intermediary.py:302
│ │ │ │ │ │ │ │ │ │ │ → while True:
│ │ │ │ │ │ │ │ │ │ │ • locals: self=<src.dev.nakurity.intermediary.Intermediary object at 0x0...
│ │ │ │ │ │ │ │ │ │ │ → if not self.queue.empty():
│ │ │ │ │ │ │ │ │ │ │ • locals: self=<src.dev.nakurity.intermediary.Intermediary object at 0x0...
│ │ │ │ │ │ │ │ │ │ │ → await asyncio.sleep(5)
│ │ │ │ │ │ │ │ │ │ │ • locals: self=<src.dev.nakurity.intermediary.Intermediary object at 0x0...
│ │ │ │ │ │ │ │ │ │ ╰↩ return <Future pending> [ 5.479s]
tracer said something different
I'm not sure what you meant by this
Its 40 pages for a reason.
but I just look at the latest logs, its not that long anymore
It has worked
but that watcher is the fake neuro os integration no?
OOOH RIGHT
I made intermediary disconnect clients
I forgot about that
so server can access this function of intermediary
the problem is, nakurity client is not actually attatched to client
Nakurity Client connects to the Neuro Backend via a websocket
And I assume by client you mean the neuro backend, right?
neuro backend is server
Its a class that directly links Nakurity Client and Intermediary
nakurity client connects to neuro backend (to server)
I added it since from yesterday, it wasn't connected to eachother
this was done directly, not via websocekts or servers
no errors
Oh, yup I see the issue
is this init function ever called?
I forgot to add some methods for communications
Yes by __main__.py
this calls init?
Yes
and passes the Nakurity Client to it
client is nakurity client
init is basically the constructor for a class
I am fixing some methods
Neuro API expects something else
hmm right it all seems to be attached
https://neuro-api.readthedocs.io/en/latest/command.html#
There is this useful documentation for it
but this function isnt getting called for some reason
I'm cross-referencing the payload relays
It might not be called in the code
Lemme check
here how i call it
oooh
this prints
this doesnt print
yeah, that does the same thing as forward to neuro. but just more nested
lets compare
this is actual client who actually sends to neuro and works
this is object who is accessed from intermediary
they are the same
so why does it not work 
I have no idea either
But I am fixing something, and it seems to be working on my end. Tho I haven't commited it yet
Its the greatest thing ever, fixing something. But you don't know what you're fixing
I have no idea how or why my brain does this
hmmmmmm
I might have an idea
is it how its called?
Class Intermediary expects Class NakurityLink for self.nakurity_outbound_client
here how its called.
im getting a feeling that its because of functions having different types of arguments
its "dict" in the intermediary but "bytes" in the client
did you trying typing it with dict(payload) ?
that should force it to be a dict
atleast. when you send it to send_to_neuro func
do i leave client function like this
it seems like it works properly when its "bytes", other parts do it normally
neuro expects a dict
i want to do some loop in client to see if client is actually online and works
There is an example of this here:https://neuro-api.readthedocs.io/en/latest/command.html#complete-game-integration
maybe client dies midway and discards commands
will neuro print something on invalid command from client
i think i got it
we are calling function with different type of argument
its calling non-existent function or smth
what is dict?
is it a string?
A dict is a nested json object
Neuro expects something like this
this is for a direct implementation into the neuro API
could be helpful for the Nakurity Client
client expects bytes
we need to convert dict to bytes so client can send it
something something base64
it actually does this for its own transmissions
can we do something similar to dict?
encode it
let me try
hmm no effect
let me try make client accept dict and transform it himself
still doesnt work, function doesnt print
let me try send it from main..
fuck
what happened?
Are you okay?
Wait no- ignore that last part
pretend I never sent that
that's sadge
ok so i pull
Though I did fix something
Randy received something!
Idk what I did, I just asked the miracle function to do its thing :p
ok new pull killed everything
first i get this frozen numpy error
oh its 2 lines
what the hell 
who sends this? integration?
how does it go? integration->server->intermediary->client->neuro?
test harness acting as integration, yes!
yup!
Miracle function was a miracle
why does this miracle function not work for me
and what does it actually do?
where is this function located?
i mean, how can function make it go through server and intermediary and client
it was from the toolchain import you didn't have so it gave an error
i can see this function just sending it to everyone
Its basically vibecoding, cheating ik. But it worked.
It basically recursively called openai until it was fixed
what if it does this
Caused me 500 usd lol
it would be funny tho lol
polls integration, then sends result to client intermediary and server separately
so the whole structure of program is bypassed
because intermediary still has no way to send things to neuro
or client
I'll roll out a new commit you could check it at the github repo page if you wanna see
idk you can just describe it if you dont want to share it
I meant the fixed code
recursively calling openai with relevant context and errors, tries a fix until it is fixed
vibecoding, cheating. ik
I just pushed it
uhmmm, hopefully it should work on your end
Its 3am again, my sleep schedule is great!
Stash the changes
i dont get it when i open stash it only lets me discard or restore
is stash like "one slot"
it's in the branch section
can I see the screenshot?
i dont but i found the button
uhmmm git stash?
it seems it actually fixed it
intemediary now uses "send_event" of client which works
bye bye
I think, I may have accidentally pushed the debug tracer to the main branch, oopsies!
Now back to integrating neuro-relay into neuro-os
The issue has been fixed
We might need to test neuro-os more to see if actual relevant context is being passed to the neuro backend
this is where unit tests are supposed to help...
really?
I have been adding unit tests already
just haven't pushed them to main yet
I am also adding the remaining actions schema
if tests pass, doesn't mean it might help the LLM. Prompt engineering, from my experience. is not as easy as test passing, but okay!
it's supposed to test stuff that doesn't really change, you can never test with LLMs due to how they are designed to be non-deterministic
actually wait
how are you determining what is relevant context
yeah! that's what I meant with relevant context being actually passed to neuro!
Context like where her mouse is, what region she has selected with her mouse. What is currently fullscreened (i.e. a app like disocrd, etc)
yeah
but if you are saying "if actual relevant context is being passed"
how are you determining what is relevant at any given time?
The current moment, past actions that led up to this moment, what she was doing, the current task, etc
that sounds like not a good idea but... sure?
idk
Linux support when 
should probably already support linux. But I have no way of testing that at the current moment
But that assumes from that the windows api has an unintended benefit of also supporting linux
I'm not sure about other modules like the UI regionalization stuff I'm building rn
Status on the neuro-os project
neuro-relay is now working.
I'm currently working on neuro-os's regionalization for user interfaces
oh right! this. Can I ask a bit more about what is DOS? I'm not sure what it is, and maybe I could add it?
cassitly said start and then left the job on me lol
this could also be something like.
This is such an awesome suggestion
Tag: For Nakurity
Or something like that
now I'm stuck, I need some suggestions to know, what to do next.
Soooo, @wheat reef can you add the WIP tag for this now? I just realized you were online
huh-
I was supposed to be set to offline, but okay!
[WIP] Neuro Windows Integration
There
uhhhhhmmmmmmmm, it could be niwatori.....?
okay...
But I still think its u lol
its pretty funny overall, so I don't mind
atleast some things are still true
Uhmmmm, sooo. Here is a conclusion as of 21/10/2025
Status of the project
Regionalized UI elemental context has been implemented for neuro-os (basically, neuro gets regional contexts of UI elements)
- Neuro OS still has to implement the neuro relay intermediary connection
(This is so that neuro os can host and manage neuro-relay via a custom management ui for vedal ig)
Configuration management across all neuro-os dependencies.
This project might actually start relying on a bunch of windows specific packages now:
I lost the url to the image I was gonna attach, sorry.
I'm gonna go now, goodnight guys!
what happens if you attach randy to it when its done
can it randomly do something bad
Like, command lines.
Randy doesn't have the LLM components like neuro, it requires you to send your own actions via http in its localhost server on port 1337.
Unless its a forced action, it won't automatically complete the actions
I also just realized the speed of the actions jippity can do (jippity is trying to mimick neuro using OpenAI calls), as there is absolutely no delay. And I almost couldn't ctrl+c the process, as it kept alt tabing lol. This happened last night before I went to sleep. I was testing the interface
I'm also thinking of adding an automatic testing mode for Nakurity Backend in neuro-relay.
I'ma provide a free api with nakurity.com ig
Update on jippity testing
apparently, neuro-os broke while I was sleeping, and now today. Jippity doesn't want to work with neuro-os
I assume something rolled back on my pc or something
I'm gonna provide a free api lol
probably session-based api keys
is this a good idea? @plucky marlin , anyone?
I dont get it
I really dont know how that is going to work
its a session based system where you don't need an api key. but a session connection
Are there no local vision to text ai
Is it limited amount of tokens
I am paying lol
No limited tokens, just ratelimiting
Its gonna run on my home server
like everything
but its gonna forwarded to the internet
since I am thinking of using that for prod
Whats the difference between ratelimiting and token limit
idk
token limit is like a limit of how many tokens u can use
and ratelimit is how much you can send messages to the api per a set amount of time
Ok then i meant this one
So if i use it too much it will say "sorry out of tokens wait 3h"
How is that not a bad thing
Noo
its gonna be like 120,000 tokens per 1 minute ig. Thats the ratelimit ima set
since neuro-os does use alot of tokens if im gonna use it for context updates
And what do you mean "you are going to pay"
So if you ever stop paying it will stop working
the electricity cost on my house lol
its going to run on my home server, didnt I say that?
but yeah!
Oh
is this a good idea?
yes, and the server
why?
Because it becomes dependant on stability of your internet
i guess
Also security risks and shit
Idk i would think more about it at least
See what others think
Im not very good in this topic
you could've run git blame
also just use tony
like
randy is the forgotton child 
also link? i never heard of tony ngl
tysm!
I've also implemented a local ocr model just incase you want to use the local one ig
This is such an improvisation for neuro's vision not being on lmao
Btw, should I jump neuro-os straight to v2.0.0 ?, right now we're at v0.0.1-alpha
hmmmmmmm
@steel dagger ?
wb v0.1.0-beta?
I'm not sure which version to put this release in
I'll go with v0.1.0-beta ig, I still need to test some stuff
but its functional enough that I can deem it a release
test those stuff first
but testing it with jippity is my main priority atm
and then make your decision
and I have to buy more credits for openai rn, and I'm waiting for my allowance. An I can't really test with jippity rn
Okay, from my assessment, it is not infact ready for beta.
Let's stay in v0.0.1-alpha and move 0.0.2-alpha then
I don't have any ideas on if I should implement that or not. hmmm
I'm gonna start on this plan now.
YAY! neuro-relay is now working with neuro-os!
logs were a little large, so heres a file!
can you record a cool video 
what video should I record (srry, I was asleep)
Help wanted
I would want some help with the Neuro Windows Project, implementing some stuff
HOW DO I LOSE THE CONFIGURATION FILE FOR NEURO-RELAY IN NEURO-OS, WHAT
@steel dagger I.... kinda... need help
Idk what's calling this:
│ │ │ │ │ │ │ │ │ │ │ ╭▶ retry_queue() intermediary.py:330 [ 6.590s]
│ │ │ │ │ │ │ │ │ │ │ │ args: self=<src.dev.nakurity.intermediary.Intermediary object at 0x0...
│ │ │ │ │ │ │ │ │ │ │ 💥 CancelledError: intermediary.py:330
│ │ │ │ │ │ │ │ │ │ │ ╰↩ return None [ 6.598s]
│ │ ╭▶ connect_outbound() client.py:156 [ 6.604s]
│ │ │ args: uri='ws://127.0.0.1:8000', router_forward_cb=<bound method NakurityBackend._handle_intermediary_forwar..., max_retries=10, retry_delay=2.0
│ │ 💥 CancelledError: client.py:156
│ │ ╰↩ return None [ 6.608s]
│ │ │ │ │ │ │ │ │ │ │ ╭▶ start_outbound() __main__.py:246 [ 6.612s]
│ │ │ │ │ │ │ │ │ │ │ 💥 CancelledError: __main__.py:246
│ │ │ │ │ │ │ │ │ │ │ ╰↩ return None [ 6.619s]
│ │ │ │ │ │ │ │ │ │ │ ╭▶ main() __main__.py:277 [ 6.637s]
│ │ │ │ │ │ │ │ │ │ │ 💥 CancelledError: __main__.py:277
│ │ │ │ │ │ │ │ │ │ │ ╰↩ return None [ 6.643s]
│ │ │ │ │ │ │ │ │ │ │ ╭▶ run_server() server.py:351 [ 6.655s]
│ │ │ │ │ │ │ │ │ │ │ │ args: self=<src.dev.nakurity.server.NakurityBackend object at 0x0000..., host='127.0.0.1', port=8001, ssl_context=None
│ │ │ │ │ │ │ │ │ │ │ 💥 CancelledError: server.py:351
│ │ │ │ │ │ │ │ │ │ │ │ → async with websockets.serve(handler, host, port, ssl=ssl_context):
│ │ │ │ │ │ │ │ │ │ │ │ • locals: host='127.0.0.1', port=8001, ssl_context=None, self=<src.dev.nakurity.server.NakurityBackend object at 0x0000...
│ │ │ │ │ │ │ │ │ │ │ ╰↩ return <Future pending cb=[shield.<locals>._outer_done_callback(... [ 6.675s]
I disabled the trace function but its still being called
that log looks terrible here
and by this I mean this giant verbose stuff
Its meant to look like this
coloring and stuff, its for debugging and showing the program is working internally. This is supposed to be combined with VSCode's breakpoint stuff
@plucky marlin We have another issue! Great!
Tracer for neuro-relay can't be disabled
(This is in the neuro-os project repository)
Project Maintainence / Development
I might take a break a bit from this project when school starts at the end of this week
I asked Cassitly to post this here, in hopes some people would be willing to help me work / maintain this project since its a very big idea.
And 3 people maintaining it might be too little, so I'm not sure if this would be finished at all.
It gets enabled by some random function
"Set trace" or like that
Try commenting entire trace and see where it will error
Its where trace is called from
(maybe this is all outdated and your code changed very much, but in case this part is the same, you can try)
okay!
Minor stuff
that might confuse people, I renamed the repository from neuro os to neuro windows. Since it might be misleading
should I implement permissions for neuro os (now neuro windpws integration) ?
what is abstractions?

have you fixed this?
just curious how you doing
Yea! I have
I'm doing okaay, I guess!
There's a few new issues with neuro-relay forwarding & returning actions and stuff
i might try to help
tysm!
so the issue is: huge latency and tony returning 0 client id right?
no, that's tony's issue. not ours
there's this new error:
that was cassitly's commit, no?
Oh uhm, the latest issue appeared out of nowhere, while I was trying to debug the top error
also, what programming language are you best at?
I can maybe use that next time, so you can understand the codebase better!
python is fine
i dont have much practice in any languages, but there are some i actually touched and used for something:
c, c++, python, lua, pascal and st
i do st professionally but its not used in anywhere besides my job field
alright then lets forget this issue for now. since its fixable by common restart anyway
Yea
i mean client id 0 issue
now this...
I understood what you meant
Wait- you work at somewhere important like nasa or something?
what's the error here?
that language looked so like assembly
neuro windows somehow exiting, and an invaild types, which the types stuff caught
noo im making shitty industrial automation programs, like heating system automation or some oil valve remote control.
its much simpler than it might sound, and the language we are using was created for "specifically for engineers who are not very good at programming"
which was the latest bottom error:
[Nakurity Backend] failed to send action to windows-user: sent 1011 (internal error) error in fragmented message; no close frame received
oooh
Is it at a lower level?
it's okay I can understand asm too
all the hard stuff is usually handled completely by the ide and there is not much to do.. its safe but limited and simple
yeah no I'm stupid now I don't know wtf is going on in the code anymore
try turning on our tracer and using vscode breakpoint
and timing your pauses
it will lock at a very useful log output if you correctly time it every time something happens lol
do i use debugging branch
something something I'm too lazy something something should really be doing some study right now
wait- no use the neuro-windows repo
its in the Nakashireyumi github
where is the error here, cant find it
oh here
wait there is a bunch

can you help me set up the environment please
so im downloading tony now right?
or wait do i have it
it comes in neuro sdk bundle
okok see it
dayym this takes long
why the hell
it tells me that it requires trio >= 0.31.0 but installs 0.30.0 and then complains its under 0.31.0
and neuro api requires 0.30.0?
so after i install neuro api it says this
ok it wants trio 0.31.0 lets install that
after installation of trio 0.31.0
ohh wait its neuro api and neuro api tony
i get it now
neuro-api wants 0.31
neuro-api-tony wants 0.30
hm
they want me to do "..venv\Scripts\activate"
but i dont have this folder in tony
looks like pip install neuro-api-tony command was supposed to create venv but it just never did
sometimes i hate this dependency hell we live in so much
so i cloned tony repo with github desktop and then run "pip install neuro-api-tony
" command
so i assume this chain of 2 actions is somehow wrong
maybe pip install tony installed it into some random folder on my computer and not the folder i was running my terminal from
pip install <x> never creates venv
uv auto-creates venv
cloning tony repo doesn't matter if you run pip install neuro-api-tony
weird
no that's intentional
they suggest to do just this
yup
alternatively they show "github way" where venv is made
so i chose path without venv
you use github mode
if you use pypi it does not matter if you clone github
yeah well right now you're just installing everything into your global package dir
Sorry for not being here. I was eating ||the duck on neuro's head (jkjkjkjk)||.
hence why you are running into dependency conflicts
apparently installation did absolutely nothing? idk
what happens if you run neuro-api-tony
run python -m venv .venv in the neuro-windows directory
it just launched 
is it in my path now
yes
i kind of didnt want that but ok
and install the dependencies
it's not random
you installed it into your global package directory
and because it's an executable binary it adds it to your path

Vituha, can you please tell me your current repository setup of neuro windows integration?
ok lets forget tony installation now, i have tony
how do i launch neuro windows now
iirc that's how every pkg manager works
i cloned it and use master branch
this is the folder
oh so the path isnt really clogged, it just tries to run packages from the folder of python packages
it is still just one path entry with python folder 
run git clone https://github.com/Nakashireyumi/neuro-relay.git
my gitmodules didn't get commited smh
@steel dagger how did you add the git modules the first time? Mine didn't work
i just need some command like python -m src.dev.tests.test_harness
but for new version of your repo
Neuro Windows rely on neuro relay being inside its own folder
or do i use old commands and use neuro relay
so, either move it into the neuro windows folder or clone a new relay :D
ohh
git submodule add https://github.com/Nakashireyumi/windows-api.git
Neuro Windows automatically starts up neuro relay and uses it, sooo
tysm!
is your windows-api folder empty
no its full
i just dont know what to launch there are like 10 programs i think
nested in 500 subfolders
folder row doesn't matter
but yeah, its correct

how to launch your new api
you mean to tell me modularisation is confusing to someone who doesn't understand the codebase?
python -m src.dev.launch
i refuse to believe that there is a single person on earth who can instantly navigate through random repo he sees for the first time
forget an h, sorry
I could!
and see like "oh yeah i should definitely run dev/mev/src/idkkev/windows/linux/apple/lol/123131adjsldjlaneurosomehting.py, its clearly the thing i need"
yeah
i have tony running and my neuro windows repo seems to be correct, can you please tell me terminal command to launch it
do i run this
or this
or maybe this
or wait is it on your repo describtion
i never read it actually lets see 
is it python -m src.dev.launch
yea.... I just said....
.
oh sorry
anywayy
installing requirements.txt seems to kill tony 
now i start thinking i got some broken tony build by cloning tony repo
if you don't want to go through a billion folders just run this!
Very useful
## clear output file first (optional)
Remove-Item output.txt -ErrorAction Ignore
## get all .py files and append their contents
Get-ChildItem -Path "src" -Recurse -Filter *.py | ForEach-Object {
Add-Content -Path "output.txt" -Value "`n# --- $($_.FullName) ---`n"
Get-Content -Path $_.FullName | Add-Content -Path "output.txt"
}
did you use a .venv...?
python -m venv .venv and .\.venv\Scripts\activate
otherwise it installs to your global package dir
and adds everything to ur path
like what KTrain said
That's how pip works
wyd rn?
fixing the install
when i launch it it just sends 500 errors
no package no package no package
full error logs?
i did requirements but i also follow the other things in your instruction there
let me launch it again, maybe requirements was enough
but i need venv for neuro too
just get the .venv for neuro-windows no?
but this happens before i even reach venv part
iirc someone said the packaging mechanisms around python are a mess.

that might be outdated
oh
just run install the requirements.txt
or run this
ok then i do venv now and launch it
since this is an attempt to automate the installation
wait- no. try node src/bin/setup/repository-setup.js if it doesn't work
that might be better, i'm not sure who put neuro-setup.js there
btw step 4 should be optional
im so much afraid that installing all this in 3 ways will ruin everything
i feel it will all break if i do this
don't run neuro-setup
this works
but neuro setup DOES NOT work
idk its a file that someone put there at src/bin/neuro-setup.js
Neuro setup does not work
that just says u need to specify a branch with windows-api
do i just uninstall everything and start from scratch
just cd into it
and change its branch
uhmmmm
wait- actually
start over
repository-setup was meant for an installer where its not the source stupid me
I'm so sorry
so i uninstall everything neuro windows related ok then i do what
That's great news.
with venv
yea
without venv
with .venv
do i clone it first or
okay, so clone neuro-windows first
and then try launching it
if it fails
try installing the dependencies for each repo (neuro-relay and windows-api) inside the neuro-windows .venv
so make a venv in the neuro windows folder
install the dependencies of each repo inside that venv
and try launching neuro windows in that venv
to see if it works or not
its just a sanity check
if you don't want to, its fine aswell
it won't install anything, that's why we have to setup the dependencies
a bunch of "missing missing missing" errors and it crashes
okay, so that's good.
now i cd into windows-api?
nooooo
you only need to install the requirements.txt from the neuro-windows venv
stay on the venv created in neuro windows
then run git clone https://github.com/Nakashireyumi/windows-api.git
yes
then do this
then after that, run git clone https://github.com/Nakashireyumi/neuro-relay.git
then run pip install -r requirements.txt
done
then run pip install -r neuro-relay/requirements.txt
done
done.
anything else before launch?
give it like 40+ seconds to load
that's one of the latency issues
and then when actions appear on tony, it should mean it loaded
it seems to loop here
that's normal
now its doing this
its neuro windows sending neuro the desktop environment to neuro
do i try to execute "screenshot"
yea
make sure its the right schema
there should be a show schema button to check
schema is basically json format
it just says "yes" in schema column
No
executing seems to do absolutely nothing
uhm when you press execute
nothin happens
@steel dagger help this person with tony please
let me restart it all, maybe tony died while i installed some things
?
oh, I'm so sorry!
are you sure
yes i click it and nothing happens
nah alg
have you tried double clicking it
ah yes, the ninja art of double clicking
reminds me of that one time a governor said f12 was a crime
restarted tony, double clicking doesnt work anyway
what does it say
maybe spamming hopes and dreams would work lol
nothing
we need a petition to label all UI elements on every application in a easily understandable way, so that we don't have to keep describing what we meant by the top right window box
there is an official name for it
iirc
by an easily understandable way
so like. basically label it in the UI lol
unless we didn't use its name
did you slow double click or fast double click
or did you click then press execute
and it was already named
i did all this in correct order when installing tony.
before it, i uninstalled old tony with pip uninstall
log panel
you might be using tony v2
they are not
I'm probably stupid then
this sounds like a joke
im not sure if you are trolling
so if you click the action and then click execute
deleting works
i'm being as dead serious as I can at 10:20pm
idk just to see if any button in tony works
hmmm, suddenly tony starts breaking
json dialoger or smth
it will delete system32 are you crazy
yes.
shouldn't be
clone this
https://github.com/VedalAI/neuro-sdk
and then run randy
im not doing that 
there should be a readme on how to use randy like tony in here: https://github.com/VedalAI/neuro-sdk/blob/main/Randy/README.md
cannot reproduce
random os actions
then its local probably
lmao
all this dependency nightmare
I will automate that soon, dw
and it gave absolutely no profit
Randy is the same as tony
it still doesnt deploy properly
tony just implements a ui for randy
not actually
so sorry!
what would it also add?
do i try to "pip install --upgrade neuro-api-tony"
no, it just doesn't give a ui to randy
curl --request POST \
--url http://localhost:1337/ \
--header 'Content-Type: application/json' \
--data '{
"command": "action",
"data": {
"id": "blegh",
"name": "join_friend_lobby",
"data": "{\"friend_name\": \"jerma985\"}"
}
}'
If you didn't know, you could do this with randy btw @plucky marlin
does tony emit a log file?
that gives more than what's on screen?
it does emit a log file
but it's not really more than what you can see on screen I think
but then i need to know format of every action i want to try
just send it here anyways I'll see if I have the time to parse it
there's actions schema for a reason
that doesnt necessarily solve the problem
youre just making it more annoying
do i just paste it in my terminal when randy is running
https://github.com/Nakashireyumi/neuro-windows/tree/master/src/dev/neuro_integration/Actions
Neuro integration has them here
yup. but you need to change the action stuff

open a separate pwsh/bash terminal and execute the cmd
in the data tab
oh right
you have to do that too
you can't paste it into randy
randy doesn't like that. its gonna say nononono
but i need randy to run because integration has somewhere to connect
yea
and by this command i "insert" traffic into their communication
but randy will still do random shit will he not??
i dont want that
I mean, I could try implementing an actual testing server with Neuro relay
randy does random actions does it not?
I'll implement that
not always
so randy will not do anything with current neuro-windows version
since it doesnt force actions or smth
yup
i'll probably have to reinstall neuro api because it got destroyed by some package versions
since tony wants 31 and non-tony wants 30
oh nvm it works
nice
wait do i need to make this into one-line command
?
after pasting this
oh right
or do i need to format it first
forgot pwsh/ is not always bash
to be actual command and not jerma987
How...
I still have no idea where neuro windows exits
windows user is neuro windows btw
confusing name, ik

what the fuuck
oooh, its called backticks
who uses symbols absent on keyboard for anything at all
its not absent
are they stupid
absent on keyboard?
you have to change window's mappings
its literally where the ~ key is
i dont think my keyboard has it
bruh
yeah this is why we should use tony
‘ ’
above the tab button
you did have it....
waht
it made it worse
Windows API is somehow escaping
yea
great
you replaced the '' in Content type and stuff
` just needed to be after a line
curl --request 'POST' `
--url http://localhost:1337/ `
--header 'Content-Type: application/json' `
--data '{
"command": "action",
"data": {
"id": "blegh",
"name": "join_friend_lobby",
"data": "{\"friend_name\": \"jerma985\"}"
}
}'
try this
@plucky marlin
I fixed it
oh my fucking god
it's because pwsh aliases curl to Invoke-WebRequest
oh, then I'm stupid
i would just shift to wsl entirely but this is actually windows integration so i need windows 
You could try using it with arch
it would create more problems anyway
you can run randy on wsl and connect it to the integration I think
Cassitly did say it partially worked
but ubuntu wasn't tested
arch was, since cass has it installed
yea
I love how the drive letter is just D:
as it's sad, lmao
STOP SPAMMING REACTIONS, NAKURITY
wsl says this
srry!
that's probably networking stuff
Idk anything much about networking
@steel dagger do u know?
i could disable firewall
so what
it sees it as another adapter i think
you haven't started randy on wsl yet have you
vm is a virtual machine
randy in wsl
That is powershell....
thats pwsh
waht
are you running windows in....
wtf....
yes
makeItMakeSense
double the workload (on your pc lol)
probably because its not the same wsl instance
i think it should work?
that's the http port that randy opens for u to tell it what action to send
so wait
running another wsl in another powershell terminal starts up another wsl, so basically a second pc
oh
you would need a special command to dock into that wsl instance I assume
but idfk what that command is
command / argument of wsl
i'll go do something real quick, 10 min break
kay
WAIT, I just had an great idea.
nvm
it wasn't that great of an idea
@plucky marlin wyd rn?
wait is wsl connected to windows
yea
but 2 wsls are not to each other
so i just run randy in windows and command form wsl
same error, cant connect
wsl seems to be isolated from windows too
it is connecting to vscode though, but it might be some special case
or maybe it doesnt im not sure
i was using vscode as text editor i think
and stopped when it failed to connect actually
riight.. now i remember.. same shit
vscode has some magic stuff that connects its terminals to wsl
maybe try running the command in a vscode terminal?
hmmm
do you have git bash?
run randy in windows
and run the command in git bash
im not sure i have git bash, some random command from stackoverflow didnt work
start "" "%PROGRAMFILES%\Git\bin\sh.exe" --login didnt work
curl --request POST \
--url http://localhost:1337/ \
--header 'Content-Type: application/json' \
--data '{
"command": "action",
"data": {
"id": "blegh",
"name": "join_friend_lobby",
"data": "{\"friend_name\": \"jerma985\"}"
}
}'
this command btw, this command is for git bash
try searching git bash in the windows search bar
if you have the command for git
you should have it
I would assume
i think
kay, if it truly worked
ok so i need to edit this to make screenshot now
neuro windows should've have an error
yes
this is where all the actions schema is:
https://github.com/Nakashireyumi/neuro-windows/tree/master/src/dev/neuro_integration/Actions
is this wrong
curl --request POST \ --url http://localhost:1337/ \ --header 'Content-Type: application/json' \ --data '{ "command": "screenshot", "data": { "type": "object", "properties": { "name": {"type": "string"}, "region": { "type": "array", "items": {"type": "integer"}, "minItems": 4, "maxItems": 4 } } } }'
also how do you post it fancy like here in discord
right click and copy one of my text to see:
curl
this also works for python:
from neuro_api.command import Action
import pyautogui
def schema():
screen_width, screen_height = pyautogui.size()
return Action(
"click",
f"If coordinates for the mouse is provided, move the mouse to that area, then click. Screen dimensions are {screen_width}x{screen_height}. X must be between 0 and {screen_width-1}, Y must be between 0 and {screen_height-1}.",
{
"type": "object",
"properties": {
"x": {"type": "integer", "minimum": 0, "maximum": screen_width - 1},
"y": {"type": "integer", "minimum": 0, "maximum": screen_height - 1},
"button": {"type": "string", "enum": ["left", "right", "middle"]}
},
"required": ["button"]
}
)
i think it works
