MCP -> Neuro-SDK Translator

[jolly lantern]

Why I build this

MCP has a large community and there are a lot of interesting MCP tools. To build such a translator will allow Neuro to play with these tools with a a few commands.

How it works

First you start a standard MCP server using Streaming HTTP to deliver tools (e.g., expose /mcp endpoint), then you start Neuro or a mock Neuro server, finally you start this translator. It connects with MCP on one end and Neuro's websocket on the other, handles action registrations/network traffics and so on.

A short description of the game & how to get the game

The "game" is MCP server. So you need to select a MCP server and install it. This is usually done with npx. I tested this translation layer with playwright-mcp server, you can install and run it with npx @playwright/mcp@latest --config path/to/config.json

A link to the repository

https://github.com/ECHO-HELLO-WORLD424/NeuroMCP

Quirks to take note of

• IMPORTANT: As mentioned in the Limitations, I haven't solved the graceful shutdown or un-registering actions. Accidentally shutdown the MCP server ot Neuro test server when bridge is connected will cause errors and stop the bridge

• Out of the same reason, you must start Neuro test server and MCP server first, then start the bridge. Otherwise it will likely crash.

• I don't know for others but sometimes I find the bridge's access to localhost is somehow denied. A common workaround is to bypass any proxy settings, which I hard coded into the script. However if your application/MCP server need to use proxy, you may need to try some other approach.

Some Limitations

• Currently one "translator" can only accept one MCP connection and one Neuro connection
• Some features in Neuro SDK is not available in MCP protocol (action force, for example), thus not implemented. Which means that this is probably not intended for tools that asks Neuro to "Respond Now!".
• Currently it can't close or un-register actions gracefully [Solved, but with modified python ver of Neuro-API. Not commited yet.]

[thorn lodge]

@jolly lantern we were actually planning on doing MCP support through VS Code @ this thread: #1350968830230396938, would it be alright if we combined our efforts?

[jolly lantern]

Of course! I'll see this later I have a discusion session

[thorn lodge]

thanks!

btw, the reason why I asked was because vs code extensions can contribute their own tools as well, and also it would be better if an entire coding suite for Neuro was in one (more or less) package
(I'm aware that MCP isn't strictly code-related, there are MCP servers that have nothing to do with code, but majority of MCP is code-related these days)

[jolly lantern]

Exploring neuropilot codebase. I'll try to cook some working examples tonight (hopefully?)

[thorn lodge]

sure!

if you need some help you are more than welcome to chat in our thread

I think there might be a invokeTool method somewhere inside the VS Code API, but do check that just in case

[broken vortex]

What's MCP?

[gusty verge]

Seems to be a waiter for AIs

[jolly lantern]

It's kinda like the USB-C port for LLM. It provides a standardized way to connect AI applications to external systems like browsers, IDEs, internet or whatever. Think of LLM as a "OS" then MCP will be some "Applications" that you can install on that OS with a few clicks. Image credit: https://modelcontextprotocol.io

[thorn lodge]

hmm, it would be good if this can connect to multiple servers at once

[jolly lantern]

Actually this is on schedule. I'm thinking of a "router" alike module that integrates all MCP servers running into a single large MCP server and handles permissions internally, then use existing code to translate tools/register/unregister actions etc. This also allows us the isolate STDIO which may break if you accidentally calls print(), console.log, println() or any similar code in downstream programs

Neuro <--WebSocket--> [Existing Single MCP Integration] <--Streamable-HTTP--> [Router] <--STDIO/Streamable-HTTP--> [mutliple MCP servers]

I believe this would be helpful for both NeuroMCP and VSCode extension.

Maybe later because now I'm trying to make the single MCP VSCode extension work first.
And I have a number theory midterm so ...

[thorn lodge]

sounds good

As a trade for helping with MCP on our side, I can help with this once you're ready

[jolly lantern]

Wow thank you for that

[thorn lodge]

no problem :)

Long term I would like to adopt this as one of the integration projects I help with, alongside NeuroPilot and Neuro's Canvas ()

[thorn lodge]

by the way, are you planning to write this in ts or py?

also when I said that before I meant that the NeuroMCP interface directly handles multiple MCP servers but this works too

[jolly lantern]

I plan to make 2 version, one py and one ts.

[thorn lodge]

Also, integration with MCP registries could be cool

[jolly lantern]

Since py in VScode extension seemed to be bad idea

[thorn lodge]

since it's emitting streamable http don't we only need one?

[thorn lodge]

technically possible but wasm is a bit of a mess...

[jolly lantern]

Yes, that would be easy for developer but user have to download python/uv and setup envs etc.
If this module is not very large (it shouldn't be, I guess) then I can have 2 versions of it
I just translated the NeuroMCP to ts last night (with some help from claude, I confess), it takes some efforts but not that hard.

[thorn lodge]

hmmm

you didnt have to

(translate to ts)

[jolly lantern]

But since Vedal is the main user, setting up Python should not be a problem for him

Anyway after a second though, you are right about this. It's better if we just handle multiple connection in this NeuroMCP

[thorn lodge]

other way around

developer has to set up multiple runtimes, WASM is just a standard that allows users to run compiled code from other langs in JS essentially

[thorn lodge]

Honestly the idea of a permissions-based relay server is good in concept

I would like to do some work on such a relay server as part of VSC-NeuroPilot

this is mainly because it would integrate well if we could turn it into a discoverable MCP server registry that can be connected to VS Code

but also I want to keep it interop with other things so projects like this can benefit

[jolly lantern]

Actually, I think it might be good idea to make the relay as a MCP server itself (like "a MCP that coordinates others"). This requires minimal setup, you can deploy the relay wherever you want and VSCode can even automatically launch it in theory

By doing this we can also offload some difficult async programming to the relay, and this relay can also be used by other applications/projects like PyCharm or Claude Desktop

[thorn lodge]

That can be separate, but my idea was to be able to have a single coordinating server that hosts connections to/from other servers and/or retain a listing of servers that are allowed to be accessed

The reason why I wanted to make it an MCP server registry is because Copilot can discover other servers on your behalf and use them (at least in VS Code), so we can have something like that for NeuroPilot this way

but also if we can make it so that it's something you can use as well it would be very healthy

[thorn lodge]

Actually no, saying "that can be separate" is a lie, since we can combine both an MCP registry and MCP relay into one

we just need separate endpoints for both

and also needs to be lightweight so you can start it locally

yet still have a GUI for it, notifications for permissions (for requests, assuming we have the same auto/co/no like NeuroPilot has it)

and also a web API from the server that controls what is and isn't revealed

[jolly lantern]

emmm but does this means user need to permit actions in both NeuroPilot and in this GUI

[thorn lodge]

Depends on how it is configured, the reason why I said "notifications for permissions" is because for other apps (such as NeuroMCP) this relieves the permissions side from the client

We also don't have to have auto/co/no

and instead control with a binary state (true/false) for whether or not it is exposed to Neuro

[jolly lantern]

Oh I see.

This looks like a large project, and can really benefit a lot of projects in the community. Maybe we should create a new repo for it when we start implementing it

[thorn lodge]

Yes

I wasn't thinking that making the MCP relay in your repo directly was too smart of an idea considering now it could potentially be used by us

I'm still not sure on who should own the repo though

[jolly lantern]

Ideally this should be put to some sort of "Neuro Community GitHub Orgnizations" or so but I don't think there is any.

Since this relay can be widely used, even outside of Neuroverse

[thorn lodge]

Once we get to implementing it, I'll ask about keeping it in VSC-NeuroPilot, since we can make use of it (and imo we should, for efficiency reasons that I don't want to elaborate on right now)

worst case - just leave it in your account

[jolly lantern]

Yes we should try to keep it in VSC-NeuroPilot if possible when we start implementing this.
Maybe after next dev stream, I have 2 math exams this week

[thorn lodge]

I think maybe a little while after that even, since on our side of things I want to knock out a bit more of our task list first

give it like another 2 dev streams just in case we need to make some stream improvements

[jolly lantern]

OK. Meanwhile I see that you are implementing some new feature in neuropilot. Some of them are helpful for the MCP integration as well. I'll further explore neuropilot and try make single MCP work first, then we start work on this relay.

[thorn lodge]

which feature are you looking at?

[jolly lantern]

The registration part discussed this afternoon, and you mentioned that there is a vsc-neuropilot/client-sdk thing

However I'm not 100% sure. Still exploring the project.

Anyway thanks your guys for making neuropilot project so modular and understandable

[thorn lodge]

it's not quite modular yet

and also we try to make it understandable in case a code review is needed

[thorn lodge]

so that all forms part of our extension API that can be used by other vscode extensions only, the @vsc-neuropilot/client-sdk is a helper sdk that exports a class to abstract some code away, mainly so that I don't have to repeat and fix the same logic 5 different times across all 3-5 extensions we plan to split into

I would recommend you look at the PR for it

if you want to learn more

however it's barely started so we got a while to go before it is usable

[jolly lantern]

I'll look into that
Gotta go now, I'll post the progress in VScode thread when available

[thorn lodge]

thanks for the help :)

[thorn lodge]

I for some reason decided to think about this today and I may have just thought of a tech stack to use...

It might be a bit too experimental though so if it gets too annoying to work with I'll switch out some of the stuff to be less broky

[lime rover]

I'ma just borrow this for neuro desktop, may i?

#1428980640895733820

[jolly lantern]

Be my guest!

Happy to see that this project is useful

But do check the limitations.

[lime rover]

Oooh, okay!

why... are you posting this here

is this a bot? Mods!

[jolly lantern]

What

[lime rover]

nvm

[lime rover]

there was a job description msg here that @torpid maple posted

seems like they deleted it

[jolly lantern]

...?

Alright..

[lime rover]

can this connect to multiple mcp servers at once?

hmmm

ill play around with this when i get home later

[jolly lantern]

Not now. You can see the discussion with FM/KTrain.ts

[lime rover]

its planned?

but not yet implemented, ky!

[jolly lantern]

Yes, not implemented yet. We decided to plan it after 2 dev streams

[lime rover]

ooh

[thorn lodge]

@whole fiber sorry for the ping in here, do you think we could make a repo on VSC-NeuroPilot for a locally (& remotely, since the stack in mind allows for it) hostable MCP relay/registry? my thinking for leaving that in our org is that we can use it as well (at least the registry side of it) by allowing neuro to independently discover MCP servers at runtime (something Copilot does) which saves on resources

also pinging here because it's not directly related to NeuroPilot, if it's better to move the discussion back into there though then that's fine but we already have enough to sort through as is.

-# yes, we aren't implementing it yet, but I figured since I have an idea for it right now I could at least set something up in advance

[lime rover]

thats what I do

procastination on my end, when I have an idea, instead of actually getting around to doing it.

[whole fiber]

I'm not sure I understand what that means, I have never worked with MCP before so I don't quite know how it works yet. What would the relay/registry accomplish?

[lime rover]

more capabilities for neuro

[thorn lodge]

relay was for this project but also allows enforcement of tools at the server level instead of the client

registry was so that neuro can self-discover and self-start MCP server connections, which saves on resources so that the client (NeuroPilot, in our case) doesn't have to maintain a connection to all servers while working, she can stop and start MCP connections as she (or Vedal, if we add the capability for it) sees fit

MCP is just the protocol that allow models to have tools, so you can e.x. add GitHub's MCP server and let Copilot delete all your repos for you /j

to put it in Neuro API terms, tools are actions which are "registered" with models, that the models can invoke

[whole fiber]

So if I understand this correctly, an MCP server that can connect to other MCP servers and connects to them only when requested? If so, that sounds like something which would likely already exist.

[thorn lodge]

Nope, combed through the official MCP list (edit: of community-made items around MCP) and that does not exist

also the idea includes allowing Vedal to toggle what servers are added to the "registry" at any time

like, it's not a registry in the literal sense

[whole fiber]

Meaning it would have some UI or endpoint?

[thorn lodge]

both

well, I plan to have both

UI is confirmed at least

also working with @jolly lantern for it too

[whole fiber]

I have nothing against doing this on the org then.

[thorn lodge]

Alright

[thorn lodge]

@jolly lantern I have a repo almost set up locally for the relay, once I've gotten something set up and pushed I will add you with write access to the repo

[jolly lantern]

Got it. Ping me when you're ready.
Meanwhile I'll fix some issues in yesterday's PR (by tommorrow perhaps)

[thorn lodge]

cool

I'm just setting up some CI right now before I push

[thorn lodge]

added, forgot to tell lol

[jolly lantern]

Saw it .
Also need sometime to review Typescript because I've stuck in a... kinda desperate Python LLM project since last year and didn;t touch much other langugage until recently

[thorn lodge]

all good

[jolly lantern]

Oh and it suddenly occurred to me that "tool merge" should be added to post-v1 targets

for example

[thorn lodge]

?

[jolly lantern]

give me a sec

[thorn lodge]

like merging tools into categories or what?

[jolly lantern]

Kinda

like

Instead of have :

@tool
def open_steam():
  ...

@tool
def open_chrome():
  ...

Should be:

@tool
def open_app(name: str):
  ...

urrr

bad example

But the main idea is that:

1. this relay is likely to integrate mutliple mcp server

2. each server tend to provide dozens of tools

3. This will definitely confuse neuro, as I know that even huge open/close source models tend to be confused when there are too many tools

[thorn lodge]

Yeah that is a post-v1 thing too

you can add it if you want idk

[jolly lantern]

I need to think about it, idk if it's even possible (should be, I know how to make it happen in Python but is a bit complex. I'm not sure if it should be there).
Will deal with all these in tomorrow's coding session

[thorn lodge]

cool

I think neuro can handle using a lot of tools but even then she seems to be dead set on using a specific group of tools as opposed to the entire toolset

[jolly lantern]

Yea that's typical for agent with lots of tools as attention matrix is keen on some tokens in descriptions of some tools

[thorn lodge]

ah

[thorn lodge]

somehow this disappeared?

[whole fiber]

I think it disappears if there is no activity for a week

[thorn lodge]

I guess so

[thorn lodge]

@jolly lantern How's your stuff going? Assuming nothing happens on NeuroPilot's side (estimating to be the case given it's Abandoned Archive's release day, so dev stream probably won't have neuro coding), I can probably start working more on the relay stack.

It's kinda not super functional right now but it can work if done right, I might see if I can make some changes to the stack to make it more stable though

[jolly lantern]

I guess I'll still be busy until Dec.3, which is some sort of paper DDL
Anyways maybe just get these done before Dec.19

[thorn lodge]

16-day turnaround time? that's probably doable if we lock in hard

I'll get started on implementing this stuff a little bit first for headstart purposes

[jolly lantern]

OK I'll catch up later and do the translation stuff

[thorn lodge]

I already have a base for writing the MCP server endpoints and such, I just haven't finished the part where I do dynamic tool registration and setting actual servers to connect yet

[thorn lodge]

you don't need to yet

there is basically nothing there

to even base it off yet

I also need to evaluate something

[jolly lantern]

Later means maybe a couple of days later
Anyways I'll write the code when I can

[thorn lodge]

Sure then I guess.

[jagged lily]

This is cool, but this isn’t a game. This is more of a tool. I’m more interested in tools like this than games though. Looks pretty cool.

[thorn lodge]

Yeah, the reason why it's included is because the rules state the following for integrations:

Hence why the word game was in quotes

Also it used to be called the Neuro Game SDK, so...

[thorn lodge]

Also, the name "tool integrations" is better than the term "non-game integrations" but it's too close to MCP terms (ironically) so idk we gotta come up with a better name than that

[jagged lily]

Ah fair enough.

[thorn lodge]

I just don't think vedal + whoever wrote the rules would think that people would also be integrating programs that aren't games

it's quite fun ngl

other than the fact there are limited uses for an action force

[jolly lantern]

@thorn lodge Can you create a dev branch in neuro-mcp-relay-registry so that I can fork dev branch? Things may become messy if main is forked.

[thorn lodge]

You have direct write access

[jolly lantern]

Oh
I have access on web but github desktop says no??
Weird, lemme check

Oh proxy issue, no problems here

[thorn lodge]

@jolly lantern did the mcp spec update recently?

[jolly lantern]

Sorry for the delay

MCP team always like making frequent update and you will need to check the source code of SDK and write some conversion functions from time to time.

This could be annoying but I think there's nothing so revolutionary that requires signaficant code refactoring since the streamable http update (An example is the tool return format. There is a conversion function in the server/routes/mcp/clients/util in the mcp registery project that reflects this change )

I think the last update was made around 11.25 or so. Anyways the specs are here if you need references:
https://github.com/modelcontextprotocol/modelcontextprotocol/tree/main/schema

[jolly lantern]

In a word, not really, although the SDK and protocol is under active development, user shouldn't feel much differences.

[thorn lodge]

yeah true

[jolly lantern]

Although the tool return format stuff mentioned above delayed the relay server for a while, a backend providing necessary endpoints should be ready in around a week

[thorn lodge]

👍

I'll take a look at it, I'm aware the tech stack chosen for it is... a little interesting.

so I'll take a look at it

[jolly lantern]

Will write a document for API and call you when finished. Maybe I'll leave the frontend to you cause I'm not good at it LOL

[thorn lodge]

Sure, I'll probably need to do some sort of WebSocket stuff connecting to backend though so I'll try my best on that too

Also, we can test and merge the backend first, I'll get the frontend working later

no point in frontend if backend doesn't work

[jolly lantern]

I plan to fix CI stuff after I got core functions online (routing with lock is basically ready, I'm working on permission stuff now, and after it is done I also need to make sure the locks are added to correct positions), then test and merge.

[thorn lodge]

👍 I'll give a review as well