#MCP Support
coarse niche
I can drop obs-y stuff I’m focused on and get started on this, but I’m leaving for PTO on Friday so it’s unlikely I will get anything done
restive delta
I can drop obs-y stuff I’m focused on and get started on this, but I’m leaving f...
ok then better to stick to the current assignment! There will be plenty left to do when you get back
coarse niche
9:32 AM] cwlbraa: i like this pragmatism 🙂 if we actually wanna ship one, cc @limpid mirage , idk which approach is the sensible one: shell or gql (or both)
[9:33 AM] cwlbraa: for the shell approach we definitely need shipped shell documentation and cookbooks
[9:36 AM] cwlbraa: the other open question is if we're optimizing for distribution: do we need to be able to do this as a dagger-mcp-server-dagger-module or do we just hack it up and run/distribute it via docker
[10:01 AM] solomon: It's easier to ship now: just plug it on my llm branch. The "shell or gql" decision is already handled by the bbi system.
[10:01 AM] solomon: The hardest part will be bikeshedding the API 🙂
[10:11 AM] cwlbraa: tbh i do not concretely understand how the BBI system sidesteps this... the MCP endpoints have to take something here (this is the bikeshedding as i understand it) to ultimately translate to dagql calls... the BBI takes what as input? natural language? (plz note this question is coming from a place of ignorance not doubt lol)
[10:11 AM] solomon: Instead of making mcp() a sibling of llm(), I'm tempted to make it a child... So you have to integrate more deeply and give dagger model config. The risk is that some devs get annoyed "I don't want your llm integration, you're just a dumb tool server, know your place". but we can argue that 1) mcp is a bridge to a better architecture that requires llm access and 2) that tool configuration might be different between models, the same way prompts are different. Unfortunately that's not true today, we're missing a technique that adapts tool interface depending on the model. But not hard to believe difference models are good at different BBI
[10:13 AM] solomon: BBI is how a given Dagger endpoint is mapped to a set of tools. Different BBI drivers implement different mapping techniques. So gql, shell, "flat" are all potentially bbi drivers
[10:14 AM] solomon: mcp server implementation can just target the bbi frontend and not worry about drivers
[10:15 AM] cwlbraa: i see, so it doesn't sidestep, it answers "gql or shell" with "yes"
[10:16 AM] cwlbraa: the word "potentially" is pulling some weight though lol, does that mean rn one of these is implemented and others aren't?
[10:17 AM] solomon: yes only "flat" (Andrea's hack day strategy + some improvements) is implemented 🙂
[10:18 AM] solomon: before implementing more we actually need to upgrade the bbi interface slightly to support multiple objects.
[10:18 AM] solomon: (at the moment it assumes only 1 object in the llm state eg. withToyWorkspace or withContainer but not both)
exporting this convo here because it's relevant and starts poking at requirements for an MCP integration that wires into the BBI system. a couple requirements buried in here, mega up-for-discussion because i'd prefer a "do X first, ignore Y for now" approach:
- like llm() or service(), our mcp integration should exist in the core API and attach to arbitrary dagger objects.
- mcp support should allow created mcp servers to be accesible both from a dagger-orchestrated llm() OR an external client. the external client needs to provide host "networking" services similar to Service().Up(). (quotation marks here bc on the host, this is ideally stdio networking)
there are easy ways to fake the "arbitrary dagger object" support without getting tied up in the core code, but, intuitively, approaching from that direction will likely cause either nonexistant or awkward UX around wiring MCPSes into melvin-like agents. i don't think that's ideal, but i'd like to state an actual concrete product requirement that's not feasible from an external "dagger MCP server"
restive delta
Yeah 100% it has to be in the core. The requirement for me is seamless intregration in the dagger ecosystem. You don't just want a wrapper tool that can expose one dagger object as one mcp server. You want any dagger module to consume or expose a mcp endpoint at any time IMO
Not 100% confidence just because we haven't actually prototyped it that way
But it panned out with llm(), I'm at 100% there - no way we could implement llm() as it exists outside of the core.
So it's an educated guess that the same will turn out to be true for mcp
coarse niche
a. consuming requires a MCP client, which is kinda interesting... that might be the thing we really really cannot do outside of core, but there's a shit ton of fog-of-war here cuz neither me nor alex hacked at the "mcp-client" part of LLM< ->mcp-client<->mcp-server->tool... in all our experiments that thing is claude desktop.
b. exposing any dagger module could be something that the caller impl does, if that makes sense? like you're building an agent, so you use a combo of x-container stdio service support + a dagger module that wraps an arbitrary dagger object in suitable MCP API fluff to expose what you wanna expose. there's several blockers from that approach rn, but end of the day they both look like {core.,module.}MCP(object).Up()
restive delta
Mmmm yes it's true that maybe we don't need the client to be in the core
coarse niche
(also fwiw i also wanna put it in core, i'm just using the hypothetical to try to understand what that buys us)
restive delta
(also fwiw i also wanna put it in core, i'm just using the hypothetical to try t...
agreed I like the approach, keep ourselves honest
Focusing on the server side: I think there's a strong parallel between MCP-server and LLM implementation - namely the BBI.
With imminent multi-object, it's not even a single Dagger object that we're exposing to the llm context necessarily. And what we want to expose as MCP is exactly what we would expose to the LLM if we prompted it ourselves. It really is tightly coupled
If the BBI is a pair of VR goggles, we want to say "we can put the goggles on any LLM you want, whether via OpenAI or MCP protocols" 🙂
coarse niche
slapping the BBI goggles onto claude for the tool calling might immediately get us into the MCP client thing? do you know if claude already has some lower-level, more-similar-to-OpenAI-tools, non-MCP toolcalling mechanism? or is that the exclusive domain of MCP?
restive delta
Yeah all endpoints support OpenAI-style tool calling, Claude included
coarse niche
bc if that's the case, that's the reason to wire into core: any llm that can call MCP tools can plug into BBI, and then the "expose anything as an MCP" is almost syntatic sugar in support of that
weird
mcp continues to confuse me lol
in the "why is this a thing" way
restive delta
MCP is an additional layer of tooling that Anthropic is trying to popularize, for plumbing external tools - ultimately they all get consumed by the same OpenAI-style tool calling anyway
So basically MCP is a tool call reverse proxy
coarse niche
unsurprisingly we can probably proxy a proxy outside of core lol
restive delta
It makes sense from the POV of Anthropic where the most important app is Claude Desktop, and all other software is an external plugin to be orchestrated
So it's a highly centralized model with a big app in the middle, and little plugins around it
coarse niche
the other possibility in terms of what MCP support in core buys us is that BBI just makes implementation way easier, i don't have a good read on that yet and it'd take me a couple days of poking.
restive delta
By open-sourcing MCP, Anthropic popularizes that model, and gets other "big apps" like Cursor etc, to join the ecosystem
yeah MCP server endpoint on top of BBI is probably 100 lines of code
coarse niche
previously in my career i've made a point of distrusting staff+ engineers estimating loc lol but i don't know whether i should apply that rule to you or not XD, it seems unfair in this case
coarse niche
Yeah iirc me and Alex used both mcp-go and mcp-server-proxy in the hackathon
restive delta
limpid mirage
hmmmmmmmmm, i wonder if we actually just need a special dagger mcp CLI command
might be missing something, but not sure how 9479 helps since Claude still just expects to run a command and pipe mcp over stdio
9479 sounds to me like making it possible to route HTTP (really TCP) => stdio, CGI-style, which is actually pretty nifty but seems like a different problem
restive delta
maybe we go one step further and ship a separate CLI called mcp 😛
powered by dagger of course
I'm thinking we could kill 2 birds with one stone:
-
Magically run all existing MCP servers (they either require
docker build + docker runornpm runor some other ad hoc command). Surely we can daggerize that -
Also magically run any dagger module and/or shell expression
Oh my bad. Goose seems to support both exec & sse
limpid mirage
no Windows desktop app 😭
restive delta
Imagine if I could do:
mcp serve github.com/dagger/dagger
or:
mcp serve --name "Dagger dev environment" 'github.com/shykes/melvin/workspace --github-token op://Dev/kjhsdfjhsdfksdj/credential --repo=github.com/dagger/dagger'
Then from Goose: "start a draft PR adding this or that feature"
I guess maybe dagger mcp makes more sense - it's not like all MCP servers are amenable to being containerized anyway
Looking at the official list here: https://github.com/modelcontextprotocol/servers/tree/main/src
Most of those are so limited anyway... This is the redis MCP server. Basically gives you access to ONE redis server - batteries not included, you have to run it yourself out of band
limpid mirage
found a Windows build 🎉 https://github.com/block/goose/actions/runs/13268069432
restive delta
dagger mcp 'github.com/dagger/redis | client'
hot kraken
From here: https://community.n8n.io/t/provide-and-use-model-context-protocol/63799/5 n8n people wanting MCP
limpid mirage
🎉 - dagger call -m github.com/vito/daggerverse/mcp server up => add a SSE extension pointing to http://localhost:8080/sse
restive delta
ha ha 🙂
So, is the lesson that we can easily have dagger mcp [-H URL] <SCRIPT|MODULE> then client just connects to URL to consume SCRIPT|MODULE over MCP?
Or, is the lesson that MCP support should be implemented as an external tool or module, and doesn't require core dagger support at all?
I think I still prefer the latter because it leaves the user in control of module installation, secret injection etc using regular Dagger UX
limpid mirage
honestly it's hard to say without a particular use case in mind. is Claude/Goose the primary target? or are there other things to explore, like whether we need to implement an MCP client? (e.g. llm | with-tool $(github.com/vito/daggerverse/mcp server))
if Claude/Desktop are the primary target, I'd want something that involves editing my MCP server config only once, since that's the most annoying part
restive delta
Multi-variable equation...
limpid mirage
side note: tried using toy-workspace in Goose, very quickly hit the token rate limit - probably from the schema since it's pretty big? kinda weird though, I never hit limits in Claude but I do remember hitting them quickly in OpenAI (with a MCP <-> tools translation layer). Maybe Claude gets special treatment?
wonder if Goose supports dynamically changing the set of tools
interesting, some tricks Goose does to reduce token usage: https://block.github.io/goose/docs/goose-architecture/#context-revision-token-management
restive delta
use find and replace instead of rewriting large file
@shell remnant mentioned that technique as well
limpid mirage
gonna need a MCP directory directory soon
restive delta
I think MCP is an artifact of people really really wanting a standard way to drop LLMs into a programmable environment, and a standard ecosystem to build these environments from.
I feel like Dagger can really help with that, in a way that is complementary to MCP.
Does that resonate with you @limpid mirage ?
I see two distinct problems right now:
-
deploying the environment ("I have the github url of a mcp server I want to use, now what?") -> gotta jump through lots of hoops still. I heard mcp.run for example wants to solve that? 🤔
-
composing the environment. Those servers are monolithic. No awesome way to compose them either on the software side, or on thr model side
coarse niche
I think MCP is an artifact of people really really wanting a standard way to dro...
resonates with me for sure
coarse niche
I see two distinct problems right now:
1) **deploying** the environment ("I hav...
these i'm less aligned on...
- "gotta jump through a lot of hoops" i agree the UX for adding these to claude desktop is awkward, like i don't wanna hand-edit the repo's provided json blob, and mcp.run's a better way to configure using a smart proxy that remembers your configurations, but i'm skeptical that this is really an important issue.
- less skeptical about this one, though. x-servelet (i'm adopting mcp.run's terminology here for mcp servers bc it's confusing calling them "mcps") agent utilization does not work well today, and using dagger as a platform layer may help meaningfully with things like "crib from xyz github repo to implement x using fs_servelet edit_file on file_y.go"
limpid mirage
swing and a miss... would be really nice if Claude were trained on MCP 🙃
so hard to find answers to things like "how to scope MCP to a chat session?"
restive delta
@limpid mirage you saw that email from MCP crew? They mention that
restive delta
ha ha. funny because I'm still searching for it
restive delta
@silver sentinel 👋 this is where all the fun MCP conversations are happening 🙂
@silver sentinel I think @limpid mirage 's PR actually does what you were asking me about. eg. dagger mcp -m github.com/shykes/hello will expose my Hello module as a mcp server. Essentially any dagger module becomes a MCP server automatically
silver sentinel
perfect -- yeah, I wasn't sure whether the PR does thatt. But yeah, running something like dagger functions -m github... --list-mcp-tools would give me the tool definition for everything that specific module exposes -- having that would then allow the agent to call anything against that specific tool.
restive delta
For example you could pre-configure my git module with your ssh key - then the llm can make authenticated ssh operations on your behalf:
dagger mcp -c 'github.com/shykes/git --ssh-key=~/.ssh/id_ed25519 | clone https://github.com/dagger/dagger'
oh, not currently, but that's a good idea, lemme add it (it doesn't currently support -m)
restive delta
For example you could pre-configure my `git` module with your ssh key - then the...
This will expose the entire GitRepository as a collection of MCP tools + state attached to them (so not just a tool, but a whole environment)
@silver sentinel I'm very curious to learn more about your use case btw
silver sentinel
This will expose *the entire GitRepository* as a collection of MCP tools + state...
so think about someone who doesn't know about dagger at all etc. and is not creating agents/calling llms with dagger either. They are using langchain or crewai or other agent frameworsk out there that have the ability to "connect" to external tooling via MCP.
perhaps this illustrates it better :
# Create server parameters for stdio connection
from mcp import ClientSession, StdioServerParameters
from mcp.client.stdio import stdio_client
from langchain_mcp_adapters.tools import load_mcp_tools
from langgraph.prebuilt import create_react_agent
from langchain_openai import ChatOpenAI
model = ChatOpenAI(model="gpt-4o")
server_params = StdioServerParameters(
command="dagger",
args=["-m", "somemodule"],
)
async with stdio_client(server_params) as (read, write):
async with ClientSession(read, write) as session:
# Initialize the connection
await session.initialize()
# Get tools that the dagger module exposes
tools = await load_mcp_tools(session)
# Create and run the agent
agent = create_react_agent(model, tools)
agent_response = await agent.ainvoke({"messages": "run something from the dagger module"})
and if the dagger module is the HelloDagger one from here, then tools list would have a single tool called "build"
limpid mirage
oh, not currently, but that's a good idea, lemme add it (it doesn't currently su...
(pushed)
restive delta
perhaps this illustrates it better :
```
# Create server parameters for stdio ...
Is this code snippet representing an external agent app built with langchain, that might be extended by a Dagger module exposed over MCP?
It kind of looks like a manually developed MCP wrapper?
silver sentinel
exactly
they use a wrapper to connect to the MCP and parse the tools and add them to the agents to pass along to the LLM when making the call
silver sentinel
I think that would open up the whole ecosystem of dagger functions to anyone working on agents
restive delta
There's also the option of generating python bindings, or even Langchain-specific bindings if there is such a thing - as an alternative to MCP
@silver sentinel do you find it confusing that Dagger can simultaneously 1) implement an agent loop itself, and 2) be consumed as "dumb" tools by an external agent loop?
silver sentinel
I don't think it's confusing; it gives people more choice on how/what to use dagger for. they can build agents OR just consume the tools from their existing agents
the first probably requires a more intimate knowledge of how dagger works, while consuming the tools is just a matter of knowing which tool you want
restive delta
@silver sentinel do you have any opinion on mcp.run? We were looking into that as a) possible inspiration, and b) possible integration (not sure which)
silver sentinel
<@286195980599296000> do you have any opinion on mcp.run? We were looking into t...
I had a plan to look at it this week, so no opinion on it yet 🙂