selectQuery

[rapid timber]

@charred quiver I'm lacking some context about how/if selectQuery is relevant in the context of multi object

my impression that even after selecting a tool, the LLM should still have access to all the previous known objects. Trying to understand a bit more where/how the losing track of Query happens.

[charred quiver]

the problem is that Query is a special case - it's not an object, there's no QueryID etc

which is why it has a special API, LLM.withQuery, which sets it as the initially selected object, even though that's kind of a lie (it's an object, but it has a nil ID)

i'm not sure how to square this with the new Environments API, either

[atomic orchid]

separately from that, the word "query" is meaningless to the LLM

and confusing to the caller

so there's pending bikeshedding on that

[charred quiver]

yeah exactly

"root"?

[rapid timber]

yes, agree. Just calling it query as an internal reference

[atomic orchid]

Well I think we can be even more specific. It's really about privileges to 1) access the core API 2) access the caller module's context

we could take it even further and make it about each core API subsystem, like "ability to create a container", "ability to create a directory", "ability to access the host" (kind of a big one) etc

@charred quiver we were just talking about the need to manually curate what core functions are masked etc

[rapid timber]

@atomic orchid questions:

1. Everything (selectQuery + privileged) seems to be related to the Environments API, shall we scope this discussion under that issue / thread?

2. There's a current limitation to build agents which is to lose track of the root context / core api / etc. Shall we provide a stopgap in the meantime?

3. Can we push the privileges discussion a bit further down the road? Or we should start designing this scopes based on that?

Also 3, re permission model: MCP has adopted OAuth as their authentication protocol. I'd assume that will inherently affect the permission model MCP servers adopt. I guess we might want to take that into account while designing our permission model to at least make sure we can make them compatible eventually?

[atomic orchid]

Everything (selectQuery + privileged) seems to be related to the Environments API, shall we scope this discussion under that issue / thread?

Yeah makes sense - it's basically "last minute API design before release" 🙂

[atomic orchid]

There's a current limitation to build agents which is to lose track of the root context / core api / etc. Shall we provide a stopgap in the meantime?

not sure there's a meaningful different between "stopgap" and "not stopgap" at this point, we just need to fix it

Can we push the privileges discussion a bit further down the road? Or we should start designing this scopes based on that?

at the moment there's an argument to dag.LLM() called withQuery. It won't make sense to anyone. So no we can't push back the design of it. Otherwise we'll need to support it forever

[rapid timber]

thinking about adding a selectQuery kind of tool to allow the LLM to have access to the core API in the meantime until we flesh out how this plays with the Environments API

[rapid timber]

name to be bikesheed fwiw

Keep track of core API issue

[atomic orchid]

• Option 1: we have selectQuery except with a different name (eg. selectSystemAPI or something that would be "familiar" to the LLM, should be easy to bikeshed)

• Option 2: we special-case query functions, and always overlay them on the current selection (would grow the baseline cost)

[rapid timber]

I like 1 and seems like it was along the lines to what @charred quiver was suggesting

[atomic orchid]

Honestly it's not user-facing, so purely MCP implementation detail I think

(in other words, whichever performs best)

Separately I care about the user-facing API for this, ie. dag.LLM(withQuery: Bool!)

[rapid timber]

👍 I'll add that to the environments API thread once we resolve the implementation part 🙏

[atomic orchid]

selectQuery

[rapid timber]

I've synced with Alex and we'll go with the selectQuery tool given the trade-offs. Will try to get something working tomorrow morning 🙏

[charred quiver]

just a quick note to reinforce: the name of this tool should really be determined by what works for the LLM - selectQuery probably won't work, but maybe selectSystemAPI or even deselect or something would work