#@vito questions about llm/mcp stuff in

1 messages · Page 1 of 1 (latest)

neat holly
#

Is there any reliance on call.ID being reloaded from one session in another? Like a call.ID gets persisted somewhere, the main client disconnects, a new client comes in and connects and then reloads old IDs from old sessions?

#

I can't really tell what's going on in that code, especially core/mcp.go lol

normal jetty
#

Not currently, but I'm working on persistent (user local dir) LLM sessions on my llm-workspace branch, which will involve that yeah.

#

there's an LLM.__withObject("Container#1", <generic ID type>) as part of that, which becomes how the MCP code persists objects in the session (right now on main it's in-memory), but this is all highly experimental

#

but tbh, the way things are going, we're kind of trending away from having the LLM persist and directly interact with objects anyway. so I'm curious if there are challenges you're worried about with it

neat holly
#

Okay yeah, I'm asking because like I mentioned in meeting earlier, I am changing how recipes get stored entirely (they still exist, just not in a string that client's pass around, it's all centralized in the server cache). Which simplifies an outrageous amount of things

  • can explain further if interested, but the recipes currently "infect" any object result stored as a field on a core / user module type, which then propagates confusion/complication of passing them around and caching the ID vs. caching the result of the ID further, etc.

It actually shouldn't immediately break any attempt to store a call.ID on a local dir, it's just that the call.ID will by default be an int that's an index to the cache. Which can be re-loaded but won't be guaranteed to still exist because of pruning.

I did add a new optional arg to .id and .sync that let's you trigger the full recipe call.ID, but mainly because that's useful in integration tests and/or debugging. I guess you could use it here if you wanted to

#

Caveat being that storing recipes is obviously problematic once you hit anything from a client

#

Probably the right long-term solution here would be to have some deeper integration such that the engine refuses to prune something that a client has exported externally and still wants saved, like in your use case here

normal jetty
#

ah interesting - so are IDs still some sort of base64 encoded protobuf thing, but they can be 'shallow' (contains int index) or 'deep' (today's giant strings)? depending on how you call .id/.sync?

neat holly
#

In the cache PR we have a DAG of "sharedResults" (shared because multiple recipes can end up at the same underlying result). Previously I was trying to use call.ID with those but it devolved into utter insanity, especially with persistence in the DB.

Once I switched those to natively store call metadata and do the various digest'ing that used to be on call.ID themselves "natively", everything started making sense again.

normal jetty
#

nice

#

well - the LLM save/load stuff doesn't exist yet so feel free to ignore it, i'll do a reality rebase assuming your PR lands first 😛

#

worst case ontario i can just save LLM sessions in an ID-decoupled format even, just a JSON log of the LLM request/response history, and re-create it when reloading, the IDs are just convenient for skipping all that

neat holly
normal jetty