@Yves porting his agents to 0.18 🧵 | Dagger | Page 1

open dust Apr 1, 2025, 4:42 PM

#

@versed shard @zealous bison @stuck plinth @regal cobalt we have some papercuts while porting Yves' module to 0.18

#

Looks like a bug when retrieving outputs

versed shard Apr 1, 2025, 4:43 PM

#

anything needed for v0.18.1?

final veldt Apr 1, 2025, 4:45 PM

#

So, this is my code:

env := dag.Env().
        WithAlpineWorkspaceInput("workspace", dag.AlpineWorkspace(source), "Alpine Environment with the source codebase mounted").
        WithAlpineWorkspaceOutput("result", "Workspace with the development tools installed")

    resEnv := dag.LLM().
        WithEnv(env).
        WithPromptFile(dag.CurrentModule().Source().File("qwen_dev_env.md")).
        Env()

And then, if I try to get the outputs through resEnv.Outputs(ctx) for instance (and to loop on them), the binding has no name and type void.
If I resEnv.Output("result") it says binding is nil

#

Not sure what I'm doing wrong
I can push the code if that's easier to look at

grizzled junco Apr 1, 2025, 4:54 PM

#

@final veldt yes please

final veldt Apr 1, 2025, 5:05 PM

#

Here is the diff of the migration to 0.18 https://github.com/eunomie/local-agent/commit/9e573759e7af3a212698993ace54b0d55c53134b

GitHub

migrate to env API · eunomie/local-agent@9e57375

grizzled junco Apr 1, 2025, 5:47 PM

#

@versed shard One thing i noticed is that there's a WithPrompt method on LLMSession but no WithPromptFile on LLMSession is that expected ?

#

the withprompt on llmsession does a syncvar before passing to lower-level llm.withprompt. WithPromptFile only exists on the lower level llm.WithPromptFile

versed shard Apr 1, 2025, 5:48 PM

#

no reason it couldn't be added, that type is just to facilitate the REPL so things are added as-needed

grizzled junco Apr 1, 2025, 5:48 PM

#

yes but regarding syncvar i'm not sure what's expected

#

also i'm vibe analysing here, didn't test my theory

versed shard Apr 1, 2025, 5:50 PM

#

ah think i'm missing context - what's your end goal? is it related to the issue yves hit? that one seems like it's happening in code, not the repl?

grizzled junco Apr 1, 2025, 5:53 PM

#

yes, debugging yves

#

i'm gonna confirm or infirm my theory soon

versed shard Apr 1, 2025, 5:54 PM

#

you can do this now too: agent=$($agent | with-prompt-file ./foo)

grizzled junco Apr 1, 2025, 5:54 PM

#

will just try WithPrompt(file.Contents()) instead of WithPromptFile(file)

grizzled junco Apr 1, 2025, 6:12 PM

#

@versed shard small UX nit but i just got tricked by the shell completion when i didn't see there was a env | with-string-input because i would have had to scroll down to see it but didn't know there was more to see.

#

also why is there no with-string-output ?

versed shard Apr 1, 2025, 6:15 PM

#

@final veldt fwiw I ran your module and it worked for me. I wonder if whatever model you used didn't call return? https://v3.dagger.cloud/dagger/traces/62baebe5b999b2eecced6358253df455

#

which is an issue some of them have - they just stop talking, instead of calling return to complete the task and return the outputs

final veldt Apr 1, 2025, 6:16 PM

#

I'm using this one: eunomie/qwen2.5-coder-14b-instruct:q5_k_m

#

What should be returned by the model as what I'd like is the container updated, so not something returned by the model, no?

versed shard Apr 1, 2025, 6:22 PM

#

grizzled junco also why is there no with-string-output ?

currently only objects can be returned. strings and other types could be added, but we need to figure out where to draw the line - once you consider arrays for example there's an explosion of complexity. there's also potential danger in allowing it - for example LLMs are very bad at accurately reproducing text that you've given them, if there are escape sequences or whitespace or [...], but generating "from scratch"/"from prompt" is presumeably fine

grizzled junco Apr 1, 2025, 6:23 PM

#

ok so for now i can do with-file-output insteaad of string

versed shard Apr 1, 2025, 6:24 PM

#

final veldt What should be returned by the model as what I'd like is the container updated, ...

when you declare outputs, the model has to call a return tool to fulfill them, like return {"result":"Workspace#1"} - it's transparent to you as a user, but the mechanism relies on the model to follow that instruction correctly, and some don't reliably follow it at the moment. it's likely that quen2.5 is among those, as even claude-3.5-sonnet was refusing to at times (though mysteriously seems to consistently use it now)

versed shard Apr 1, 2025, 6:25 PM

#

grizzled junco ok so for now i can do with-file-output insteaad of string

you can also use last-reply and just have it reply with the desired text, but that might be subject to markdown/"human" formatting or something, depending on what you're trying to do

grizzled junco Apr 1, 2025, 6:30 PM

#

i'll be adding more env-related questions to the Environment API thread

#

i think i found the issue

#

Missing key field on Output binding

#

the thing is i'm unable to do a repro of Yves' yet, so i'm still just vibing here

final veldt Apr 1, 2025, 6:36 PM

#

If you have the latest Docker Desktop, you can enable Docker Model Runner feature (in experimental), docker model pull eunomie/qwen2.5-coder-14b-instruct:q5_k_m and that should work out of the box with the .env in the repository
(but it means to run the model locally on your machine, and it only supports apple silicon)

grizzled junco Apr 1, 2025, 6:43 PM

#

#agents message

#

fwiw https://github.com/dagger/dagger/pull/10046 i'm still not convinced this is the only issue but it could fix your issue Yves

GitHub

env: fix missing output binding key by tiborvass · Pull Request #1...

With this fix, env | with-file-output foo description | output name now returns foo instead returning an empty result.

grizzled junco Apr 1, 2025, 7:10 PM

#

My theory about withPrompt vs withPromptFile didn't hold, withPromptFile worked in my small repro attempt, so right now i'm just hoping my fix makes it work for you @final veldt . Sorry I can't pull the model and test it right now.

Are you able to build dagger main ? If so let us know if you still have the issue.

final veldt Apr 1, 2025, 7:10 PM

#

I just tried, and it's not better, sadly 😕

#

I'm trying with a bigger model (gpt) just to see if that's better

grizzled junco Apr 1, 2025, 7:11 PM

#

ok, then you may still try my stupid theory: replace WithPromptFile with WithPrompt

#

the only difference in the codebase is a suspicious function called syncVarsToLLM

versed shard Apr 1, 2025, 7:12 PM

#

@final veldt do you have any Cloud traces I can look at? that would surface whether it's the return-not-being-called issue

#

(you could also see that locally but it may be hard to see what's not showing up but should :P)

final veldt Apr 1, 2025, 7:18 PM

#

With GPT (works): https://dagger.cloud/eunomie/traces/360038ec393fb34512703b94be36f43d (there's a return at the end)
With my local model: https://dagger.cloud/eunomie/traces/6bdb31b2263eabb290fa62a51842e6d (no return)

#

That works 🎉

#

Returns the modified development environment workspace.

Always return the modified development environment workspace

#

At least with main, I'll try now with 0.18.1

#

Sorry, that was a prompt issue, but thanks a lot for your ideas and 👀

versed shard Apr 1, 2025, 7:25 PM

#

final veldt Sorry, that was a prompt issue, but thanks a lot for your ideas and 👀

not your fault at all - that's on us, we need to make sure the models understand this. looking into it! maybe we can add a README tool for basic things like that

#

will test locally with the same model

final veldt Apr 1, 2025, 7:26 PM

#

working on 0.18.1 too 🎉
I can demo that tomorrow then 🙂

#

Now I have an other demo that didn't worked, but I'll see if I can fix it

grizzled junco Apr 1, 2025, 7:27 PM

#

I don't have access to the traces, what was the issue ?

final veldt Apr 1, 2025, 7:28 PM

#

Here is the fix: https://github.com/eunomie/local-agent/commit/a71e19a0173a0d87671fd571e6f4c93adae89595 🙂

GitHub

be sure the model returns the result · eunomie/local-agent@a71e19a

grizzled junco Apr 1, 2025, 7:29 PM

#

yeah it's a bit annoying to have to handhold the llm and that it can't figure it out from the provided env keys' description.

final veldt Apr 1, 2025, 7:54 PM

#

Is that possible to use a model with the new env, with inputs but no outputs? Like I want the LLM to summarize some stuff and get back with the response, the lastReply should be enough, right?

versed shard Apr 1, 2025, 7:55 PM

#

final veldt Is that possible to use a model with the new env, with inputs but no outputs? Li...

yep

versed shard Apr 1, 2025, 9:22 PM

#

final veldt I'm using this one: `eunomie/qwen2.5-coder-14b-instruct:q5_k_m`

curious - what led you to this particular model? seems very very specific, i'm still learning the ropes with the open ones

#

i'm making progress getting qwen2.5-coder:14b to call return but now it's calling it too eagerly lol

final veldt Apr 1, 2025, 9:25 PM

#

Well, I started to use this kind of models while trying to have it generate code.
And when you run locally, there's a lot of bad surprises, models not able to call tools, able but not calling them, etc. And even with two sizes (like between the 14b and the 7b) differences can be huge.

#

So the smaller the model, the more specific you want it

versed shard Apr 1, 2025, 9:28 PM

#

🧑 💬 Research and record a finding three times. Don't return until you have three findings.
🤖 🛠️ selectTestspace {"id":"Testspace#1"}
🪙 Tokens Used: 583 in => 29 out
🛠️ 💬 {"selected":"Testspace#1"}
🤖 🛠️ Testspace_research {}
🪙 Tokens Used: 708 in => 18 out
🛠️ 💬 {"result":"How interesting! The current time is 2025-04-01 21:25:34.913142257 +0000 UTC m=+0.089612706."}
🤖 🛠️ Testspace_withFinding {"finding":"The current time is 2025-04-01 21:25:34.913142257 +0000 UTC m=+0.089612706."}
🪙 Tokens Used: 788 in => 73 out
🛠️ 💬 {"previous":"Testspace#1","selected":"Testspace#2"}
🤖 🛠️ return {"out":"Testspace#2"}
🪙 Tokens Used: 821 in => 31 out
🛠️ 💬 ok

versed shard Apr 1, 2025, 9:30 PM

#

final veldt So the smaller the model, the more specific you want it

any idea what the q5_k_m part means?

final veldt Apr 1, 2025, 9:32 PM

#

This is the quantization. 5 is a sweet spot, not as exact as a full model, but good. q8 is almost true. <q4 is lighter, but less precise. And k_m is optimized for metal (apple) if I'm right

#@Yves porting his agents to 0.18 🧵