#Why doesn't `ANTHROPIC_API_KEY` get loaded automatically in my module?

1 messages · Page 1 of 1 (latest)

steady dawn
#

I have a simple module where I am testing making calls to multiple LLMs. In the shell when I call the llm primitive I have no problem as it just uses my env var that is already loaded. When I try to run my custom module however I get an error.

Module code:

package main

import (
    "context"
    "dagger/dagger-agents/internal/dagger"
)

type DaggerAgents struct{}

func (da *DaggerAgents) MultiLLM(ctx context.Context) *dagger.Container {
    environment := dag.Env().
        WithLLMInput("llm", dag.LLM(), "LLM for prompting various different AI models").
        WithContainerOutput("result", "A container with the result")

    result := dag.LLM().
        WithEnv(environment).
        WithModel("claude-3-7-sonnet-latest").
        WithPrompt(`
    Ask the following models to write a poem about a beaver:

    claude-3-5-sonnet-latest
    gemini-2.5-pro-preview-03-25
    gpt-4o

    Then return all poems annotated with their model name
    `)

    return result.Env().Output("result").AsContainer()
}

Output:

~/Workspace/dagger-agents via 🐹 v1.23.8 took 31s
❯ dagger
Dagger interactive shell. Type ".help" for more information. Press Ctrl+D to exit.

✔ llm | with-model claude-3-7-sonnet-latest | with-prompt "hello" 1.2s
│🧑 hello
│ ┃ 0.0s
│
│🤖 Hello! How can I assist you today? I'm here to help with information, answer questions, or discuss various topics. What would you like to talk about?
│ ┃ 1.2s
LLM@xxh3:b29c94ac3721cdb5

✘ multi-llm 0.6s
│🧑   Ask the following models to write a poem about a beaver:
│ ┃
│ ┃   claude-3-5-sonnet-latest
│ ┃   gemini-2.5-pro-preview-03-25
│ ┃   gpt-4o
│ ┃
│ ┃   Then return all poems annotated with their model name
│ ┃ 0.0s
│
│🤖 0.2s
│ ! POST "https://api.anthropic.com/v1/messages": 401 Unauthorized {"type":"error","error":{"type":"authentication_error","message":"x-api-key header is required"}}
! input: daggerAgents.multiLlm POST "https://api.anthropic.com/v1/messages": 401 Unauthorized {"type":"error","error":{"type":"authentication_error","message":"x-api-key header is required"}}

dagger-agents ⋈
tiny frigate
#

Unfortunately, there's no current workaround I think 😬

indigo ginkgo
#

@tiny frigate @steady dawn you can do dag.LLM(dagger.LLMOpts{Model: "claude-3-7-sonnet-latest"})

steady dawn
indigo ginkgo
#

I was going to say that you can configure your env to be "privileged", which means it will have access to the caller's context - core API, dependencies... but I just remembered that we currently hide llm from the environment even in privileged mode

#

@steady dawn you can create the 3 LLM instances yourself, and bind them as 3 separate inputs though

steady dawn
#

Thanks. I will try that

indigo ginkgo
#

you can name each binding after the model name if you want, to give the "orchestrator" llm a hint. Or you can leave it opaque, depending on what you want the LLM to know (the binding names and descriptions are included in the llm context)

steady dawn
#

Fixed it this way 🙂

package main

import (
    "context"
    "dagger/dagger-agents/internal/dagger"
    "fmt"
)

type DaggerAgents struct{}

func (da *DaggerAgents) MultiLLM(
    ctx context.Context,
    // list of models
    models []string,
) *dagger.LLM {
    environment := dag.Env().
        WithLLMOutput("result", "The result of the LLM call")

    for _, m := range models {
        environment = environment.WithLLMInput(m, dag.LLM(dagger.LLMOpts{Model: m}), fmt.Sprintf("LLM that uses the %s model", m))
    }

    result := dag.LLM(dagger.LLMOpts{Model: "claude-3-7-sonnet-latest"}).
        WithEnv(environment).
        WithPrompt(`
        Call each of the LLMs available to you and prompt them to write a haiku about frogs

    Then return all poems annotated with their model name
    `)

    return result.Env().Output("result").AsLLM()
}
indigo ginkgo
#

That will do it for the inputs 🙂 But not clear what happens with that output?

#

It will be forced to choose one llm output to return I think

tiny frigate
#

that's why he had to refactor the code to pass the model as a function argument

#

the initial example was a multi agent workflow which tries to use the llm core type to call other models

indigo ginkgo
#

@tiny frigate you should catch up the full history

tiny frigate
#

@steady dawn you can create the 3 LLM instances yourself, and bind them as 3 separate inputs though

@indigo ginkgo this you mean?

#

you can do this, still I perceive it differently since you need a code change to support a new model. In the original approach you could hint the multi-llm to use a new model by just changing the prompt.

I'd assume that once the WithModel issue is fixed, this should work as initially intended, correct?

indigo ginkgo
tiny frigate
#

I guess it really depends on what the OP is expecting here 😄

steady dawn
#

Totally unrelated but how would I rewrite this function to just return the final result to stdout instead of AsLLM()?

indigo ginkgo
#

You can get the LLM's last reply message with LLM.lastReply()

#

What format do you want to use to combine the multiple haikus? Concatenated in a single string?

#

One thing you can do, is add a Directory output, and instruct the LLM to write each haiku as a distinct file in the same directory, with <model>.txt as the filename. Then you get that directory out

#

Or, you could have as many outputs as inputs - one per haiku 🙂

steady dawn
#

Ah ok. Yeah just wanted all the haikus listed in the same final response

indigo ginkgo
#

Yeah then lastReply() is probably the simplest. We also added Env.withStringOutput in main, so you can get a string value that way

steady dawn
#

What does adding an output to the Env do? I had a "result" llm output originally but whenever I tried to get lastReply from that output it didn't give the last reply from the parent llm but from one of input llms...

indigo ginkgo
#

yeah your output is of type LLM which is pretty meta...

#

you're literally asking the main llm to return a a value of type llm

#

(I'll call the main llm the "agent" for clarity here)

#

My guess is that the agent is not sure what you're expecting (ambiguous instructions) so it returns any object of type LLM (it has a bunch at its disposal)

steady dawn
#

Ah ok so an llm can manipulate an output as a tool

indigo ginkgo
#
  • inputs are values you provide to the llm, read-only
  • outputs are values you expect the llm to return to you. the description of those outputs is super important, because it's basically a declarative prompt
#

That's the key, to realize that when you define your agent's inputs and outputs, you're constructing both the tools available to the agent, but also its prompt. Because the name and description of the inputs and outputs will be readable to the llm, and will influence what it does.

If you craft your names and descriptions carefully, you may barely need a top-level prompt at all, beyond something generic like "use all available tools to return the expected outputs to the user"

steady dawn
#

Awesome. Thanks! I think I'm finally getting the hang of dagger 🙂

#

Any ETA on external MCP support?

indigo ginkgo
#

It's actually already shipped, experimental, but pretty buggy still. We're polishing it as we speak

#

dagger mcp is a valid stdio dagger mcp server but don't tell anyone 🤫

#

At the moment it only works with goose (that we know of) because it makes heavy use of dynamic tool selection, which is in the spec but most clients don't support yet (including Anthropic's own client!)

steady dawn
#

Ah I meant using third-party MCP servers from within dagger

indigo ginkgo
#

Ah. Also work in progress but not shipped yet

#

I'm curious which MCP servers you would be interested in using?

steady dawn
#

Nothing specific. I've just been building and playing around with them and also maintain a Go SDK for MCP. Would be cool to be able to attach them to my agents.