general

[bitter sierra]

@sand sierra I do export as well and it work's nice here, all files exists beforehand trough, just tested with another export directory /tmp/dagger_plugins all files are owned by me.

Now that I think about this, I'm a normal user I have no permission to write files as root so I think dagger as well.

[sand sierra]

You guys are absolutely right. I am trying to reproduce my issue but it works

Sorry for wasting your time

😔

[sharp marsh]

np! glad that worked!

[tired moth]

Which library dagger uses for tui?

[sick monolith]

A quick look, i think its https://github.com/dagger/dagger/blob/main/go.mod#L77 - https://github.com/vito/progrock by @elfin frigate

Im not sure if the tui is progrock or... midterm, one relies on the other

Looks like they use something else too, http://github.com/charmbracelet/bubbletea

https://github.com/dagger/dagger/blob/main/go.mod#L13

[elfin frigate]

yeah now it's just bubbletea - no more progrock; if it's in the dependency tree still, it's probably from something silly. the TUI implementation lives under dagql/idtui (somewhat confusingly)

[tired moth]

@sick monolith @elfin frigate ok thanks a lot for clarifying.

[analog raven]

Community Call starting in a few minutes!

[cunning jolt]

Can you share a youtube link please? I don't see it in the channel

[analog raven]

https://www.youtube.com/@dagger-io/streams

[cunning jolt]

huh, interesting, not seeing it there either. I did get a notification since I'm subscribed but I can't find the actual live stream

[analog raven]

try this one -> https://www.youtube.com/watch?v=LbUgs1UMQcE

[cunning jolt]

is this on my side?

[analog raven]

Hmmm, it might be.. it's working on my end. Are you able to watch on X? https://x.com/i/broadcasts/1lPKqODawmbJb

[cunning jolt]

No acces to X 😦

[analog raven]

Linkedin? https://www.linkedin.com/events/daggercommunitycall7232102438793584640/theater/

[cunning jolt]

ok I can see on linkedin 🙂 Wonder what restricted mode on YT is. I don't have a problem seeing other streams

[queen bramble]

Any plans on daggerizing the internal steps that are currently referenced in the config file (e.g. the coverage run or even the generation step) as another module to be triggered?

[slender star]

@analog raven @chilly arch 👆

[inland coral]

@cunning jolt Did you try toggling the feature as is suggested here or is that locked down by your employer?

We'll look in to the Youtube studio to make sure it's not set as adult content etc.

[queen bramble]

@solomon I must have missed this interface thing. Does that mean a function can expose a structured input (interface definition) so that a different module can create a corresponding object and pass it over?! Is it this one?https://docs.dagger.io/manuals/developer/interfaces/ ? Cross-language?

[narrow nymph]

yes ❤️

exactly that

[tired moth]

does dagger cloud tracing coming to tui soon?

[cunning jolt]

I tried turning off restricted mode but it instantly returns back to "on". So I suspect my company started enforcing it when on network 😦

[narrow nymph]

i think only go can accept and declare interface types, but any language can consume the returned types

i know there's ongoing work to expand this support though 🎉

[elfin frigate]

unsure what you mean here - the TUI is representing the same info, but in a format more suitable for the TUI. but, you can expect more improvements soon as the two UIs converge - lots of work in flight on both

👾

[tired moth]

where i can track that work? and after this implementation dagger cloud tracing experience will be as same as TUI also?

[cunning jolt]

I agree, I don't use interfaces that way myself!

[inland coral]

Ok, I'm not sure why the stream is getting flagged...we'll dive into the stream settings in YT studio to see if there's anything causing it to get flagged. Glad you're able to see it via linked-in though. Sorry you're experiencing this.

[winter linden]

That was great @karmic sand ! <--- That's Kambui from the community call demo btw

[split olive]

Thank you all!

[elfin frigate]

most of it is in our private Linear or in my head, sorry - but yes, you can expect both the TUI and Cloud to have the same features, at least as close as possible, and hopefully with shorter turnaround time if this WebAssembly experiment pays off (looking good so far). That being said there may be features that are only in Cloud just because it's where historical data is kept, so it won't be 100% parity, but we want switching between them to feel as natural as possible

[tired moth]

ok cool but which WASM experiment? please share detail or issue link

[slender charm]

CI Env Abstraction

[winter linden]

This is an experiment in Dagger Cloud, which is not open source. If/when we ship it, we will post about it to share details

(it's cool enough to deserve its own blog post tbh)

[tired moth]

ok cool, we would like to learn how to ship such things in web. thank you

[cunning jolt]

omg this is so exciting! Can't wait and I truly hope it works out

[winter linden]

Which part are you excited about?

[cunning jolt]

The TUI being close (or in parity) to what the Cloud traces are showing today. I think that will be a big selling point for Dagger.

[winter linden]

I expect it's the web version that will benefit the most

[tired moth]

dagger tracing ui in cloud is costom build or any library is used for that?

[elfin frigate]

here's the real feature parity you can expect in Cloud

[elfin frigate]

right now the Cloud UI is all custom TypeScript + React with a re-implementation of the TUI logic, but it's not 1:1 parity. with WebAssembly it'll import the same code as the TUI to cut out all the extra implementation effort, and use https://go-app.dev for the actual frontend parts

[tired moth]

ok got it, awesome. thanks

[cunning jolt]

This is great! So this can bring best of both worlds to TUI and the Cloud UI.

[coarse iris]

Hello all. I have the following shell script that I run using dagger:

export MY_ENV="123456"
printenv MY_ENV
echo "MY ENV is " $MY_ENV

Which prints this output:

123456
MY ENV is    (blank)```
Is it possible to set your own env variables that are not exported to dagger inside the dagger container?

[cunning jolt]

Is it possible to set your own env variables that are not exported to dagger inside the dagger container?

Do you mean without using WithEnvVariable?

[coarse iris]

yes, right inside of the command / script I run

[sharp marsh]

👋 that's because echo doesn't really understand about variables, it's the shell that expands those. If you change your WithExec to ["sh", "-c", "echo $MY_ENV"] that should work

[coarse iris]

what I actually do I export this script as a file to the dagger container and then I run this file like this:

WithExec("bash -c sh /tmp/script.sh")```

and it doesn't work

[sharp marsh]

const script = `
export MY_ENV="123456"
printenv MY_ENV
echo "MY ENV is " $MY_ENV
`

func (m *Lala) Test(ctx context.Context) (string, error) {
    return dag.Container().From("alpine").
        WithNewFile("script.sh", script).
        WithExec([]string{"apk", "add", "bash"}).
        WithExec([]string{"bash", "-c", "sh script.sh"}).Stdout(ctx)

}

^ this worked

dagger call test
Full trace at https://dagger.cloud/marcos-test/traces/104e1dc1aa1323171345f3caebbf41f1

✔ connect 1.1s
✔ initialize 3.2s
✔ prepare 0.0s
✔ lala: Lala! 0.0s
✔ Lala.test: String! 2.0s

123456
MY ENV is  123456

having said that, the bash -c is redundant. You can just call ["sh", "script.sh"]

[coarse iris]

thank you, will check my code, it has to be not connected with dagger then

[candid gyro]

It seems like the current behavior is that at default verbosity (without -s), fmt.Println (and equivalent) messages are going to be swallowed by the TTY progress output (unless the function errors out). They're visible if plain but tty is better UX. Are there perhaps any plans for introducing some kind of user output / log API for reporting back to users in a friendly way that would be visible during --progress=tty ? I could also see wanting to even show the ouput if -s , with -s being scoped specifically progress (dagger engine tracing). Any thoughts around this ?

[sick monolith]

dagger -h does not show dagger -m as an option, but the docs still show``dagger -m 'uri' call...`

dagger call -m is the correct way right?

[sick monolith]

example: https://docs.dagger.io/manuals/user/chaining

[sick monolith]

@winter linden you recently showed daggerverse docs with "examples", but theres nothing on the docs/daggerverse explaining how to do that. I had a look at your gha 'examples/go' but not sure how to do an example for 'shell' or something else. Tried to find daggerverse things in github but couldntr find anything to peak how i should go about that. if i make an examples/shell folder... does that even work?

[sick monolith]

ah, i think i know how it works, that shell tab wont ever do anything, since those examples are literally dagger module functions i nthe example folder.

[sand sierra]

What does exclude do in the dagger.json file? If i provide my root directory with --root . as a *dagger.Directory will it not copy the excludes?

[sick monolith]

i think it works like the docker ignore file

for example im using python sdk, but i wanted intellisense with pycharm and to use venv... that makes a folder like .venv and .idea, so you can add those to the exclude section

https://docs.dagger.io/manuals/developer/ide-integration

there's a TIP section which explains

[sand sierra]

I thought so also

But If i just share a directory

with --dir .

It also copies the exclude

[sick monolith]

your dagger module is usually in a folder like .dagger and your src might be in somehting like src

[sand sierra]

Yeah i've tried both

[sick monolith]

hmm

[sand sierra]

and using .dagger

and using the . dirctory

I can work around it

but its a little more tedious

Still love the expierience

but it has a few rough edges

[sick monolith]

https://docs.dagger.io/cookbook/#examples-2

there's something on this example about exclusions too, not sure if this helps you though.

[winter linden]

the excludes in dagger.json are only for the loading of the module itself. It's to avoid uploading unnecessary files when your module mixes dagger and non-dagger files

[sand sierra]

Ah ok tnx for the information

I've fixed it for now in my code 🙂

[sick monolith]

something feels weird/off with trying to install a dependency module above (i.e using solomons gha examples folder approach. I am getting strange context / directory issues

something like : failed to collect local module source deps: failed to collect local module source dep: local module dep source path "../.." escapes context "/mnt/c/Users/patri/Documents/Projects/dagger-prompt/examples/python" not sure if its a windows/linux thing. Tried a bunch of different things

[sick monolith]

yeah, not sure how to actually make a subfolder of 'examples' and install a folder dir as a dependency module. I even tried making a folder outside of the main module, and couldnt get that to work either. So i thoguth mayube it was a bug introduced, downgraded to the same dagger version in solomans example 0.12.5 but still no luck. So im not sure whats going on there. 😦

[winter linden]

@sick monolith here's what I did:

1. mkdir -p examples/go
2. cd examples/go
3. dagger init --sdk=go --source=.
4. dagger install ../..
5. Implement the examples module (make sure to name your functions correctly so that Daggerverse picks them up)

[sick monolith]

step 4 fails

ok hold on, let me go back to 0.12.7

[winter linden]

did you mistakingly do the reverse before? Like dagger install ./examples/go from the main module?

[sick monolith]

i deffo did that too

failed to create module: select: failed to update module dependencies: module "examples" dependency "" with source root path "..\\.." does not exist or does not have a configuration file

hmmm

This is the 'main module'

{
  "name": "prompt",
  "sdk": "python",
  "exclude": [
    ".venv",
    ".idea"
  ],
  "source": ".",
  "engineVersion": "v0.12.7"
}

this is the 'python' example dagger.json

{
  "name": "examples",
  "sdk": "python",
  "exclude": [
    ".venv",
    ".idea"
  ],
  "source": ".",
  "engineVersion": "v0.12.7"
}

[sharp marsh]

yes, but once call is invoked, it doesn't really matter if -m if either before or after the call command, Dagger picks it up regardless.

[sharp marsh]

@sick monolith dagger-prompt needs to be a git repo. That's the only way dagger install will actually allow you install ../../ as it need to know what the scope of the project is

have you executed git init in the dagger-prompt foldear already?

[sick monolith]

I did not, because my thought was to get things running locally prior to doing any gitting!

This certainly needs documenting 😦

I'm going to try what you mentioned shortly, hopefully that fixes it for me

[sharp marsh]

yes, agree. cc @heavy karma

[sick monolith]

i even think the cli should somehow detect and report no git when trying to install dependencies

got it working on the mac after doing git init. However.. Windows is borked: ! failed to create module: select: failed to update module dependencies: module "examples" dependency "" with source root path "examples/python/..\\.." does not exist or does not have a configuration file

[cunning jolt]

I agree.

[sharp marsh]

cc @swift inlet mind checking this one out now that you have a Windows machine? 😛

[sick monolith]

https://github.com/dagger/dagger/issues/8363

[swift inlet]

yes taking a look

[sick monolith]

there is a workaorund with WSL, open wsl, do what you need to do to get codegen done and then you can swap back to windows IDE

since wsl dagger (linux ) will use linux pathing and so you can kinda workaorund it for now, so not entirely blocking if users have wsl

[sick monolith]

.5 needs looking into. I know it's still kinda rough or draft. but there's no documentation on this at the moment. I couldn't get python examples working so I made a go example and it was picked up. I assume this is still a wip so not even sure if I should be raising an issue on this as its not documented, or I could t find the documentation

[winter linden]

Yes we are missing docs on this.. Sorry

[unreal latch]

what port(s) does engine use?
_EXPERIMENTAL_DAGGER_RUNNER_HOST=tcp://<address:port> if i want to run dagger container in another host.

[modest oasis]

hello im new in dagger, i would like to know if requires local setup for certain configurations like dagger.jsonif is possible generate the dagger.json without cloning the repo and dagger init --sdk=python --source=./dagger ?

[sharp marsh]

by default the engine doesn't start with a TCP port. You have to assign it yourself via the --addr flag when starting the engine.

[sharp marsh]

hey, Welcome! Not sure I understood what you're trying to achieve here 🤔

[modest oasis]

heym thanks for reply i try to run a dagger pipeline on azure, i got an error "Error: failed to get configured module: no dagger.json found in directory . or any parents up to git root", i have set up all over azure devops direct on the repo, i wonder, do i need to clone the repo and run the "dagger init --sdk=python --source=./dagger" in order to create the .json i need or i can generated dynamically on the azure devops directly on the pipeline ? or is needed a local set up in order to generate this .jsonv?

[sharp marsh]

ok, thx for clarifying. The idea is that you commit your dagger.json into your repo so after it's cloned, you can directly call dagger call my_function.

[modest oasis]

i got it, I still requires local setup for certain configurations like dagger.json. This contrasts with traditional CI/CD tools that allow pipelines to be defined entirely via YAML files directly in the repository. i need clone create this .jason and then push , how does dagger works in a cloud antive env where devs not allow to modify the pipeliens or other config files, for example generate this .json not allowed because of company policys? as i understood dagger function like terraform or pulumi where you need do a pre work locally, plese correct me if im wrong

[sharp marsh]

ok, I see what you mean. The way that you can generally make this work in a more controlled environment is the following:

1. Imaginate that you have two repositories, the github.com/app one and the github.com/ci one. The app repository is owned the application developers than should only commit code. The ci repo, is owned the the devops / infra team and they define their Dagger functions there so the rest of the organization can use them.

2. You trigger a CI pipeline with the app repo context and then using the dagger CLI, you cand do dagger -m github.com/ci call my_function. What this does is to call a Dagger function that lives in the ci repo where your teams can have access to create the dagger.json file and the corresponding pipeline there.

[modest oasis]

i seee, understatood, thanks man, i will deo a PoC and see how it goes, thanks, i hope you have a good day

[sharp marsh]

sure, np! You too!

[winter linden]

@modest oasis I think there may be a confusion. You mention dagger.json is different from traditional CI/CD yaml configs, but in terms of lifecycle, it's very similar. You commit a configuration to the repo, then your CI system can run it.

[stoic knot]

feedback: with new account and no builds, there is a spinning indicator that never goes away

[sudden ocean]

Does +ignore directive in Go SDK support multi-line list?

[stoic knot]

feedback: The link when there is no data to display is broken

feedback: after initial oauth login, I'm sent to the /<org>/runs endpoint, where there appear to be no builds

I noticed that the terminal output includes the /<org>/traces endpoint, and once I'm there I see the traces

From the initial login, there is no UI link to get to traces

[narrow nymph]

Ah not yet, if you could open an issue for it (you can assign it to me, or just ping me, etc), that would be awesome 😎 I was meaning to take a look at that, but it's a bit fiddly, we might not be able to use the current regexp we're doing atm 😂

[sudden ocean]

Sounds like we need a small parser to parse the list. Looks fun. 😄

[narrow nymph]

Yeah exactly, I love my little parsers 🎉

[sudden ocean]

Here it is https://github.com/dagger/dagger/issues/8383

[narrow nymph]

Aside: have you ever read crafting interpreters? It's a favorite, all about making languages from scratch

[sharp marsh]

I read / practiced the creating an interpreter in Go from Thorsten. Had a fun time going through that one

[vague aspen]

Good evening there, I would like to kindly ask you for help. I am developing POC pipeline in python, and it looks like everytime I make a change in the code, dagger calls uv and downloads/checks python dependencies again.
Is it possible to skip this step? I was looking in the documentation, github and in the #1030538312508776540 channel, without any sucess.
Thank you very much
Jozef

[sudden ocean]

Never read it before but at the moment I reading create interpreter in Go. 🙂

[stoic knot]

https://docs.dagger.io/cookbook#cache-application-dependencies

The other thing to note is that you only want to add the dependency tracking files, not all the source. This way, the uv only runs when dependencies have changed

[vague aspen]

THX I will try it

[sharp marsh]

feedback: after initial oauth login, I'm

[sharp marsh]

feedback: with new account and no builds

[stoic knot]

We have concurrent yarn install commands that sometimes error in our CI workers. (conflict with the shared cache in the user home dir).

In our POC Dagger code, we have a mounted cache for dependencies per language. Should I expect this error to manifest in the same way if multiple Dagger builds (on the same worker) use the same mounted cache?

[winter linden]

Yes, tools that don't handle cache concurrency well (like yarn) will have the same issue if they share a cache volume. Ideally you'd manage that in the pipeline logic

[stoic knot]

Is it now possible to conditionally go into a Terminal session with a Dagger container after a build error occurs?!

• https://docs.dagger.io/cookbook#continue-using-a-container-after-command-execution-fails
• https://docs.dagger.io/cookbook#debug-container-builds

[sharp marsh]

Yes, the -i flag in dagger call enables that

[stoic knot]

Do I need the -i if I'm using dagger run and doing the checks internal to our code? Like is that a requirement to use Terminal, or is -i something that opens a terminal an fail without code changes?

[sharp marsh]

The last part -i is mostly for debugging purposes. It'll open the terminal as soon as a withExec fails without any code changes

[vague aspen]

✔ connect 0.7s
✔ initialize 19.1s <---

I meant initialize phase

[worthy ivy]

Hi i try to run the quickstart hello dagger. i got this error

tls: failed to verify certificate: x509: certificate signed by unknown authority

how can i ignore verify tls or to put ca as valid

[vague aspen]

hello, you have to build your own dagger engine image
for example

FROM ubuntu:jammy AS ubuntu
FROM registry.dagger.io/engine:v0.12.7 AS dagger-engine

# Add CA certificate
COPY --from=ubuntu /usr/share/ca-certificates/company-ca/* /usr/local/share/ca-certificates
RUN update-ca-certificates

ADD Dockerfile /Dockerfile

then export it via env variable before executing dagger

export _EXPERIMENTAL_DAGGER_RUNNER_HOST=docker-image://localhost/jj/dagger-custom-engine:v0.2.17
dagger call ...

more can be found in https://docs.dagger.io/manuals/administrator/engine

[vague aspen]

the error message looks like some fishy situation

[worthy ivy]

ok, actually I run on mac with z-scaler (it from work)

if I do own dagger image i cant use with cli local computer. correct?

[vague aspen]

you can use it, but you have to upload image into some kind of internal docker image repository, well it is tricky... and I am not that far, I am basicaly one step before you 🙂

[sick monolith]

https://discord.com/channels/707636530424053791/1281388151738208329

[modest oasis]

hi, hope your doing well, based on what you suggest i created a new repo and run the "dagger init --sdk=python --source=./dagger" this created a init.pu inside teh src/main directory, i try to call the function fro the default code is there "@object_type
class Pipelines:
@function
def container_echo(self, string_arg: str) -> dagger.Container:
"""Returns a container that echoes whatever string argument is provided"""
return dag.container().from_("alpine:latest").with_exec(["echo", string_arg])"

i run the following command :

dagger call container_echo --source=. -vv

got this error Error: unknown command "container_echo" for "dagger call"

[lapis slate]

Hello @modest oasis 👋
If you call the dagger functions command what output do you get?

[modest oasis]

in that example the function is "container_echo" correct me if im wrong, the i run the comand "dagger call container_echo --source=. -vv" i got this out put:

Error: unknown command "container_echo" for "dagger call", i followed the intruction on the documetnation i cant make it works i also created another .py file with a function called build same error
@object_type
class MyPipeline:
@function
def build

same error

[lapis slate]

in that example the function is "

[slender charm]

This is really cool, I didnt realize it worked with dagger run as well!

[storm verge]

has anyone investigated or tried using dagger in a harness pipeline ?

[bitter sierra]

Is there a +defaultSecret like +defaultPath ?

[winter linden]

not yet! But auth is on our list

it will probably have to be more restricted, because auth access is more dangerous than same-repository access

[bitter sierra]

Howto debug Publish() -- "invoke: input: container.from.withDirectory.withFile.withUser.withWorkdir.withFile.withFile.withDirectory.withEntrypoint.withRegistryAuth.publish resolve: failed to export: failed to push registry.example.com/project/container:2332 invalid reference format"

[bitter sierra]

I need that for registry access 🙂

[bitter sierra]

Ignore that, I have no use-case for defaultSecret, sorry.

I was thinking I need to provide a default but I don't luckily.

[raven stag]

Hey, dagger pins dependencies (dagger install xxx) to the SHA of the specific commit when the install was done. I've seen https://github.com/dagger/dagger/issues/7898 which talks around the dev experience on seeing the tag that initially was used to install the dependency.
Is it possible in any way, or considerable for the future, to allow pinning to a tag but not the SHA? (I've left a comment on that discussion with more context but feels that It wasn't the right place for the topic 😛 )

[glacial hearth]

Is it possible to call dagger modules and functions from code, or is it only from the dagger cli?

[winter linden]

Yes 100%, that's a big feature of Dagger. Everything goes through the same API. You can call it from the CLI or from code

@glacial hearth when developing your own module:

• dagger install ADDRESS_OF_MODULE
• This will automatically generated client bindings for your language, in your module
• In your module implementation: dag.Foo() and all available functions should auto-complete, with the right types
• This works regardless of your module's language, and of your dependency module's language

[glacial hearth]

Ahh, cool, so I can call the function via: client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stdout))
if err != nil {
panic(err)
}
defer client.Close()

client.Moduler().....

[winter linden]

At the moment the codegen for other modules is only available when developing a module yourself. If you're developing a custom client (which you seem to be) then unfortunately you won't get the generated bindings. That's something we plan on adding.

In your case, that would mean moving your client code from client-side code (probably a custom CLI?) to server-side code (a Dagger Module to extend the Dagger API)

[glacial hearth]

Ok, but if I do dagger install ADDRESS_OF_MODULE, how would you then call a function?

[winter linden]

You call the functions via the generated bindings added by dagger install. In the Go SDK they go in internal/dagger/dagger.gen.go for example

@glacial hearth this step in the quickstart gives a good practical example: https://docs.dagger.io/quickstart/simplify

[glacial hearth]

Ahh, ok.. so I need to be in the "dagger" folder, outside existing code. I have some automation code in golang, and was looking to call dagger module. It is still cool thogh it can be done from the dagger cli client... Maybe for now I could do something like "exec.Command(dagger...."

Thanks solomon, I will have a look and wrap my head around this cool daggerverse 🙂

[sand gorge]

btw, on our docs I don't think we have a way to set a SINGLE env var on a container. The only example here is with MULTIPLE env vars, but then the code complexity goes up.

https://docs.dagger.io/cookbook#set-environment-variables-in-a-container

Do we have a code example for this?

with_env_variable

Thanks 🙂

[glacial hearth]

dagger init --sdk=go --source=./dagger

in main.go:

type Projects struct{}

func (m *Projects) GetHello(ctx context.Context) (string, error) {
return dag.Container().
From("alpine:latest").
WithExec([]string{"sh", "-c", "echo hello"}).
Stdout(ctx)
}

This works:
dagger call get-hello

This remote does not work:
dagger call -m github.com/<user>/projects --get-hello

Any idears?

[sharp marsh]

there's some examples here: https://docs.dagger.io/manuals/developer/functions/#return-values

[sand gorge]

Ok, someone's gotta add it to cookbook, because the natural way users are using the docs, is to open cookbook, and just search for things on that page.

You have the complex example (multiple) but are missing singular

[sharp marsh]

👋 is that a public or private repo?

[glacial hearth]

Private, I did "export GITHUB_TOKEN=<token">

[sharp marsh]

feedback taken. We need to find a good use-case for that as the idea of the cookbook is mostly for use-cases. We can't add a cookbook entry for each of the different Dagger API calls cc @heavy karma

[sharp marsh]

this just landed today https://docs.dagger.io/manuals/user/remote-repositories. cc @swift inlet

[sand gorge]

Fair enough Marco, but people aren't going to click around the docs in lots of pages to find their thing.. they land on a page, and can just CTRL+F for something they need.

Especially for the basic stuff when they're just learning .. they don't know where to go, gotta spoon feed them

Your call really 👍

[sharp marsh]

yes, I totally agree also. Maybe the cookbook is not the right place. An alterantive could be using variables somehow in the quickstart so users are exposed that concept earlier in the Dagger journey

[sand gorge]

indeed.

[glacial hearth]

dagger -m github.com/<user>/projects call --get-hello

Error: failed to get configured module: failed to get module ref kind: input: moduleSource resolve: failed to resolve git src to commit: failed to load cache key: error fetching default branch for repository https://github.com/<user>/projects: git error: exit status 128
stderr:
fatal: could not read Username for 'https://github.com': terminal prompts disabled

Create token:
Administration -> Access: Read-only
Contents -> Access: Read and write
Metadata -> Access: Read-only

[swift inlet]

Hey, is this a private repo?

[glacial hearth]

Hey, yes, I have done "export GITHUB_TOKEN=<token>" and created the token with these: Create token:
Administration -> Access: Read-only
Contents -> Access: Read and write
Metadata -> Access: Read-only

[swift inlet]

PAT support is ongoing. Currently, dagger can only access modules stored on private git repositories through SSH refs: https://docs.dagger.io/manuals/user/remote-repositories/#supported-protocols-and-schemes-1

[glacial hearth]

Correct, well... well same thing with public repo.. It is in this repo: https://github.com/JCzz/projects

[sand gorge]

Hey! Secret vs No Secret? 🤔

This postgres module - the credentials for user/pass are defined as dagger.Secret

https://daggerverse.dev/mod/github.com/grouville/daggerverse-tom/postgres@2b30c86c46201e470748df42505da8e77af3bd54#Postgres.user

I actually don't want a secret, I just have Strings here, because this is a tutorial and I'm not wanting to over-complicate things right now ..

Shall I just ditch this plugin and manually put together my postgres Service?

Opinions?

UPDATE: I found another module, so I'll just use this one as it takes strings - https://daggerverse.dev/mod/github.com/kpenfound/dagger-modules/postgres@57352e06a1cfbcb5307c009d37c0201b2719b935#Postgres.client

But I was curious about the conversation in general, really.

[sick monolith]

what makes dagger.Secret complicated?

i'd personally keep secret, its a great feature and shows security is treated as a 1st class citizen

dagger call example --user=cmd:"echo user" --password=cmd:"echo password" - I think that might be good for an example

[slender charm]

Nice catch, I am surprised to see this, I only ever use WithEnvVar, I didnt even know about the other option. Feels like the cookbook should show both ways

Added an issue here:https://github.com/dagger/dagger/issues/8439

[sand gorge]

It might be good for an example .. but not for my example ..

plus there are no command line passwords, because the DB DSN URL is an environment variable set up, rather than user provided password at call time.

Hope that made sense?

[sand gorge]

nicely done 🙌

[slender charm]

It feels odd to me to show multiple but not single, that section feels like an obvious place to add both to me

I actually didnt even know about the multiple one... I only ever use single

I agree with your general point, but this feels like such a common operation that its worth adding IMO, especially in a section that already talks about this topic

[glacial hearth]

I can see errors in dagger.gen.go, should I re generate orsomething?

[sick monolith]

so do you mean it wouldnt be good to show dagger call command --secret=env:$MY_SECRET ?

i suppose depends on what parts your demo is going to show

[sand gorge]

@sick monolith that is an option .. and yes like you said .. it depends on which "part" of the tutorial/learning process we're at ..

I'm right at Dagger 101, so I'm not adding more stuff .. yet ..

[sick monolith]

func (m *Spikes) Secret(ctx context.Context, user dagger.Secret) (string, error) {
    val, _ := user.Plaintext(ctx)
    return dag.Container().
        From("alpine:latest").
        WithExec([]string{"echo", val}).
        Stdout(ctx)
}

seems concise to me, but maybe you can do your first steps "without secrets" then someone may say "heyy what about security" etc etc. then you can do a before/after showing the minor differences. Then its not too much info overload when theyre first presented dagger as a new tech

[sand gorge]

I completed agree with you 🙂

[sharp marsh]

yes, agreer that adding both examples there is totally worth it

[sand gorge]

I'm using AI to scan the dagger docs and find me what I wanna know 😛

[sand gorge]

Hey @wispy tapir I found a bug in your postgres module - this env var should be POSTGRES_DB not POSTGRES_DATABASE 🙂

I just spend FAR to long debugging my code, trying to understand why my stuff doesn't work 😅

https://github.com/quartz-technology/daggerverse/blob/627fc4df7de8ce3bd8710fa08ea2db6cf16712b3/postgres/database.go#L23C30-L23C47

[winter linden]

@elfin frigate dagger --no-exit is

We should have made that a topic in the 0.13 release post honestly

I didn't realize it was already merged!

dagger -E call ... -> if your pipeline fails, you stay in the TUI, and you can navigate the trace to inspect the failure

[elfin frigate]

glad ya like it! now we just need a keymap equivalent for when you realize too late haha

[winter linden]

https://tenor.com/view/cher-turn-back-time-time-machine-oops-oh-well-gif-17650330

[raven stag]

Random question: Are there plans to add defaults to the socket type arguments? (Like Directory type has right now)

[winter linden]

Dagger 0.13 is out! we have lots of goodness in this release:

• First-class support for monorepos, thanks to context directory access and pre-call filtering
• Support for private modules! I know many of you have been asking for that!
• A new CLI command: dagger core to call the core Dagger API directly from the command-line. Great for debugging, experimentation, and quick one-liners
• dagger -no-exit (-E for short) which lets you stay in the TUI after the call is complete. Great for investigating an error!
• A module called compatchecker, which helps you... check your module for compatibility (love a nice descriptive name)
• Faster Typescript SDK, with more performance improvements coming soon
• And more.

Check it out at https://dagger.io/blog/dagger-0-13

Please consider tweeting, posting, etc. etc. We appreciate your support!

[glacial hearth]

Do dagger keep some caches around, I have some errors that was for an older project with same name. I have tried: rm -rf ~/.dagger
and docker system prune -a

[sick monolith]

is it bad to admit i only just noticed now that the dagger logo is an impression of a rocket launch.

i was looking at the twitter post and it just clicked in my mind

[sick monolith]

adds to the weight of daggerverse needing to show 'tested' modules with examples somehow, like a quality gate

[sand gorge]

YEAH BABY!!! 🥳

-E is 🔥 so you can hang around in the daggerverse for just that bit longer, after pipeline completes

[wispy tapir]

daggerverse/postgres/database.go at 627f...

[inland coral]

Ha, I'm glad someone got what I was going for with those launch pictures...not fully on brand but trying to subtly alude to it...

[bitter sierra]

My PoC is done. Auto Deploy to Kubernetes with OSS only:

Software:

• dagger
• k3s + rancher
• gitea
• harbor (goharbor)
• drone
• argocd

I'll write step-by-step guide now.

[sharp marsh]

how happy are you with the outcome?

[bitter sierra]

Well it does what I wan't, I push to a specific branch and about 5 minutes later it's live 🙂 I'm not happy about the complexity of the pipeline. Not sure there's an easier way to make that happen.

#!/usr/bin/env bash
IFS=',' read -r -a DOMAINS <<< "$1"
SHA=$2

if [[ ! -e ~/bin/yq ]]; then
    VERSION=v4.2.0 BINARY=yq_linux_amd64
    wget https://github.com/mikefarah/yq/releases/download/${VERSION}/${BINARY} -O ~/bin/yq
    chmod +x ~/bin/yq
fi

git -c credential.helper='!f() { echo "username=${GIT_USER}"; echo "password=${GIT_PASSWORD}"; }; f' clone https://git.example.com/Project/chart.git
pushd chart/Project

for DOMAIN in "${DOMAINS[@]}"
do
    ~/bin/yq e -i '.frontend.image.tag = "'${SHA}'"' ${DOMAIN}-values.yaml
    ~/bin/yq e -i '.backend.image.tag = "'${SHA}'"' ${DOMAIN}-values.yaml
    git add ${DOMAIN}-values.yaml
done

cd ..

git commit -m "${SHA}"
git -c credential.helper='!f() { echo "username=${GIT_USER}"; echo "password=${GIT_PASSWORD}"; }; f' push

popd
rm -rf chart

This little script is very important in my pipeline, it patches the example.com-values.yaml with the new image tag.

[sick monolith]

I don't see the full picture here obviously, but just seeing some things which come to my mind. wget - there is a dag.Http function, which can return you a dagger.File. You could then mount that to a dagger.Container.

[bitter sierra]

This is my .drone.yml (the pipeline):

---
kind: pipeline
type: exec
name: default

platform:
  os: linux
  arch: amd64

steps:
steps:
- name: submodules
  commands:
    - git submodule update --init --recursive

- name: Install dagger
  commands:
    # Install dagger
    - curl -fsS https://dl.dagger.io/dagger/install.sh | BIN_DIR=$HOME/bin DAGGER_VERSION=v0.12.7 sh

- name: Build and publish the backend image.
  environment:
    REGISTRY:
      from_secret: registry
    REGISTRY_IMAGE:
      from_secret: registry_image
    REGISTRY_USER:
      from_secret: registry_user
    REGISTRY_TOKEN:
      from_secret: registry_token
  commands:
    # Run dagger
    - $HOME/bin/dagger call publish --source=. --reg="$REGISTRY" --reg-image="$REGISTRY_IMAGE" --reg-user="$REGISTRY_USER" --reg-token="env:REGISTRY_TOKEN" --reg-tag="$(echo ${DRONE_COMMIT_SHA} | cut -b -7)"

- name: Build and publish the frontend image.
  environment:
    DOMAINS:
      from_secret: domains
    REGISTRY:
      from_secret: registry
    REGISTRY_IMAGE:
      from_secret: registry_frontend_image
    REGISTRY_USER:
      from_secret: registry_user
    REGISTRY_TOKEN:
      from_secret: registry_token
  commands:
    # Run dagger
    - /bin/bash ./.drone/dagger-publish.sh "$DOMAINS" "$(echo ${DRONE_COMMIT_SHA} | cut -b -7)"

- name: Stop dagger
  commands:
    # Stop dagger
    - docker stop -t 300 $$(docker ps --filter name="dagger-engine-*" -q)

- name: Update chart
  environment:
    DOMAINS:
      from_secret: domains
    GIT_USER: drone
    GIT_PASSWORD: 
      from_secret: git_password
  commands:
    - /bin/bash ./.drone/argocd.sh "$DOMAINS" "$(echo ${DRONE_COMMIT_SHA} | cut -b -7)"

trigger:
  branch:
    - drone
  event:
    - push

[sick monolith]

there' also a dag.Git which you can use in order to mount/extract files

[bitter sierra]

dagger-publish.sh

#!/usr/bin/env bash

IFS="," read -r -a DOMAINS <<< "$1"
SHA=$2

for DOMAIN in "${DOMAINS[@]}"; do
    $HOME/bin/dagger call frontend-publish --source=. --backend="$DOMAIN" --reg="$REGISTRY" --reg-image="$REGISTRY_IMAGE" --reg-user="$REGISTRY_USER" --reg-token="env:REGISTRY_TOKEN" --reg-tag="$SHA"
done

So... I have to write a complete Guide now, instead of spreading peaces around.

[bitter sierra]

I like you ideas, for now I'm out-of-energy to touch that stuff again.

[sick monolith]

https://tenor.com/view/its-friday-nick-cage-tgif-gif-14296081

[bitter sierra]

@sick monolith do you think you could replace the whole file with Dagger code?

Tomorrow is another day so .... 🙂

[sick monolith]

I think you could, BUT, it doesnt mean you SHOULD, you condensed the yaml from your main pipeline into something portable, and thats the most important piece here.

from an observation what I would say, and this more my gut feel, you could likely utilise some other core dagger primitives, if something is a file, dag.File, if you're writing loops - could that be written in the language sdk as a loop, as apposed to in the bash. You have some other CLI usage in that bash, some may or may not be available as dagger modules, like yq. - But a strong note that this is obsevations, and i think the more exposure to dagger you get, how you see/approach things will change over time, and i say that because that's what happened with me.

[sharp marsh]

this is generally the recommended approach to start with Dagger. Hopefully, as time progreses, you'll start feeling that force within you that will hopefully push you towards Daggerizing the rest of the pipeline 😄

[bitter sierra]

Maybe I gonna write a reusable gitops module that does:

• git clone
• patch values
• git push

Without any external deps

[sharp marsh]

that'd be awesome. @rigid pebble might be interested also

[bitter sierra]

https://github.com/mikefarah/yq <-- is go so easy to embed.

[rigid pebble]

We did something like that for deploying some of our apps, using https://daggerverse.dev/mod/github.com/matipan/daggerverse/image-updater@dc37011c6578e44b83e61976ca92336350d96b02

[sand gorge]

Last night I introduced my friend to Dagger. Gave him a little workshop.

He was mesmerized.

Today he sent me this.

We have another fan 😁

[bitter sierra]

@rigid pebble thanks, I'll copy&paste some stuff for my helm + argocd approach 🙂

@rigid pebble if you want I show you the whole approach via screen-sharing.

[thick igloo]

this is the only way we can get a failing github action 😅 forgetting to align the dagger version of the action with the one in the code 😂

[pine crag]

hey folks! does anyone here use Preview Environments part of their CI setup with (or without) dagger? We recently created a Dagger module which helps you to set up Preview environments pretty easily with Dagger. If this is something you're interested in, please let me know I can help you get started. More details can be found on #okteto

[cerulean wraith]

👋 hello folks. Over the weekend, I found myself in the need of having a proxy to spin-up my dev environment for a personal project, and I implemented a very simple caddy module to reverse-proxy the requests using Caddy: https://github.com/rajatjindal/daggerverse/tree/main/caddy

[sand gorge]

sounds good 🙂

[sand gorge]

Hey. Can we take graphql calls and generate code out of them?

Like if you build it in Golang. And then can somehow convert that code to PHP because we know the graphql calls

[sharp marsh]

you mean generate a GraphQL client out of the Dagger GraphQL API?

[sand gorge]

no. Not make the client, but the end-code ... if I have golang code here .. I could generate PHP code out of that golang code, if we can write a generator from the graphql spec.

It's similar to what we do currently, in the SDK's themselves, but generating ->withExec()'s and ->from()'s pertaining to each specific language

like python has @function, and PHP has [#DaggerFunction]

make sense? (i hope :D)

[vestal creek]

Why would you want to generate a module in another language when Dagger can consume modules in any language?

[sand gorge]

I'm not generating a module, the context is for writing tutorials and documentation.

If there are code examples in one language and I want to generate the same code in another langugae .. then I need to convert it by hand .. whereas we can have a generator do the work for us, one that understands the language-specific implementation that we've built.

An AI tool won't work, as it doesn't understand all the little details about our SDK implementation.

The information is there, in the graphql already, and I'm wondering if someone has already implemented this yet?

[winter linden]

Unfortunately depending on the example, not all the information is there. An example might involve several GraphQL queries, with values being copied from one to the other in the client code.

[sand gorge]

Okay, yeah you're right. Maybe not worth it then 🤔

[sand gorge]

Apart from our homepage, is there a page that answers the question of "why dagger?"

[sick monolith]

there is an inkling of that answer here: https://docs.dagger.io/adopting#look-for-red-flags

And on the main site, i suppose also is a 'why' but not part of the docs: https://dagger.io/dagger-engine

[sand gorge]

cool, thanks a lot

[sharp marsh]

Gotcha, we faced the same challenge while implementing the Daggerverse example support and we landed that eventually with LLMs should eventually generalize that pretty well even by looking at very little of Dagger code. And as Solomon mentioned above, not all the information was there so we can implement this in a straightforward way

[winter linden]

My guess is that with enough context, LLMs can solve this

With a dozen or so manually crafted multi-language examples, maybe the raw api reference in the context, I bet the new o1 model could get to 90% success rate

[sand gorge]

yeah! let's see how things evolve. exciting times ahead! 😄

[frozen basin]

Back in the disc after a long time. Reading through this spec on caching and exploring the new dagger-engine local-cache API . Very cool stuff guys

Revisited some old build pipelines and revamped with cache vols, we've cut some of our build times from ~5-6hrs to 1.5 (which is about the min due to some other constraints).

Loving the progress! 🔥

[lime snow]

Is there a way for me to export the generated /schema.json with --interactive? Basically I'm looking for an --on-fail-copy-this-file=schema.json

[inland coral]

all the case studies. Select case studies from resources page... Our users do the best job articulating the value of dagger 😉

[sand gorge]

Chris, I've been looking at --export recently. Thinking of how to get stuff out of the container. It's not "on fail" it will copy it evey time.

But you can check the return code of dagger call to see if it's failure or not.

Does this help?

[sick monolith]

export a dir. add the file if there was a failure. then you only need 1 dagger call. that could also work?

[prisma magnet]

I worked a bit with docker container testing using dagger. While checking e.g. files or envs was pretty straight forward I did not find a way to check out running processes. I.e. I want to check that a process inside is actually running. Is there a way to do this?

looking at services they did not look like they would solve that, since I can only interact with them via network, but cannot look inside

[lime snow]

Unfortunately not in this instance, I'm trying to install another module and the schema generation is failing. I need to get hold of that file, preferably without trying to export the entire container

[sick monolith]

you dont need to export the entire container. You can export a file, or a directory

function do-something -> dagger.Directory (.... do things, return a dir of the container) - grab a file out of a container in a dagger function, or a log file etc dependong on exit code, then add that file to a dag.directory and return the dir.

export exists on container, to tar the image, also on a dir to export a folder within the container or on an individual file

[chilly arch]

New blog post alert - inspired by @solar raptor's community call demo 🚀 Thanks to @slender star for updating the modules too!

Multi-Cloud Container Publishing with Dagger Functions - https://dagger.io/blog/multi-cloud-container-publishing

[lime snow]

Hi Patrick, thank you for the help that would work on a custom function.

The issue is I am calling:dagger install <my-test-module>. So I cannot change it to return a directory or anything, it's a built in function.

I can get into the container with dagger install <my-test-module> --interactive but the /schema.json file is very long... It would just be handy to be able to export that file.

[sick monolith]

interesting... so the dagger install is only when you want to call that from another module. can you not provide a container result from that module and then do some wrapping in the consuming module to get the file out? also what's the use case here? is this simply debugging a module you are designing?

If the dagger install is failing... I feel like that's a dagger bug or issue not giving feedback on why it failed.

[lime snow]

I'm developing a feature for the PHP SDK and checking if it works by installing it on another module (with both based on my branch)

So... its a bug in my local branch... working on it!

[chilly arch]

🚀 Dagger Community Call Tomorrow! 🚀

Join us for our bi-weekly Dagger Community Call to explore exciting updates and connect with fellow Daggernauts!

🔍 What’s on the agenda?

• Meet Devs Where They Are: Why IaC Must Be Real Code by @slender star
• Dagger 0.13 Recap by @winter linden and team

📅 Date: Thursday, September 19th
🕘 Time: 9:00 am PT

You can join the live stream on LinkedIn or Twitter.

👉 Add the series to your calendar: https://calendar.google.com/calendar/u/0?cid=Y19mZDEyYWRjOTI3ZTQ0MDVkMGY2NmUyZjA2Yjk0OTI3YzI4ZjA2YzJkMWQzZDFiOTRlNTIwMjZkNGU0MjMzNWQxQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20

See you there!

[tired moth]

Q1: How can Dagger help achieve zero dependency on IaC and other methods when interacting with Google Cloud products solely through their API? Could creating a Dagger module for the REST API to orchestrate Google Cloud products be a solution?

[chilly arch]

Community Call live now! You can join from LinkedIn, X, or YouTube:

https://www.youtube.com/watch?v=JmcwIPLgeVA

https://www.linkedin.com/posts/dagger-io_join-the-dagger-team-and-fellow-daggernauts-activity-7236783070874378242-SgL4?utm_source=share&utm_medium=member_desktop

https://twitter.com/dagger_io/status/1836797901898744087

[tired moth]

cant' we get real state through api only before applying changes, is it good way to use dagger for this?

[chilly arch]

@slender star 👀

Here is the demo that Jeremy just shared on the call - https://github.com/jpadams/daggerized-pulumi-examples-aws-ts-containers-dockerbuild

[chilly arch]

If anyone missed the 0.13 recap, you can check out the details here - https://dagger.io/blog/dagger-0-13

[timber sphinx]

renovate preset for dagger

[chilly arch]

AWS re:Invent 2024 - you going?

[thick igloo]

hi there, anyone with a templating utility module ready out there? pretty simple in my mind, just like File in File out and some way to pass values to fill in. What do think? Does it even make sense?

[narrow nymph]

yup, that definitely makes sense! I don't think there's anything out there as far as i know, but if you're using go, i'd give go templates a go? That would be a pretty good place to get started imo

[thick igloo]

right, I'm also strongly biased by go templates 🙂 I'm giving it a try then, let's see

[wanton pumice]

silly question:
If i use cacheVolume in dagger, does it automatically create a cache in the dagger cloud, or only on the host machine (where dagger engine container is running)?

[winter linden]

If your engine is connected to Dagger Cloud, and your Cloud account has experimental distributed cache enabled, then by default each cache volume will be synced up to Cloud after the run completes (in an eventually consistent way - not in real time)

[wanton pumice]

Okay I see.
So by default, the dagger engine is not connected to the cloud. If I enable dagger cloud (by following the docs) and enable distributed cache, then it's synced to cloud.
otherwise, the cache is on the host where dagger engine is running

[winter linden]

Yes, correct. We are working on adding hooks to the engine, so that you can customize how the cache is replicated (ie. to avoid dependency on Dagger Cloud for distributed cache)

[wanton pumice]

okay, i understand.
thank you very much for taking the time to reply!

[slender charm]

No utility module but I have one that uses Jinja in case you need some inspiration on one approach to work with the generated file

I think this could use some more love/refactoring/abstraction but it works for me 🙂

https://github.com/levlaz/daggerverse/blob/b247aafe6d2204c4bdd2568ad75dabc7cd2ddf9d/miniflux/dagger/src/main/__init__.py

[thick igloo]

In fact, for once, I start thinking that type "semi-unsafety" would be handy in this case. Making something generic enough in Go is tricky...

[winter linden]

the most generic way to pass values would probably be a json string I'm guessing

[thick igloo]

Yeah, exactly.. and creating a dict on the fly would be way easier than a struct at that point I guess

[winter linden]

graphql doesn't have dict/maps unfortunately. so yes that's what probably the closest equivalent

[sharp marsh]

we could define our own scalar type in the engine to represent this as we have for the JSON value, right? https://docs.dagger.io/api/reference/#definition-JSON

[fossil pine]

How to Helm in Daggerland: https://github.com/sagikazarmark/demo-dagger-helm

Blog post WIP

[sudden otter]

Can I suggest having long commands in documentation in multiple lines, like this? They would be much more readable. I'm willing to send a PR, if you like the idea

The current one-line format forces to do a horizontal scrolling with the mouse, for each single example

[sick monolith]

those only work on bash.

can't copy paste into powershell since it uses backtick `

but I guess anyone using powershell like me gets karma for not running a Unix os I guess

[slender charm]

Ugh I never even thought about that, does that make reading a lot of docs miserable for you? 😭

Using \ is such a common thing I see.

[sick monolith]

yes and no

your windows users will not like it

your wsl users won't mind it

[sick monolith]

honestly why they decided to use ` over \ now I'm curious why they thought this was a good idea

oh my god. its because of Windows file path being \ instead of the Unix /

but I have noticed and my opinion as a windows users, the docs are heavily Unix only samples. another example is piping strings into the dagger cmd like graphql. it's different in powershell/cmd

but most windows users know the story. it's common in the industry. we decide to walk on broken ice and windows users just don't want to admit it.

[slender star]

I think of backtick in powershell as "the littlest backslash"

[sick monolith]

haha

the cute shell trying to be like a proper shell like bash.

it is cross platform now though!!

[slender star]

no disrespect, since powershell is a powerful beast, but maybe we can have a view option that flips ` and \

[sick monolith]

balance between bloat and clean docs though

I think it will add too much noise but I wonder if there's other cli tools with docs that do similar with windows/Unix samples

unless you mean a bit like a dark/light mode button in the top right

[slender charm]

I was thinking about using window.navigator in JS to detect the OS and then show you something else by default haha

[slender star]

Could be a logged-in docs experience with preferences. Anything is possible! 🤩

[sick monolith]

they might be a heavy wsl Linux user

[thick igloo]

Brief update on this, I'm experimenting with this https://daggerverse.dev/mod/github.com/dciangot/daggerverse/dagger-templates@bbefe24b817386ffcc4b64d6823c05e418d36a60 and looks pretty much what I was looking for. Comments, ideas?

[tawdry imp]

Hello! So following the latest dagger release with support for private git repos, I spent some time playing with it again, and it's awesome to (finally) be able to use private dagger modules. A little voice on twitter told me to share my feedback with @swift inlet 😉 so thanks for your work!
However, if you manage to make it work without the ssh-agent, it will be easier to use for us in the context of gitlab-ci jobs. it's not a big deal, but it requires a few more lines of YAML to prepare everything: install ssh-agent if needed, start it, add the key, etc. If we want to sell dagger as "1 cmd to do xyz", but you need 10 lines before for the setup...
anyway, thanks again, I've been waiting for this feature since last year, I'll be able to break down our monolith dagger pipeline into modules that are easier to re-use for different projects - and then spread some dagger love in other teams!

[lethal escarp]

Hello, I've been looking into using Dagger in my teams workflows. I'm mostly a golang dev and dagger was very straightforward for that but my team is co-joining into a bigger Java team and I have bit a of a knowledge gap I hope someone could fill in. That Java team uses some combination of maven and JIB, would there be any trouble trying to use JIB with Dagger?

[slender star]

Hello, I've been looking into using

[slender charm]

Hello! So following the latest dagger

[sudden otter]

Today I have introduced, for the first time, in an official productive .NET project, Dagger. I'm presenting it to the rest of the company in the next months.

[winter linden]

How exciting! Let us know how we can help. Do you see any potential obstacles that we can help with? Is the lack of a .Net SDK a problem, or not really?

[chilly arch]

That's great to hear @sudden otter ! Thank you for sharing. We look forward to hearing how your presentation goes.

If you need a Dagger intro deck to use for your company presentation, feel free to use any of these slides - https://docs.google.com/presentation/d/1q3pvstVED0i2zP5eNeHZcQWWFKUTZxOke4I2CMFY2dc/edit?usp=sharing

You can see an example of a Dagger pitch using similar slides here - https://youtu.be/Whp86W0afg0

[fossil pine]

I did a podcast episode with Kyle from Depot discussing how Dagger and Depot supercharged our CI pipelines: https://www.youtube.com/watch?v=4MNDCntVyq8

They did a follow up blog post: https://depot.dev/blog/comparing-github-actions-and-depot-runners-for-2x-faster-builds

And here is our original blog post: https://openmeter.io/blog/how-we-made-our-ci-pipeline-5x-faster

[dense gate]

how can we use the dagger private module in Github actions? tried to call the private module using https://github.com/dagger/dagger-for-github/tree/v6/ but it failed..I passed module as [e-mail]:org/repo.git/path/to/module in Github action.

[raven stag]

Am I correct in assuming optional args are not supported in dagger interfaces?

(at least, not in the golang sdk?)

[sharp marsh]

Hey Aditi! Do you have a trace by any chance? The way to do this currently is to use SSH keys until PAT is supported (https://github.com/dagger/dagger/issues/7990). There's more info here: #1280877385805332550 message

[elfin frigate]

these should be supported - but you might have to set // +optional even if you're already setting // +default, not sure if that's fixed yet in the Go SDK. based on recent firsthand experience, I think not. relevant issue: https://github.com/dagger/dagger/issues/6749

[elfin galleon]

Hi there, I'm trying to get my company to buy into this thing, but hitting a hurdle with large files in the github repository, impossible to get rid of them as our company has a culture of using github LFS and checking in binaries of 160mb~400mb in size. I see there's a lot of stuff going on in the dagger ignore space but I can't figure out what direction it's going or if there's something I can adopt right now. To make matters worse, I have to use windows here and docker on windows is an absolute klunker, performance wise, particularly with respect to file operations. Can anyone hit me with the clue stick please 😄 (engine version v0.13.3)

[old lotus]

is there a way to annotate/curate the spans in dagger UI? I have a lot of spans that I don't really care about, and others I'd like to make easier to see or filter. I thought the container labels would be a good use for this.

[slender charm]

Custom Spans

[dense gate]

dont have traces..if you need i can regenerate it and give...but it seems first we need to setup ssh within github actions. After that we will be able to use private modules of dagger inside github pipeline..

[golden cypress]

I've used Dagger to build a couple of experimental pipelines, but I’m wondering if it can go beyond CI/CD. It seems to overlap with tools like Nix or devenv for setting up environments, and Docker Compose for running services. Is anyone working on using Dagger for these kinds of things, or is it better to focus on its strengths in CI/CD?

[thick igloo]

your not alone! I think is a common symptom after a long exposure to the open Daggerverse 😅 Joking aside, what I see very unique is the possibility to put together tests and dev environenment in a coherent way. Like, install dagger and you will have our suggested dev environment AND a test suite that is IDENTICAL to what runs on our CI pipelines. The next to next step is to say also "you can use it also as a CLI", that cannot apply all the times though

[fossil pine]

Here is what I did last: we are migrating from OpenAPI to TypeSpec (for obvious reasons). But we still generate SDKs from OpenAPI 😬 , so we need some backward compatibility. I wrote a function that runs openapi-changes (https://pb33f.io/openapi-changes/) to compare the old and the new OpenAPI, so we can discover breaking changes.

Source: https://github.com/openmeterio/openmeter/blob/main/.dagger/dev.go#L108
Module: https://daggerverse.dev/mod/github.com/sagikazarmark/daggerverse/openapi-changes

But I agree with @thick igloo : once you get hooked on Dagger, it's hard not try and pack everything in there.

Although watching for file changes and rerunning parts of the pipeline is really missing.

[slender charm]

Here is what I did last: we are

[coarse hatch]

On https://docs.dagger.io/manuals/developer/state-functions/ both the module fields and the module constructor's arguments are marked with pragmas but I don't see how the pragmas from the struct fields (Greeting in the example) would be taken into account since the New constructor already handles the pragmas. Is it a docs oversight or am I missing a subtlety ?

[slender charm]

Im hacking on some random GNOME stuff and every time I see a README like this is makes me want to make a dagger module just to specify what the actual dependencies are 🙂

https://gitlab.gnome.org/GNOME/evolution/-/wikis/Extensions

Got started on one here: https://github.com/levlaz/daggerverse/blob/main/evolution/src/main/main.py

I am on a mission to turn every complex README into a dagger module 😁

[celest grove]

Hey @slender star ! by any chance do you have these slides that you used here https://www.youtube.com/watch?v=Whp86W0afg0&t=67s 'shareable' somehow?

[slender star]

Introduction to Dagger

[winter linden]

Hi everyone, the Dagger team is spending the week together at our annual team retreat in sunny San Diego, California. The bad news is that we'll be slower than usual in answering questions. The good news is that we'll be working hard to ship many more improvements to you 🙂

[fossil pine]

Have fun!

[ancient escarp]

Do we know when 0.13.4 is planned for release?

[narrow nymph]

next week sometime, maybe tuesday? maybe wednesday?

[chilly arch]

Thanks to @high crystal , we have our latest Dagger Case Study published. Check it out here!

" Everybody's just happy to have the consistency problem solved. You can log on to that site or this site, or a cluster from a different customer, and all the deployment steps are always the same. Everything just works, thanks to Dagger."

https://dagger.io/blog/safespring-case-study

[elfin galleon]

I can't find a way (using dagger) to tag an image, but not push it. I want to be able to create an image, but I don't want to push it to a remote repository. This would be quite common when I'm working with just Docker locally, I haven't reached the stage of project development where I set up private repository + I'm itterating rapidly, I don't want to unnecessarily use up my (starlink) connection. Any suggestions? I could easily just run the docker build on it's own but it seems ugly to break out of the pattern of using dagger to interface with all container stuff.

[tired moth]

can anyone share a dagger powered monorepo example with typescript code and with TS SDK of dagger? PLEASE HELP

[sick monolith]

I dont know any but until someone can give you a good response i have an idea, though it would require some more searching to see...

Github advanced search

https://github.com/search?q=path%3A**%2Fdagger.json++typescript&type=code&ref=advsearch

this should give you all repos where there is a dagger.json file, and also typescript is found, hopefully sdk: "typescript", but this doesnt mean the entire repo is... but it might be a good way to find stuff out there on github that is public

[stoic knot]

I have a nice monorepo + Dagger TS setup, but it is

1. private
2. based on the SDK, not Dagger "functions", i.e. custom cli

I have a pattern of creating a single entrypoint at the root for all operations in my monorepos

So I can type ./dx build foo,bar or ./dx changeset start <name>, some of which is dagger, some of which is not, and we can do complex operations without devs having to remember a series of commands. The CLI aligns with the work tasks they perform

[tired moth]

@sick monolith ok thanks

[tired moth]

Hey guys we need help here #1291391958668939264 message

[chilly arch]

We hope everyone is having a great week!

No Dagger Community Call today as the team is at the retreat. The next call will be back on October 17th.

Add the call schedule to your calendar here: https://calendar.google.com/calendar/u/0?cid=Y19mZDEyYWRjOTI3ZTQ0MDVkMGY2NmUyZjA2Yjk0OTI3YzI4ZjA2YzJkMWQzZDFiOTRlNTIwMjZkNGU0MjMzNWQxQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20

[cunning jolt]

wow what a backdrop!

[winter linden]

We're cooking up some good stuff for you all, more soon 🙂

[tired moth]

Sneak peek, PLEASE...

[ancient escarp]

Interesting post/perspective on sustainable open source in light of the WordPress stuff that's been kicking off this week. https://dri.es/solving-the-maker-taker-problem

[slender charm]

Unexpected but delightful thing happened today when I noticed that the "w" link from the interactive terminal "just works" inside of CircleCI

[stoic knot]

Bluesky test?! Is Dagger going to join Bluesky/ATProto? 🙏

[slender charm]

No this is just one of my own modules that lets you post to bluesky from dagger. I have one for other social media sites too 🙂

https://daggerverse.dev/mod/github.com/levlaz/daggerverse/bluesky@dda735fd9f1effde4d7857e12594b2cb1dc3fd08

[stoic knot]

Dagger should join ;] Kelsey Hightower has said he tried to post to Bluesky first now (social proof?)

small feedback on Daggerverse, it would be great to view all of a user's modules from something like https://daggerverse.dev/mod/github.com/levlaz/daggerverse (i.e. removing the module name from the URL to get to an index, similar to how you can do this on github)

[warm temple]

Good idea! I've opened an issue https://github.com/dagger/dagger/issues/8648

[slender charm]

Are you on there? I’ll follow you 🙂

[stoic knot]

followed you

y'all be excited to hear I just got the green light for 2 weeks of daggerization

[slender charm]

That is awesome, let me know if you wanna pair on anything happy to do that any time 😄

[stoic knot]

sure, it might be more interesting for you to see how we handled a mega/poly repo by wrapping the SDK in a CLI. I have a pretty good handle on the Dagger aspects now, we worked through most of the IRL issues with the dev experience. I may end up writing something akin to Dagger functions (whereby I use Go introspection to find what commands can be run for a particular service (directory) in the repo)

The goal is to create something more end-to-end, so far we are mainly building binaries and then exporting them and then running docker build/tag/push on Dockerfiles. The aim is to run the full build/test/sonar/publish per service and then wrap the entire product in a single dagger call (which is where being able to separate out the services in the Dagger UI from a single connection would be helpful)

I have a feeling that building 20+ services in a single call is going to make the traces and logs pretty long (though that is how they look today in our current CI & Jenkins UI anyway...)

[frozen basin]

hey guys - wondering what ever happened to the graphql playground. Can I run that locally to see what modules I have loaded etc?

[frozen basin]

ahh finding dagger listen again, but forgetting where to find auth

[sharp marsh]

yo can set your own token with:
DAGGER_SESSION_TOKEN=foo dagger listen --listen localhost:8081
and then curl -u "foo:" localhost:8081/query

[frozen basin]

ty

[sharp marsh]

IIRC there's an issue to print the session token with dagger listen

[old lotus]

i like the recent release's (0.13.4) addition for services to have centrally defined hostnames. Would it make sense to also do something similar for mounts? Something like:

dag.CacheVolume("myvolume").WithPath("/shared/data")

[sharp marsh]

you can kind achieve that today using the With helper method in the Container type. For example:

func WithCache(name string) func(*dagger.Container) *dagger.Container {
    return func(c *dagger.Container) *dagger.Container {
        return c.WithMountedCache("/tmp/cache", dag.CacheVolume(name))
    }
}

func (m *Lala) Test() *dagger.Container {
    return dag.Container().From("alpine").
        With(WithCache("cache")).
        WithExec([]string{"echo", "Hello, world!"})

[old lotus]

yes understood that works. attaching it to the volume itself (like with the Service) would simplify my code a little bit, curious if it would for others.

[stray tendon]

Hi!

I wanted to ask what is (if there's any) the current method to run and scale dagger pipelines?
I know there's currently a issue discussing it but I wanted to hear how everyone else is doing it

And I'm wondering if it would be possible to create a cluster of dagger nodes (each node running an engine) where I'll be able to run the dagger cli from anywhere and it would schedule a job on the cluster?

I know there's k8s and friends but one issue I faced there is that if I'm using k8s based CI system, where it would create a pod each time to run a dagger pipeline, I would quickly hit the pod per node limit (which its like 110~), so its not really scalable in that case...

[fossil pine]

Blogged: https://openmeter.io/blog/supercharge-helm-chart-development-with-dagger

If you want to get your hands dirty: https://github.com/sagikazarmark/demo-dagger-helm

[sharp marsh]

Nice! that's definitely a use-case a lot of people with resonate with. Great writing mark!

btw, I recall we spoke about adding some features to the k3s module to simplify some of the part, right? I should open some issues so I don't forget about them 🙂

[sharp marsh]

https://github.com/marcosnils/daggerverse/issues/5

[fossil pine]

I actually started working on that, but ended up overcomplicating things and just threw it away. 😄

It would also be nice if Container.Publish supported name resolution, so we can ditch the skopeo part.

[sharp marsh]

yes! there's an issue for that IIRC. Shouldn't be very difficult to implement

[potent gate]

👋 for folks using Linux toolboxes, I created a gist on how to get the Dagger cli working inside a toolbox container: https://gist.github.com/adambkaplan/9b40970f0031c1e367d94b75d1d70bae

[smoky dock]

hello everyone, quick question about Dagger phylosophy

i'm doing a POC for me neeed with dagger, but one question come in mind.
do i need to handle with dagger which file are changed between my PR and MAIN or i let the CI/CD orchestrator (gitlab CI/GithuAction/ARgoWorflows) handle this for me ?

[slender star]

hello everyone, quick question about

[sturdy bluff]

Any daggernaut’s at hashiconf today?

[chilly arch]

🚀 Dagger Community Call Tomorrow! 🚀

🔍 What’s on the agenda?

• Deploying to Fly.io using Dagger and GitHub by @sage veldt
• PocketCI: end-to-end CI with no YAML in sight by @rigid pebble
• Scaling GenAI Inference with GPU-Optimized VMs on Shadeform by @thick igloo

Whether you’re new to Dagger or a seasoned pro, this is the perfect opportunity to stay up-to-date and get inspired!

📅 Date: Thursday, October 19th
🕘 Time: 9:00 am PT

👉 Add the series to your calendar:https://calendar.google.com/calendar/u/0?cid=Y19mZDEyYWRjOTI3ZTQ0MDVkMGY2NmUyZjA2Yjk0OTI3YzI4ZjA2YzJkMWQzZDFiOTRlNTIwMjZkNGU0MjMzNWQxQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20

[cunning jolt]

Will there be a sneak peek of the cool stuff @winter linden teased us about from the offsite? 🙂

[winter linden]

Ha ha, if there's time for it, we can certainly do that

Otherwise, we can do a separate demo 😛

[stoic knot]

@slender charm (or anyone else), I have an initial POC and some traces in the Cloud UI that I can show you. I noticed one thing for feedback (we have built,test,sonar steps, i.e. multiple WithExec, but only the last one shows up in the overview. I can see the logs in the details below). Can also give you an idea of how we are holding it and what our goals are. Monday would probably be ideal, want to get one more loop working to show you a bit more, and Tuesday is the POC deadline, and having your input for that will be helpful

[analog raven]

Starting in a couple minutes! See you all there

[cunning jolt]

I'm on linkedin and I can't read the text on the screen at all. Very blurry.

[chilly arch]

Thanks for the heads up! looking into it

shout out to @solar raptor 's Helix tool showing up at the Dagger demo. Love seeing Dagger and Helix together!

[humble sage]

Where is the community call chat ?

[slender charm]

Feedback Session

[chilly arch]

You can add comments here or on YouTube stream directly https://www.youtube.com/watch?v=mxFBOGh9jWg

[humble sage]

@chilly arch Thanks, I misunderstood that the chat was somewhere in Discord...

[chilly arch]

Sorry for the confusion. We like to take over this channel during the call 🙂

Dagger Module Diego is using - https://daggerverse.dev/mod/github.com/dciangot/daggerverse/shadeform@8dcb7ccd5a38bbe87f77b0ebe7b70aebd48eea11

dagger-templates Dagger Module - https://daggerverse.dev/mod/github.com/dciangot/daggerverse/dagger-templates@f0185187d0832b19d69a0d4cf074029da331f645

[sage veldt]

Watching Diegos demo and the curl's he's making in the functions ... did anyone try coming up with tool that takes OpenAPI spec and turns it into a dagger module?

[warm temple]

There's a few, here's one! https://daggerverse.dev/mod/github.com/shykes/daggerverse/daggy

I know @wraith niche has done a lot on this front too

[sage veldt]

oh, I actually meant OpenAPI (not OpenAI :)) -- so you take Swagger doc for example and it turns that into functions

[warm temple]

naming lol. I know that's been talked about a lot too but I don't know of any modules off the top of my head

[chilly arch]

GPUs, GPUs, GPUs :wink: :thread:

[chilly arch]

We should make a pre-requisite question for Dagger Community - Do you like Yaml or not?

[warm temple]

https://tenor.com/view/get-out-angry-speaking-head-tuco-squidward-gif-13808874276609302886

👆 if you say yes

(just kidding, stay and we will show you the way ❤️ )

[thick igloo]

Diego's demo misc links

[warm temple]

https://github.com/franela/pocketci

[chilly arch]

GHA Dagger Module - https://daggerverse.dev/mod/github.com/shykes/gha@bf36a0b37a75882e4a985179f090d86d2ad1dfb4

[cunning jolt]

lol! I'm still here 😄 But can wait

love it!

[sharp marsh]

@chilly arch presenters video is overlapping with the terminal 🙏

[warm temple]

I tried to sacrifice myself and it resized 😂

Now it's good

[chilly arch]

Sorry, was trying to make the terminal larger.

[cunning jolt]

Thank you for doing that demo

I can see usage of the shell (repl?) for development. I still can't envision how it benefits consumption of modules. But you did mention that it will become the way it's called in CI etc, so I'm curious to see examples and try it out.

[chilly arch]

PR from demo - https://github.com/dagger/dagger/pull/8611

[chilly arch]

If you missed the Community Call today, you can find the recording here. We'll share the demo snippets soon - https://youtube.com/live/mxFBOGh9jWg

Thank you to everyone who joined live!

[analog cape]

@warm temple heya friend, hope you're doing well 🙂
While I just passed by to say hi during today's community call, I heard part of discussion around GPU (GPU GPU ... did Solomon appear already? 😅 ) and I quickly checked wolfi packages (image attached).
To potentially avoid moving to far off the current alpine image you use, wolfi could be a solution instead of going "bigger" image. (but really just an idea, didn't got the whole point and I might be really confused about what was talked about)

[thick igloo]

That sounds cool! If someone is trying, count me in!

[sage veldt]

i’ll try to prototype something over the weekend and let you know!

[slender charm]

Dagger really feels like magic sometimes!

I wrote a blog of how I ported my Bluesky module from python to typescript (because I am lazy and the bluesky typescript SDK is better) but ended up leaving the smoke test suite I wrote in python since its just another dagger module and just works 🤓

https://levlaz.org/dagger-feels-like-magic/

proof -> https://bsky.app/profile/levlaz.bsky.social/post/3l6rakfi5wp2d

The old version of my module did not properly expand links and this one does! 😄

[real kite]

Is Dagger going to be present at Kubecon US 2024 next month?

[wraith niche]

Hey Conor, Yes we'll be there!

[chilly arch]

Thanks @neon warren for sharing your Dagger story with us!

Check out the latest Dagger Case study with Craft CMS here: https://dagger.io/blog/craft-cms-case-study

[naive leaf]

Depending on the context (and it goes somewhat contrary to the ethos of containers being... well self-contained) you might want to consider mounting some of the core libraries like CUDA, NCCL, MPI, etc from the host. Depending on how you're images are ultimately running on the machine (e.g. distributed filesystem) it can make a beneficial impact to startup stripping them down, and can eliminate issues related to subtle library differences.

[golden cypress]

Hi there! I noticed a small issue in the Dagger 0.13 announcement blog post. There's a link under the Examples section (right after some code snippets for Go, TS, and Python) that points to the wrong place. The link currently directs to https://docs.dagger.io/api/custom-functions/#directories-and-files, but it should be pointing to https://docs.dagger.io/api/arguments/#directories-and-files instead. Just wanted to flag this in case it needs fixing. Thanks!

[slender star]

Thank you @golden cypress !

cc @chilly arch 👆

[chilly arch]

Thanks for flagging! I’ll update it shortly

[raven stag]

👋 Is there a way to see closed/archived help posts?

I was participating in one yesterday that seems to be gone now.. And I wanted to re-read some replies 😅

[sharp marsh]

They should still appear in search results if you look in #1030538312508776540 of messages sent by you 🙏

[frail mulch]

my humble Dagger promotion here at KCD UK

[slender charm]

Thanks for spreading the good word 🙂

[daring mortar]

OT: where can I get this beautiful wallpaper, which is used in the community calls, from? 🙂

[slender charm]

@chilly arch will know best 🙂

[chilly arch]

1. Thank you for noticing the detail ❤️
2. Here is a folder of all the Dagger wallpapers. Enjoy! https://drive.google.com/drive/folders/1Q9HkfSuUGPAwHtW0ZEOmrvz_q8QewqPt?usp=sharing

[daring mortar]

Awesome. Thank you. 😊

[cobalt crag]

Hello. I am currently reading about Dagger and am considering using it at my company. However, I have a question.

• The thing that I do not like about YAML pipelines is that it is not type-safe. (Meaning that you do not get red squiggly lines in your editor.)
• I have a bunch of YAML files that call other YAML files (as "templates", which is the Azure DevOps terminology to call a child pipeline as a function). So when a variable is changed, is it painful to make everything work again.
• I am reading this page on integrations with Azure DevOps: https://docs.dagger.io/integrations/azure-pipelines
• The final example on that page makes it seem that Dagger is replacing the pipeline steps and is not replacing the pipeline itself. Meaning that that with Azure DevOps, I would still use YAML files to stich together all of my Dagger functions.
• Is that the case? Do people typically use Dagger to replace all of their YAML files? Is the point of Dagger to solve the problem of making individual CI steps be able to run locally (and not solving the problem of having to use YAML)?

[signal pivot]

Can someone explain how dagger sdks and functions work?
Specifically, the "translation" process.
Let's say I write some function in golang, does it get compiled to a binary or translated into something else without compilation?
If so, Is there an sdk for kotlin?

[rain oriole]

Can someone explain how dagger sdks and

[mystic citrus]

Hello, can someone explain the Dagger caching mechanics to me? Specifically, in the context of a Golang project.
The concern I have is that with Dagger builds take 2-to 20 seconds, while natively go builds take 0.2 to 2 seconds.

So far, I was able to figure out the following:

• WithMountedCache("/go/pkg/mod", dag.CacheVolume("dev-go-mod-"+GO_VERSION)). WithMountedCache("/go/build-cache", dag.CacheVolume("dev-go-build-"+GO_VERSION)) brings time down to 2 sec on subsequent run
• after changing dagger/main.go the build takes 20 seconds. (seems that the cache breaks)
• after changing my code, the build takes 20 seconds. subsequent runs 2.
• if a wait 1-2 minutes without any change to code or in the repo, the build takes, 20 seconds.

How can I get consistent improved times?

[sharp marsh]

Hey Vadim! Any chance you can share some Cloud traces?

[mystic citrus]

I can share the code

its public

func (m *HarborCli) BuildDev(
ctx context.Context,
platform string,
) *dagger.File {

fmt.Println("🛠️  Building Harbor-Cli with Dagger...")
// Define the path for the binary output
os, arch, err := parsePlatform(platform)

if err != nil {
    log.Fatalf("Error parsing platform: %v", err)
}
builder := dag.Container().
    From("golang:"+GO_VERSION+"-alpine").
    WithMountedCache("/go/pkg/mod", dag.CacheVolume("dev-go-mod-"+GO_VERSION)).
    WithMountedCache("/go/build-cache", dag.CacheVolume("dev-go-build-"+GO_VERSION)).
    WithMountedDirectory("/src", m.Source). // Ensure the source directory with go.mod is mounted
    WithWorkdir("/src").
    WithEnvVariable("GOOS", os).
    WithEnvVariable("GOARCH", arch).
    WithExec([]string{"go", "build", "-o", "bin/harbor-dev", "cmd/harbor/main.go"})

return builder.File("bin/harbor-dev")

}

from https://github.com/goharbor/harbor-cli/pull/222

[sharp marsh]

On mobile rn but at a glance I think the go cache volume doesn't seem to be correct. ref: https://github.com/sagikazarmark/daggerverse/blob/d9ba06776c4c1ccf6f329bd862b9b439c4582ab6/go/main.go#L176

[mystic citrus]

setting GOMODCACHE adn GOCACHE, did had no effect

[sharp marsh]

Got it. I'll try it out later when I'm at the keyboard: 🙏

[cunning jolt]

What version of dagger are you on? Have you tried v0.13.6? There were some optimizations on the initialization whch may help.

[mystic citrus]

yes we run 0.13.6

[sharp marsh]

@mystic citrus so, I've changed the go-build cache mount path in your BuildDev function to /root/.cache/go-build and times seem much better now. Here's my numbers:

1. First call, cold engine building for local architecture: dagger call build-dev --platform=linux/amd64 : 20s. #This time is very relative as it pulls dependencies, container images, etc
2. Second call, without changing anything: 0.3s #Expected, everything's cached here
3. Third call, modify /cmd/harbor/main.go: 1.2s #Using both build & mod cache
4. Fourth call, modify dagger/main.go: 1.2s also

if you add -x to your go build command, you can check if it's effectively using the build cache or not. That's how I realized the build cache wasn't being effectively used

[sharp marsh]

If you're setting GOMODCACHE and GOCACHE yourself, then the cache volume path should match those variables of course

[chilly arch]

**At OSS Japan? **

Happy Monday folks! If anyone is attending Open Source Summit Japan, let us know. One of our teammates is there, and he'd love to meet other Daggernauts in person.

https://events.linuxfoundation.org/open-source-summit-japan/

[chilly arch]

**At GitHub Universe? **

@slender star @wraith niche and @winter linden are there this week, so we'd love to meet in-person if you are around! https://reg.githubuniverse.com/flow/github/universe24/attendee-portal/page/livestream

[mystic citrus]

quick question

I have this code

    return dag.Container().
        From("cgr.dev/chainguard/cosign").
        WithSecretVariable("GITHUB_TOKEN", githubToken).
        WithEnvVariable("ACTIONS_ID_TOKEN_REQUEST_URL", actionsIdTokenRequestUrl).
        WithSecretVariable("ACTIONS_ID_TOKEN_REQUEST_TOKEN", actionsIdTokenRequestToken).
        WithSecretVariable("REGISTRY_PASSWORD", registryPassword).
        WithExec([]string{"cosign", "env"}).
        WithExec([]string{"echo", "Length of REGISTRY_PASSWORD: $(echo -n ${REGISTRY_PASSWORD} | wc -c)"}).
        WithExec([]string{"cosign", "sign", "--yes", "--recursive", "--registry-username", registryUsername,
            "--registry-password", "${REGISTRY_PASSWORD}", fmt.Sprintf("%s/%s", registry, imageName),
            "--verbose",
        }).
        Stdout(ctx)

somehow the REGISTRY_PASSWORD is not set, and cosign complains it is empty, I would like to avoid using plaintext. (unless it is as secure as referencing env vars)

[winter linden]

Maybe insert a .Terminal() to poke around?

[slender charm]

Can you share how you are calling this function?

[mystic citrus]

you mean from the gh action?

  - name: Sign Image
    uses: dagger/dagger-for-github@v6
    env:
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
      REGISTRY_ADDRESS: ${{ vars.REGISTRY_ADDRESS }}
      REGISTRY_USERNAME: ${{ vars.REGISTRY_USERNAME }}
      REGISTRY_PASSWORD: ${{ secrets.REGISTRY_PASSWORD }}
      IMAGE: "library/harbor-cli@sha256:9183ba01537f21c44d2aba0ca8ad500ffe6d80baea748f52b72d75be1dd19a84"
    with:
      version: ${{ steps.dagger_version.outputs.version }}
      verb: call
      args: "sign --github-token=env:GITHUB_TOKEN \
      --actions-id-token-request-url=$ACTIONS_ID_TOKEN_REQUEST_URL \
      --actions-id-token-request-token=env:ACTIONS_ID_TOKEN_REQUEST_TOKEN \
      --registry='${{ env.REGISTRY_ADDRESS }}' \
      --registry-username='${{ env.REGISTRY_USERNAME }}' \
      --registry-password=env:REGISTRY_PASSWORD \
      --image-name='${{ env.IMAGE }}'"

[rain oriole]

I think you need to wrap that command in sh -c '...' to expand that env var correctly.

[mystic citrus]

I tried with temp creds
https://github.com/Vad1mo/harbor-cli/actions/runs/11559832227/job/32175374522#step:6:471

I see the password on the output but luckily masked ***

[rain oriole]

Or at least use the Expand option on WithExec. Not for a secret, disregard.

[mystic citrus]

I have it now like this

func (m *HarborCli) Sign(ctx context.Context,
githubToken *dagger.Secret, actionsIdTokenRequestUrl string, actionsIdTokenRequestToken *dagger.Secret,
registry, registryUsername, imageName string, registryPassword *dagger.Secret) (string, error) {
reg_password, _ := registryPassword.Plaintext(ctx)

return dag.Container().
    From("cgr.dev/chainguard/cosign").
    WithSecretVariable("GITHUB_TOKEN", githubToken).
    WithEnvVariable("ACTIONS_ID_TOKEN_REQUEST_URL", actionsIdTokenRequestUrl).
    WithSecretVariable("ACTIONS_ID_TOKEN_REQUEST_TOKEN", actionsIdTokenRequestToken).
    WithSecretVariable("REGISTRY_PASSWORD", registryPassword).
    WithExec([]string{"cosign", "env"}).
    WithExec([]string{"cosign", "sign", "--yes", "--recursive", "--registry-username", registryUsername,
        "--registry-password", **reg_password**, fmt.Sprintf("%s/%s", registry, imageName),
        "--verbose",
    }).
    Stdout(ctx)

}

[mystic citrus]

in the GH Action or in WithExec?

[rain oriole]

In WithExec:

WithExec([]string{"sh", "-c", fmt.Sprintf("cosign sign --yes --recursive --registry-username %s --registry-password ${REGISTRY_PASSWORD} %s/%s --verbose", registryUsername, registry, imageName)})

[mystic citrus]

cosign is an image from scratch, there is no shto my knowledge

[rain oriole]

So there's no other binaries besides cosign?

[mystic citrus]

nope

distroless cgr.dev/chainguard/cosign

[rain oriole]

What if you copy/mount that binary into another image that has it? Even cat would be useful so you could put the secret in a file

Doesn't chainguard support adding additional packages?

[mystic citrus]

there is dev variant docker pull cgr.dev/chainguard/cosign:latest-dev. I assume it has an OS as its 20 MiBs bigger

does the effort make the setup more secure?

Dagger will mask the output, on the CLI and in GH action

[opal mist]

Did any ever manage to make a Dagger pipeline in Bitbucket cloud?

[fallen onyx]

Hey folks. Diving a bit into Dagger functions and trying to figure out how I can mount a directory for use with golang. For example I would like to use go-git to list git changes. What would I need to configure to use local directories within Dagger functions?

[sharp marsh]

👋 yes, I recall some users mentioning that they tested dagger in bitbucket pipelines. Are you having a particular issue?

[slender star]

Hey folks. Diving a bit into Dagger

[opal mist]

It is just the only thing i could find was that it was not not posible, due to limitations on docker.

[sharp marsh]

I think you're right. Now I recall someone brining up something around Docker and Bitbucket pipelines. Can't recall exactly what it was 🤔

[slender star]

They don't allow privileged

[opal mist]

and before i have setup a self-hosted runner, by that time it is due to be migrated to Github anyways

[slender star]

Ah, right, no priv on on their cloud hosted, right, but you can set up your own self-hosted runner like this: https://community.atlassian.com/t5/Bitbucket-questions/How-to-run-anything-that-requires-privileged-flag-in-my-self/qaq-p/1982343

[chilly arch]

Hey everyone! Not only is tomorrow Halloween, but it's also the community call. 🚀

Join @warm temple , @slender charm and the rest of the gang to learn about the latest Daggerverse updates, how we daggerized GoReleaser, and best practices for using Dagger & Harness CI.

Use the link below to add the series to your calendar! We'll be live streaming to YouTube, LinkedIn, and Twitter.

https://calendar.google.com/calendar/u/0?cid=Y19mZDEyYWRjOTI3ZTQ0MDVkMGY2NmUyZjA2Yjk0OTI3YzI4ZjA2YzJkMWQzZDFiOTRlNTIwMjZkNGU0MjMzNWQxQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20

*quality will be best on YouTube.

[chilly arch]

**🎃 Halloween surprise! **

If you’re tuning into tomorrow’s Community Call in costume, snap a pic and send it my way—I’ll send some exclusive Dagger swag your way! 👻💀

[left thunder]

Hi please any resources on canary deployment with dagger? Thanks

[unreal latch]

happy halloween all!

[cunning jolt]

Can someone post the links to the community call please?

[sharp marsh]

https://www.youtube.com/watch?v=wLornECeGEM that @cunning jolt ?

[unreal latch]

yt channel

[cunning jolt]

yt does not work in restricted more 😦

*quality will be best on YouTube.
😢

[inland coral]

https://www.linkedin.com/events/daggercommunitycall7242931109196374016/theater/

https://x.com/i/broadcasts/1YqGovkodRgKv

Solomon Hykes

Dagger Community Call

Twitter

[sharp marsh]

https://tenor.com/cfssfdjb6e9.gif

[cunning jolt]

The experience on linkedin is pretty bad btw. It keeps disconnecting every few minutes, I have to refresh the page. I don't have access to X or yt streams.

[slender star]

Sorry Nipuna, yes, I was looking at LinkedIn and had similar issue. We'll work on making that work well. Reloading helped a little.

---

Regarding Featured Modules in Daggerverse:
Yes! You can ping me if you have favorite modules that should be featured! 🙏

[chilly arch]

sadly, Streamyard doesn't promise 1080 quality on LinkedIn or Twitter, only YouTube 😦 I am talking to a few other vendors, but if anyone has a streaming tool that is more reliable, then let me know in a DM!

It should look much better on YouTube this week

[cunning jolt]

I'm happy to help with testing if you want to try figuring out how to allow it on yt restricted mode

I don't see a reason for it to be restricted :/

[inland coral]

We're looking to bring more attention people's favorite modules. If there's a module you use a lot or one you wrote you think others should know about , please reach out. We're happy to collaborate on a blog post on your favorite module, have you present on our community call or highlight it in another way.

[chilly arch]

yes, for sure! I'll DM you after the call. I am not sure why that is happening since others aren't hitting that blocker.

[cunning jolt]

I use Kyle's modules as inspiration for mine all the time!

Please document these patterns! There are so many and new ones keep coming up, I keep forgetting which one to use 🙂

[sharp marsh]

I can't feel more related @slender charm I had to upgrade https://github.com/ammen99/wf-recorder today for sway and I was already thinking about sending a Dagger PR upstream

[cunning jolt]

This should roughly work
-kpenfound

😂

[swift inlet]

I use your docusaurus module all the time, feels so magical to run any PR's docs locally

[cunning jolt]

It's not .dagger ?? 🙀

[slender star]

might be this one: https://daggerverse.dev/mod/github.com/dagger/dagger/modules/wolfi

[chilly arch]

https://github.com/levlaz/snippetbox

[winter linden]

@warm temple I tried running tests from your demo, but it only prints metadata without runnign anything:

dagger -m github.com/goreleaser/goreleaser call test

[slender star]

@split olive def looking at https://daggerverse.dev/mod/github.com/puzzle/dagger-module-git-actions/git-actions
Would be great to be updated to latest dagger 🙏

[warm temple]

Probably need stdout or coverage-report -o . because it's returning a TestResult

it would be cool to specify a 'default leaf' for structs like that kind of like what we have on core types

[winter linden]

FYI this might be because of a change in the latest Dagger release,- we changed how objects are auto-printed

(just realized this might the reason)

[warm temple]

yes I was demoing from 0.13.6

[winter linden]

Damn I literally just upgraded 2mn before the call

[warm temple]

I tried to but brew didn't get it apparently

[narrow nymph]

welcome to v0.13.7 ❤️ 🙂

[winter linden]

(the new way to get something printed is to make it a field btw. cc @rain oriole for real-time feedback 🙂

[narrow nymph]

pending here https://github.com/Homebrew/homebrew-core/pull/196242

we also have a tap for this: https://github.com/dagger/homebrew-tap
this gets the updates as soon as they're released (since we control it)

[warm temple]

I'm using the tap but I must have done it before it got bumped

[chilly arch]

Thanks for joining everyone! You can find the recording here: https://youtube.com/live/wLornECeGEM

Reminder - We are moving the series up by one week, so we'll have another community call next week to kick off the new flow. You can add the series to your calendar with the link below: https://calendar.google.com/calendar/u/0?cid=Y19mZDEyYWRjOTI3ZTQ0MDVkMGY2NmUyZjA2Yjk0OTI3YzI4ZjA2YzJkMWQzZDFiOTRlNTIwMjZkNGU0MjMzNWQxQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20

[winter linden]

thoughts on this approach to gh issues & discussions? https://x.com/mitchellh/status/1852039089547309552?s=46

Mitchell Hashimoto (@mitchellh) on X

I'm doing something new (for me) with Ghostty: the issue tracker is only for actionable tasks. Features in the issue tracker are accepted and well-scoped. Bugs are reproducible. PRs must have an associated issue (no drive-by PRs). The issue tracker is not used for discussion.

Twitter · Oct 31, 2024, 5:25 PM

[slender charm]

I like it a lot! Assuming we’d encourage more discussion here in discord?

[chilly arch]

I knew I could count on our community to follow through on this! Thanks @split olive for sharing your pic. Love the costume, and thanks for joining the Community Call today!

[winter linden]

We need a picture of @warm temple 's costume too!

[winter linden]

In his post Mitchell mentions "some discord + strong encouragement to use GH discussions"

[slender charm]

Have we used those at all? I really like the idea of indexing and search from gh discussions, or any other forum for that matter

[slender star]

We used to use them a lot more.

[winter linden]

Yeah we tried, but didn't fully commit, so it petered out. The key is that Mitchell is being super restrictive on issues, with zero tolerance of open-ended discussions there. Plus you have to be very persistent in pushing people to GH discussions, to overcome inertia. It takes a lot of commitment, you can't to it half-way.

[winter linden]

https://tenor.com/view/purple-cobra-dodgeball-gif-21968875

[celest grove]

I've seen the recent community call, and I love the featured mark for modules. Overall, I think it's a great idea. I'd like to ask to promote these two modules: GoToolbox https://daggerverse.dev/mod/github.com/Excoriate/daggerverse/gotoolbox@a83a220c71aba509a487067926b196a0c9c0b83a and Terragrunt: https://daggerverse.dev/mod/github.com/Excoriate/daggerverse/terragrunt@ff2e4f0fbe48f2790685e7feaa01b19caab9b48b
to the same level — they're quite complete, battle-tested, with tests and even examples (recipes). They're used by many folks, including internally in the company where I work. If it's not feasible, I'd love to understand the requirements for making it featured since these (and mostly my modules) are actively maintained. 🙂

[cunning jolt]

This is my experience with using GH discussions. It ends up being more work telling people to go to discussions than having a discussion in Discord (MS Teams in my company). But if you can get there, it's a good knowledge base unlike the chat. But then again, the #1030538312508776540 channel is pretty much that.

[unreal leaf]

Hi Dagger Community,

I’m setting up Dagger to run a live development server with Air for a Harbor Satellite project. In Docker, I would use a bind mount to sync my host directory with the container so that any changes in the host files are instantly reflected inside the container:

docker run -p 8080:8080 --rm -v $(pwd):/app --name my-go-docker-air my-go-docker

Currently, I’m using dagger.withMountedDirectory() to mount my local directory in Dagger, but changes in the host files aren’t being reflected inside the container.

Is there a way to achieve a bind mount in Dagger similar to Docker, where updates on the host would dynamically sync with the container? Any guidance or alternative setup approaches would be greatly appreciated!

Thanks in advance!

[raven stag]

Hey @unreal leaf , it is a known "limitation".
There's no workaround possible today for "live reloading'" using services AFAIK.
You can leave your use-case here to raise the awareness 😉
https://github.com/dagger/dagger/issues/6990

[unreal leaf]

Thanks, @raven stag , for the quick response! I appreciate the confirmation about the limitation.
I’ll add my use case to the GitHub issue. Hoping this feature makes it into Dagger soon—live reloading would be a game-changer for development workflows!
Thanks again!

[ancient escarp]

From my perspective, I find that finding good modules on daggerverse is HARD. A couple of examples, search for php -> a lot of the results are for forks of the php sdk; this seems somewhat pointless as it's built in as part of dagger already, but doubly so as these were only forks that people were using on a temporary basis to implement new features. Now search for helm; the top results include what looks to be a fairly specific setup for someone's pipeline; not a generic helm module; then you have one tagged partner which provides very limited functionality; the featured one is probably the best of the lot but even that doesn't include all the basic functions of helm (eg it's missing update as an option)

[ancient escarp]

I really think that the auto publish on use is a bad idea as it's putting a lot of poor quality results into the search, especially the forks of modules. I think explicit publishing should be required and also would be worth considering some deduplication based on module name (perhaps introducing a vendor/module naming convention could be useful here) and the ability to view the original module and any forks of it. Packagist does a really good job of this - pointing out potential existing packages and such when you submit a new one.

[thick igloo]

Good points in general, just my 2c. I don't think the auto publish will have any effect on this, since you can't really avoid to have duplicates. if 5 or 20 it's not so big of a difference. While having featured/blessed modules can help a lot and to do that you need to understand what are the modules that are more widely adopted and start from there. I suppose that is the idea at least. Also, regarding missing functionalities, it looks normal at this stage where modules are community based - e.g. you can open a PR adding what is missing or an issue asking for it.

[ancient escarp]

I totally get the idea of a PR to address a missing feature but the problem can then become - to which module should I submit to? (It also creates another fork in the daggerverse, which will live on long after the PR is merged) I really do think the auto publish is compounding the issue here, so many scenarios where I wouldn't want a module published, a fork of an existing module to fix a bug which gets merged into the original, a quick dev experiment to prove a concept, a tailored module intended to be used across several pipelines in an open source project I maintain, a module for a private project that started out in a public repo while developing the poc, a dagger workshop where I get 25 participants to make their own example module and then install and use it as part of their pipeline.

[winter linden]

@ancient escarp yeah, the issue of "noise" in Daggerverse is something we're starting to address in layers.

• Layer 1: manually feature modules that are known to be useful and up-to-date (please nominate your favorite modules for this)
• Layer 2: fork awareness. We definitely should treat github & gitlab forks in a special way, to hide them by default
• Layer 3: better ranking. We're iterating on the search rankings, to help high-quality results move to the top over time.
• Layer 4: internal modules. We want to add a way to mark your modules as internal, so that they are hidden by default.
• Layer 5: stdlib. We need a stable, trusted standard library. We've been wanting to do it for some time, but have been waiting for API patterns to stabilize, to avoid shipping a stdlib and immediately breaking it.

Note on "auto-publish". That term can be misleading because Daggerverse is not a registry: it's a search engine. Its job is to index every module you could find on your own, and help you find it better.
So when you search for "php" and are swamped by useless results, the solution is not to un-publish those registries (they are out there, and we don't want daggerverse to be blind to that fact), but to make the search results smarter. Which we intend to do!

Thanks for your patience 🙏

cc @slender star @warm temple

[ancient escarp]

@Carnage yeah, the issue of "noise" in

[robust lance]

I tell you, as soon as I get this GHA workflow wrapped up I'll be hitting Dagger real hard. Pretty excited the more I look at it. I've spent the last several years in AWS so I'm currently getting my bearings with GCP, especially with IAM. I feel like I need a GCP IAM for AWS Dummies book.

[stoic knot]

Are there zig bindings for Dagger? (trying to convince someone to take Dagger for a spin)

[slender charm]

Not that I know of. Would be neat 🙂

[stoic knot]

Are there any updates on enabling remote cache from the engine config file? It looks like the buildctl tool just passes the flag to the SolverOpts object, not sure if Dagger uses this BuildKit config object, but if it did, it would seem easy to pass config through...?
https://github.com/moby/buildkit/blob/46c8b9ee45d0f91aa935d69c53d8b25ed07fcf97/cmd/buildctl/build.go#L196
(Really wish they would add the configuration content upsteam...)

[winter linden]

Are there any updates on enabling remote

[quiet tiger]

How does one clean up/delete/remove a cache volume? https://docs.dagger.io/api/cache-volumes/ ?

[sharp marsh]

there's a stopgap you can use which is to run a pipeline that mounts the cache volume to a path and perform an rm -rf /* against that path.

[raven stag]

👋 Does dagger core dagger-engine local-cache prune clear the volumeCaches generated? Asking for a friend... 😄

[warm temple]

I don't believe so, but here's a 1 liner to clear out a volume:
dagger core container from --address=alpine with-env-variable --name bust --value $(date +%s) with-mounted-cache --path /cache --cache "my-cache-volume" with-exec --args "sh,-c,\"rm -rf /cache/*\"" stdout

(replace "my-cache-volume" with your volume's name)

or to delete all volumes, you can just nuke the engine container and it's volume
docker rm -fv $(docker ps --filter name="dagger-engine-*" -q)

[chilly arch]

Introducing Public Traces for Open Source Repositories

We’re excited to announce a new Dagger Cloud feature: Public Traces for Open Source Repositories. This functionality allows open source project teams to share trace data from Dagger pipelines publicly, helping communities collaborate more effectively and troubleshoot issues with greater transparency. Learn more in the blog post: https://dagger.io/blog/public-traces

If you have any questions, please feel free to ask here!

[cunning jolt]

Curious what the dagger shell version looks like. Should be much simpler right?

[winter linden]

.container | from alpine | with-env-variable bust $(FIXME) | with-mounted-cache /cache my-cache-volume | with-exec sh,-c,"rm,-rf,/cache/*" | stdout

We're missing a builtin to produce a cache buster

and with-exec arg could use some polish

[stoic knot]

+1 for the vertical pipes that make the steps clear

[winter linden]

@stoic knot yeah I really enjoy it also 🙂 Did you try playing with dagger shell yet?

It's missing some polish, but it does work

[stoic knot]

I have not, I'm working on moving our monorepo POC to a beta so we can build a full product with a one-liner

the process is certainly uncovering all the minor inconsistencies in Makefiles and Dockerfiles

[cunning jolt]

Should dagger shell be runnable outside of the context of a dagger module? I just noticed you can't.

[stoic knot]

Is it possible to construct a DAG in the shell? Maybe using some intermediate variables to store pipelines and then referencing them in others? or parentheses or backticks like bash?

[winter linden]

Yes, it's on Helder's todolist

[winter linden]

No variables yet, but YES it supports subshells 🙂 One of the killer features.

You can try it now:

$ dagger shell
.container | from alpine | with-workdir /src | with-mounted-directory /src $(.directory | with-file dagger-readme.md $(.git https://github.com/dagger/dagger | file README.md)  |  with-new-file  'hello there' hello.txt) | terminal

[cunning jolt]

wow that was fast!

[winter linden]

Next up Helder is working on .doc for builtin API docs

foo | bar | .doc -> prints the full API doc for the type produced by bar

[cunning jolt]

Had to ctrl+D to exit. An exit command and a history command would be nice!

[winter linden]

it would be .exit with our builtin system but yeah, good point

[cunning jolt]

it actually does save history. so I can up arrow and find stuff. But an actual history command would be nice indeed

[loud briar]

Out of interest is the primary use of this intended to be something to do with dagger 'scripts'? Or is there some other use?

[winter linden]

We see two primary uses:

1. Superior interactive UX. This will probably be the best way to interact with Dagger in the command-line
2. Batch scripts. It gives us a standard syntax to express "run this dagger thing", so that will probably become the standard "payload" that you'd find in various CI configurations for example

For now, we haven't decided whether a module should have a standard entrypoint for exposing scripts

[loud briar]

So in your view it'd eventually replace pipelines that consist of multiple dagger call -m blahblah dagger call otherthing

[winter linden]

Yes. We won't break call but we expect the shell will be a superior alternative

There's also an opportunity to use $foo as the universal interface for accessing contextual variables... We control how those variables are resolved, so we could map them to anything. For example pluggable secrets providers...

.env file support...

dagger shell --env-file .env.production -c 'deploy --token=$DEPLOY_TOKEN'

dagger shell --env-var DEPLOY_TOKEN=op://myvault/myproject/deploy/token -c 'deploy --token=$DEPLOY_TOKEN'

[loud briar]

Is some form of 'scripts' on the backburner whilst this work is ongoing? Primary pain point for now is the long chains of function variables, ends up in makefiles but then users start adding make 'magic' and what is old is new again 🤦‍♂️

[winter linden]

Even better: if we can rely on the fact that $DEPLOY_TOKEN is virtualized (and therefore portable), we can perhaps allow modules to set that as a default. So now you get:

func (m MyModule) Deploy(
  // +optional
  // +default=$DEPLOY_TOKEN
  deployToken *dagger.Secret,
) { }

dagger --env-file .env.production -c 'deploy'

Or if not defined, via static analysis we can prompt you on missing env:

dagger shell -c deploy
ENTER VALUE FOR deployment token: *******

[loud briar]

Think I saw a comment about dagger shell -> dagger, now you've described this a bit that seems like a good idea

[winter linden]

Yeah we'll make our way to that I think. With a portable reproducable shell syntax, we're eating away at that problem. Sure you may still have a long-ish dagger script, but at least it's portable, and maps directly to Dagger API. From there it's trivial to move into a new function. So as the "glue scripting layer" gets too thick, much easier to keep it lean by moving the growing logic into code

[cunning jolt]

How will the shell work in non tty consoles like GHA or Jenkins? Would I have to create a separate script and run it?

[winter linden]

Yeah IMO if I'm right about all the above, eventually we'll start thinking as dagger as being a shell. Then dagger becomes a defacto alias to dagger shell. I just don't want to rush that last part, just in case I'm wrong 😛

[winter linden]

same as dagger call. We'll make sure it works great in non-interactive environments

If you look at the best practice for retrofitting Dagger into CI, there's always a shell command somewhere in the CI config, that resolves to dagger call. This would be a natural evolution of that. Just more powerful and more portable

Also I think we can make a really killer auto-complete in interactive mode 🙂

[loud briar]

Sounds good to me, looking forward to seeing this evolve 🙂

[cunning jolt]

same!

[winter linden]

Thank you for all the questions and feedback, it's super useful & motivating 🙂

[chilly arch]

Looking forward to seeing everyone at the Community call tomorrow! We'll be live on YouTube and X.

Watch on YouTube: https://www.youtube.com/watch?v=C_1yZdt3lTQ
Watch on X: https://twitter.com/dagger_io * quality is much better on YouTube...

Add the series to your calendar: https://calendar.google.com/calendar/u/0?cid=Y19mZDEyYWRjOTI3ZTQ0MDVkMGY2NmUyZjA2Yjk0OTI3YzI4ZjA2YzJkMWQzZDFiOTRlNTIwMjZkNGU0MjMzNWQxQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20

[stable patrol]

hi 👋 who could i speak to about the 'bring your own cloud' for distributed caching?

[winter linden]

cc @slender charm 🙂

[slender charm]

Hi! Would love to chat more, I’ll DM you to find some time.

[tired moth]

Yooo...Dagger Rocks....

[chilly arch]

Community call happening now! Feel free to use this chat channel for questions or ask on YouTube.

[tired moth]

FEEDBACK
Open-source projects often struggle with documentation. To avoid this with Dagger, we need a dedicated docs feedback channel. This will lower the barrier for contributions and help us create truly helpful docs.

[chilly arch]

Thanks! Currently, we ask everyone to submit PRs since our docs are open source, but another way to add feedback is to create a help forum post with what you feel is missing and we'll make an internal issue that goes to @heavy karma to triage.

[tired moth]

alright thanks

[slender star]

Docs feedback does get scattered across discord, which can be "good" since they're in context.
Having a place to discuss in discord, perhaps with a feed of docs issues could be pretty interesting...since ultimately GitHub issues (or PRs 🙏 ) are best

[tired moth]

Yes because new devs or very busy devs dont create issues or PR but they can write quickly in docs dedicated channel...
just my thoughts...

Do we have open roadmap with no ETA?

HTTPS auth support for dagger private modules? soon? Please ship it...we need it...

Can dagger support inbuilt/native encryption for security? OR through module? just a thought...

DOCKER runs slow everywhere...from the last few years...

We are also migrating from docker to nerdctl

[chilly arch]

KitOps Modules

Thank you so much @cunning jolt and @crystal ore for the demos today! For anyone who missed it, you can check out the recording here: https://youtube.com/live/C_1yZdt3lTQ

[thick igloo]

Nice demo @crystal ore ! Just watched the recording, I love the OCI artifact decision!! In fact, with a colleague, we were discussing why oci artifacts weren't the go-to solution for model sharing already. I'm glad to see someone pushing this forward!

[quiet tiger]

Will it be "safe" to assume that I can use the engineVersion to decide which version of the dagger cli folks need? Or are those 2 divorced from each other enough that it doesn't make any sense?

I often run into issues with people not having the right version(s) of cli tools (hence wanting to move a lot of stuff to Dagger) ... but adding a new cli tools means trying to ensure folks have the right version of that too.

[winter linden]

yes engine and CLI version are very tightly coupled.

you can consider them to be one and the same

[quiet tiger]

Awesome

[crystal ore]

Thank you. Yes, ModelKits do solve and simplify a lot of the concerns for AI/ML artifacts.

[crystal ore]

If you ever try ModelKits let me know, I would like to hear if it was what you thought how an OCI solution would be.

[chilly arch]

Anyone still need a ticket for KubeCon? Our friends at MLOps have some free tickets. You can claim one of them here if you need it: https://tech.tickets/lf

[quiet tiger]

Is there any comprehensive dagger.json documentation?

Also ... This looks like internal dagger stuff?

                    {
                        "description": "[internal] copy upload <A PATH I REMOVED> from v2kivfdxyndbzw4lphakmdxq7 (client id: iwbiyo2p4vfh6o1pj25iz9rs1, session id: xphvmvqy637amfnunlc7ztt3k) (include: dagger.json, dagger/**/*, **/go.mod, **/go.sum, **/go.work, **/go.work.sum, **/vendor/, **/*.go) (exclude: **/.git)",
                        "diskSpaceBytes": 382753520
                    },

If so, this looks like it's uploading stuff for dagger use (our module source is in ./dagger atm), why does it need **/vendor/ and **/*.go ? This is in a large repo and if this is for internal dagger usage I'd like to not include those.

[rain oriole]

The issue tracking that is https://github.com/dagger/dagger/issues/8496.

[quiet tiger]

subscribed

[tired moth]

To the amazing Dagger team & specially @chilly arch @analog raven @heavy karma

Thank you so much for your support in making our recent Dagger meetup such a success! We all had a blast learning about Dagger and getting hands-on with the workshop. Your expertise and enthusiasm were truly inspiring.

We're already looking forward to the next one!

Best,
The Staytuned Team

[chilly arch]

Thank you for hosting and sharing pictures!

[tired moth]

Welcome. We are always happy to support dagger.

[tired moth]

@winter linden Thanks a lot for this git repository support for HTTP/HTTPS refs in the latest release.

[loud briar]

I notice that last missing TODO at https://github.com/dagger/dagger/pull/8805 however, can that be prioritised? Can't imagine it'll take long to do

[narrow nymph]

It's already in progress + should be merged very soon! https://github.com/dagger/dagger/pull/8904

[ancient escarp]

Going to be giving a dagger talk at PHP UK next year: https://x.com/PHPUKConference/status/1855947338461950150?t=chS_k3ZTq9XgFUdOUwpyrw&s=19

PHP UK Conference (@PHPUKConference) on X

PHPUK Speaker Announcement 🚨
Chris Riley (@giveupalready) will be joining the line-up for PHPUK Conference 2025! 🐘

Ready to supercharge your CI/CD pipelines! With his talk "Sharpen up your CI/CD pipelines with Dagger", Chris will show you how to do exactly that.

Buy a

Twitter · Nov 11, 2024, 12:15 PM

[raven stag]

👋 Are service bindings and cache volumes expected to be missing when you run dagger with --interactive flag and you get an error?

[winter linden]

no, it's a bug .. (at least I consideer it so) sorry

[narrow nymph]

@raven stag mind opening an issue? I think it's probably not too tricky to get it working 🙂

[raven stag]

Sure thing, tx

[raven stag]

for reference: https://github.com/dagger/dagger/issues/8926

[raven stag]

and https://github.com/dagger/dagger/issues/8927

[narrow nymph]

Thank you thank you 🙏

[slender charm]

We asked people to draw an emoji of CI at Cloud Native Rejekts and this is what we have so far 🙂

[narrow nymph]

particularly in love with the swirly eyes one 😄

[rigid pebble]

The empty ones are 💯 😆

[modest pier]

no 😬 ?

[proper falcon]

👋 Do cache volume names need to be globally unique?

[proper falcon]

We have long-running shared Dagger engines serving multiple pipelines for different teams, and we're wondering what will happen when volume names start clashing.

[ionic plume]

Hi, I just update dagger cli to v0.13.7 from v.0.12.3. But now my project no more working. I get rror: incompatible engine version v0.12.3. I have tested to delete de local container dagger-engine:v0.12.3. But when I run dagger call It create again container with bad version (0.12.3 instead 0.13.7). On dagger.json, I have set "engineVersion": "v0.13.7". You know what is the hell ?

[ionic plume]

Hum, I found the hell, _EXPERIMENTAL_DAGGER_RUNNER_HOST set

[slender star]

cache volume names need to be unique

[chilly arch]

Dagger v0.14 is here! 🚀

Check out all the new features in our latest blog post - https://dagger.io/blog/dagger-0-14

If you run into any issues with upgrading or have any questions, please let us know!

[lyric turtle]

on the docker website, there is an AI chatbot trained on the documentation that can answer questions which I've found extremely helpful. does dagger website have a similar concept?

[slender star]

In the works! We had one that we weren’t satisfied with, so trying again. Anything you found especially helpful or magical from the Docker experience? Anything missing that would have a huge impact?

[charred helm]

Hey, I'm wondering, is WebAssembly v3 on Dagger.cloud UI some things coming out later or am I having a configuration issue somewhere? The page returns input: trace unauthorized.

[winter linden]

both 🙂 It's not yet released, but it should already work pretty well. Would you mind filing an issue?

[charred helm]

Do you want a ticket from the UI or an issue on GitHub? I don't have much information beyond the query and the return payload.

[tired moth]

Why are we not having a separate repo for docs?

[slender star]

Why are we not having a separate repo

[opal mist]

What would be an argument for one that is promoting Bazel to try out Dagger.io?

[tired moth]

Who is working on dagger cloud UI/UX?

[sharp marsh]

many of us are! Feel free to share your thoughts here @tired moth.

[tired moth]

We have a very highly skilled in house design team with min 10 years of experience including me...
So we would like to have a dedicated meeting to share things or we can suggest a few things here...OR can't we have a dagger cloud website/console open sourced?

[loud briar]

I wouldn't hold your breath on Dagger open-sourcing the only revenue-generating product they have 😉

[tired moth]

i am asking for website frontend part

[sharp marsh]

if you already have a few items which you have some sort of written feedback, feel free to open an issue in dagger/dagger and we can keep discussion there. If you're looking for a more conversational-like chatting, we can also start a thread here. Re: open sourcing Dagger Cloud, there's no plans to open source any of those pieces any time soon.

[tired moth]

ok then we will share things after final draft. thanks

[sharp marsh]

sure, anytime

[quiet tiger]

is it possible to declare a default equivalent to --arg cmd:"path/to/some/program" ? --arg is a string and I can assign a string default with
// +default="TheDefault" in go (and equivalent in other languages), but I'd like to be able to specify a default command to use to fetch the secret this way commands are easier for folks in this internal repo.

[quiet tiger]

My assumption is no because arg handling that makes env: and cmd: possible happens in the dagger cli on the host and by the time things get to a constructor/function call things are inside the dagger runtime/docker env ?

[quiet tiger]

With that said maybe there is something like defaultPath ala defaultCMD or defaultEnv for things that are secrets?

[winter linden]

default host command

[sly wharf]

Hello, is there a self hosted version of dagger cloud ?

[stable patrol]

hey! how do i configure the port that the dagger engine is exposed on?

[slender charm]

Private Dagger Cloud

[sick monolith]

Some docs here talk about the port, but its session based. https://docs.dagger.io/reference/cli/#dagger-run. You shouldn't need to worry about the port. the bullet points under the diagram explain a bit https://docs.dagger.io/api/internals/#dynamic-api-extension.
There is some customisation but not sure if its needed for your use case: try dagger listen -h there's something about a custom port being set here for when you want to run a pipeline, https://github.com/dagger/dagger/issues/7568. This is not the same as 'setting the port the dagger engine uses'.

[jagged shoal]

Hi,
Does anyone have a slide deck for pitching Dagger for a developer team that they'd be willing to share? 🙏

[chilly arch]

We’ve got an exciting lineup for tomorrow’s Dagger Community Call, and we can’t wait to see you there! 🎉

@tawdry imp from Ubisoft and @ashen stirrup from marimo will share some incredible insights and projects.

📺 Catch the livestream on YouTube and X —don’t miss it! 🚀 You can add the series to your calendar here: https://calendar.google.com/calendar/u/0?cid=Y19mZDEyYWRjOTI3ZTQ0MDVkMGY2NmUyZjA2Yjk0OTI3YzI4ZjA2YzJkMWQzZDFiOTRlNTIwMjZkNGU0MjMzNWQxQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20

[mystic citrus]

I have the use case that might be quite common, we are chaining dagger functions buildBinary, BuildBinary, buildBinairies, BuildBinariesetc...

No matter how I twist it, I can't up with a better approach.
At the KubeCon @slender charm and @winter linden mentioned some other options, that might be however more complicated.

I am not so happy with my current setup but I also don't know what I can improve instead of creating each function twice for internal user and CLI
https://github.com/container-registry/harbor/blob/build/daggerize/.dagger/main.go

Any ideas?

[austere maple]

I am deeply distressed that Dagger Cloud went away after I explicitly asked it to stick around, expressed my interest in what it has to offer, told it to speak freely, and politely requested it never stop.

[slender star]

The only way to get it back is set GOAWAY and friends to the full value of π...
or just unset them if you're lazy... 😉

[austere maple]

I am deeply lazy and so will too be my evaluation of that full value. Let me get on that right away with my full contingent of irrational exuberance

guys it didn't work

[slender star]

Must be Friday somewhere

[austere maple]

yeah and to make matters worse I managed to evaluate the whole value of π but I didn't print it in time 😦

there goes my Fields Medal. I guess I'll try again on Friday when it's Wednesday somewhere

[austere maple]

I'd suggest nixing the idea of returning a binary file. That removes a few endpoints. Also suggest having a BuildImages function that builds all by default and takes an array for clients that only want one or a few images

I put up a PR https://github.com/container-registry/harbor/pull/46/files

If you want to add functionality for building binaries, I'd suggest a function that wraps BuildImages and pulls the files from containers. Can use the same setup of a default value that retrieves all by default

can't comment on if the build actually works, i seem to be missing a step somewhere

the argument parsing section seems to work at least

[fossil pine]

That is exactly what I did for OpenMeter: https://github.com/openmeterio/openmeter/blob/main/.dagger/build.go

I don't think it's the end of the world.

It may become somewhat easier eventually with an improved platform API:

• https://github.com/dagger/dagger/issues/6675
• https://github.com/dagger/dagger/issues/8482

You could take a look at the new range function feature in Go in the meantime. That may yield (no pun intended) slightly nicer code. But as I said, I don't think this is bad at all.

[analog raven]

Community Call starting in 10 mins! Here's how to join:

YouTube - https://www.youtube.com/watch?v=uhZjZ8Ezfy0

Twitter - https://twitter.com/dagger_io

[cunning jolt]

hey from Fort Worth, Texas!

[chilly arch]

Example of where marimo is using Dagger - https://github.com/marimo-team/marimo/tree/main/dagger

Demo example of marimo and Dagger in action -
https://github.com/marimo-team/marimo/blob/main/examples/ai/chat/dagger_code_interpreter.py

[chilly arch]

Thank you to everyone that attended the Community Call, and shout out to our speakers @ashen stirrup , @tawdry imp , Thomas (not on Discord yet ;), and @swift inlet !

You can find the recording here:
https://youtube.com/live/uhZjZ8Ezfy0

[tawdry imp]

Thomas is here 😉 @plucky pawn

[chilly arch]

**Have a great Dagger demo? **

If anyone would like to demo on December 5th Community call, please DM me! I have an extra slot 🙂

[orchid lion]

How to enable the private harbor? I want to use my self-hosted harbor. But only adding the CA is not work. https://docs.dagger.io/configuration/custom-ca

Where should I put the CA to docker daemon.json and apply it to docker?

[sharp marsh]

you need to put your certificate under Dagger's engine /usr/local/share/ca-certificates/ path.

[quiet tiger]

Is there no way to get annotations on a container?

[quiet tiger]

I can set them with WithAnnotation, but I can't seem to find a way to read them.

[orchid lion]

Yes, I put it to to this path /usr/local/share/ca-certificates/, but it still has x509 error , when I pulling image from private harbor

[orchid lion]

11 : Container.from(address: "10.10.10.240/library/metis:50fd547"): Container!
12 : resolving 10.10.10.240/library/metis:50fd547
13 : remotes.docker.resolver.HTTPRequest
14 : HTTP HEAD
14 : HTTP HEAD ERROR [0.0s]
14 : ! tls: failed to verify certificate: x509: certificate signed by unknown authority
13 : remotes.docker.resolver.HTTPRequest ERROR [0.0s]
13 : ! tls: failed to verify certificate: x509: certificate signed by unknown authority
12 : resolving 10.10.10.240/library/metis:50fd547 ERROR [0.0s]
12 : ! failed to do request: Head "https://10.10.10.240/v2/library/metis/manifests/50fd547": tls: failed to verify certificate: x509: certificate signed by unknown authority
11 : Container.from ERROR [0.0s]
11 : ! failed to resolve image 10.10.10.240/library/metis:50fd547: failed to resolve source metadata for 10.10.10.240/library/metis:50fd547: failed to do request: Head "https://10.10.10.240/v2/library/metis/manifests/50fd547": tls: failed to verify certificate: x509: certificate signed by unknown authority

the error message is like this

[cunning jolt]

@sharp marsh potentially the same bug I'm running into?

[sharp marsh]

Might be related but I'm not sure it's the same. In your case you're getting cert errors in the Go runtime container. @orchid lion 's output is while trying to pull an image which comes from the engine container

I was about to send you a message on Friday about this Nipuna but ran out of time. Will ping you in Monday morning

cc @swift inlet

[winter plank]

Hi folks, not sure if this is the right place for this question but i'm trying to wrap my head around how to work with Dagger. My team currently has some custom scripts that pull content from our CMS that get run with npm run build:content -- they're simple node scripts that make API calls and write the data into the working tree. In Dagger, would I rewrite these scripts into a dagger function? Would I just call npm run build:content from a container? It's kind of unclear to me which is the idiomatic way to migrate this workflow

[winter plank]

It's not even obvious how to copy an absolute path into a container to run the script

[warm temple]

daggerizing an npm application

[cunning jolt]

Thank you! I am going on vacation for 3 weeks on Monday. I will still check in but probably won't be able to do any testing till I get back 🙂

[frozen basin]

Is there a way to copy files from an currently running dagger terminal session to the host dir?

[tired moth]

Why we are not supporting LXC?

[frozen basin]

Is there a way to copy files from an

[limber gyro]

Is there a way for "dagger core container from --address alpine:latest" to succeed if I have container cached but are in air plane mode or have something broken on my internal registry?

[sharp marsh]

11 : Container.from(address: "10.10.10.

[sharp marsh]

yes, it works if you use the full image digest instead of the tag

i.e: dagger core container from --address alpine:latest@sha256:1e42bbe2508154c9126d48c2b8a75420c3544343bf86fd041fb7527e017a4b4a terminal

^ that works if after running it the first time you turn off your internet connection

[quiet tiger]

I ended up using labels instead since I can both set and read them.

[patent slate]

Wonder were you all got this FAQ from? 🤔 😄 😊 😘

[jagged shoal]

When building multi-architecture images using Dagger (following the example at docs.dagger.io/cookbook/#build-multi-arch-image), we end up with a list of containers that get pushed as a single multi-arch image. For security scanning purposes, should we scan each architecture-specific container separately, or is it sufficient to scan just one of them? What's the recommended approach for integrating security scanning into this multi-arch build process?

[narrow nymph]

I'd probably say that you should scan all the architectures you're using - even if the "recipe" for building all the platforms is the same, there's always the potential for vulnerabilities in one specific architecture over others

The way we do image scanning in our own process is by having a ScanEngine function that builds for each release container, and scans each of them

[ionic plume]

hi, there are a way to chain terminal with directory. It seems to lost on directory all change made from terminal ?

[narrow nymph]

yeah, Terminal is just for interactive debugging - changes on top of it are a scratch filesystem, and so are intentionally not propagated through to the next item

[ionic plume]

There are a way to look / zoom on logs (stdout) of services. When run dagger call, I can look some line of logs after increase the verbosity, but it seems not possible to display int in full screen ... I look only 5 or 6 last line

[narrow nymph]

are you viewing the logs in the terminal tui - they should be there? you should also be able to view the full logs easily using dagger cloud

i think you can focus a single service using the keyboard, up-down-enter, etc (in the tui)

[ionic plume]

yep, I use the tui

[ionic plume]

I will try the dagger cloud

[humble sage]

When developing/debugging dagger, is there another way to get experimental version than checking out and running dagger engine container ?

[sharp marsh]

Yes, you can also de dagger -m github.com/dagger/dagger engine container terminal

[old lotus]

i want to run multiple pipelines in a single step context in my CI. Basically I want to run these commands in parallel:

• dagger call test1
• dagger call test2
  I only need the trace URL's from each to see the output, rather then look at mixed output from both.
  Is there a way that dagger can do this natively? Otherwise I'm stuck doing some bash hackery

[sharp marsh]

If you need a unique trace ID per pipeline then yes, there's no current way of achieving this in Dagger natively. So you'll have to run dagger call test1 & dagger call test2

[sick monolith]

Can anyone else on a Windows OS confirm if dagger core container from --address IMAGE terminal fails for you? I've created a bug report, im pretty sure it's a bug but another windows user to make sure its not just me would be good. https://github.com/dagger/dagger/issues/9096 Ive created this for now, but if we find out it's something quirky, I'll close it.

[lapis slate]

terminal fails on Dagger windows distribution

[stoic knot]

Would/could sudo rm -rf /var/log/containers/* cause issue for Dagger while it is running or doing things?

[narrow nymph]

shouldn't do, assuming you're running dagger using the default setup (running in docker)

[stoic knot]

As in this is a bad thing to do?

[narrow nymph]

it shouldn't cause an issue - docker doesn't write to those log files, and neither does dagger

[stoic knot]

I'm having an issue where some runs start failing to push images, getting 401/404 with GCR. My best guess is that I'm getting hit by cache cleanup. Wanted to double check this rm -rf wasn't the issue here

[sharp marsh]

by any chance is this in long running pipelines? I'm wondering if by some reason the GCR token might have a very short expiration time and Dagger might not be able to renew it automatically

[stoic knot]

yes, from 30-45m, depending on cache hits. We use the credsHelper setting for Docker that references/maps registry domains to tools

[sharp marsh]

this might be one of the reasons. I can't really recall of the ~/.docker/config.json credentials are loaded in the engine. I can check in a bit 🙏

[stoic knot]

thanks, fwiw, we use a system-wide /etc/docker/config.json (or something like that, default location)

[fossil pine]

I've been thinking about the layer cache vs cache volume approach in the context of how tools (in this case vulnerability scanning tools) manage their databases...and how that works (or doesn't) with Dagger.

The two examples in mind:

• Trivy (see recent ghcr rate limit drama)
• OSV

Trivy distributes its database as an OCI artifact (not an image though, so dag.Container doesn't work). It's really-really hard to cache, because it's updated regularly (every 6 hours I believe), but it's essentially a moving tag in the registry. A good alternative would be a timestamp trimmed to every 6 hours of the day, so you can consistently calculate the last tag. Also, arbitrary OCI artifact support would be nice in Dagger (not sure how that would work in practice). But combining those two would maximize caching (layer cache in this case).

How to do it today? Use cache volume, bust the layer cache for each run and let Trivy update the database: https://github.com/sagikazarmark/daggerverse/blob/5b826062b6bc1bfbd619aa5d0fba117190c85aba/trivy/main.go#L84-L88

OSV distributes its database as files available from a GCS bucket. That itself wouldn't be a problem, using gsutil in a container could be cacheable, but there is no timestamp either to "automatically" bust the cache from time to time. You could use a little trick and bust the cache every X hours.

Offline database access is still experimental though: https://google.github.io/osv-scanner/experimental/offline-mode/

Lessons for me:

• There is no "standard" or set of best practices on distributing databases in a cacheable way, even though we have such a mechanism for other types of artifacts (container images)
• Besides container images, Dagger can't effectively work with other types of OCI artifacts (could potentially improved by userland modules for OCI (oras?) and object stores)
• Caching is key for performant pipelines, so this situation needs to improve

Any thoughts? How do you solve this with other tools?

[quiet tiger]

Is there a way to return stdout and stderr ?

And specifically as stdout and stderr

[sick monolith]

I may be misunderstanding you - once you have a dag.container, you can use StdOut or Stderr functions, which will give you the str. Or do you mean something else?

[quiet tiger]

I can concat them and return them yes, but I want to return stdout through dagger's stdout and stderr through dagger's stderr

I don't want to mix the streams

[sick monolith]

so you want something like .StdOutErr()

hmm.

[quiet tiger]

As long as that would return the strings to stdout and stderr to the caller sure

i.e. stdout -> dagger's stdout and stderr -> dagger's stderr ... so dagger commands can be better used in scripts with dagger call -s

One potential way to do this, thinking from the go api perspective anyway, is any function that returns (string,string) will map the first string to dagger's stdout and the second to dagger's stderr.

@sick monolith does that make sense ?

or is there already a way to do this that I haven't found?

[sick monolith]

yeah, i think if you could give an example it might help

i still dont quite get you, theres so many ways people have done things

i guess right now, the only option is checking if sync or a ctr returned without an err, and if it did, then you need to call ctr.StdErr()

[quiet tiger]

I run a command in a container. it can produce output it's stdout AND it's stderr.
How do I map that back to dagger's stdout and dagger's stderr?

[sick monolith]

those functions already do that for you, from the last cmd that was executed right?

[quiet tiger]

so that dagger call -s <something> > output 2> errors works

[queen bramble]

I'm looking for examples on how to make my modules more modular by leveraging chaining patterns (with optional steps). My feeling is that I'm missing something obvious or just using dagger in a suboptimal way. I appreciate any support/hint/examples on this. Thanks!

The problem I want to solve: My module has several functions that all rely on the same base but with slightly different configurations. Assume two functions can work on an unmodified base but three other functions can (but don't need to) apply the same modifications to the base. Basically I'm looking for some kind of builder pattern.

Currently I solve this case by passing through (a) base arguments (b) shared add-on config and (c) the specific arguments function in each of the "terminating" functions. I'm looking for ways to remove some of the repetition while still providing some guidance/support on what the correct order of calling them is. I've seen examples where these intermediates are fields of the module and the intermediate functions always return the module itself (see https://docs.dagger.io/api/return-values#chaining). But I think those also have issues guiding the user on what can be called at which point in the chain.

--> continues...

[quiet tiger]

Those functions get me the text, but how do I map them to the dagger command's stdout/stderr so that I can do: #general message

[sharp marsh]

I've been thinking about the layer cache

[queen bramble]

Chaining patterns with modular add-on

[sand gorge]

Sorry I've been a bit quiet recently, folks. Family stuff and preparing to present Dagger to the PHP community at SymfonyCon Vienna.

People are absolutely buzzing about DaggerPHP and excited to start using it in their projects. Here are some photos and such from today's event

They are not just applauding me, they're applauding the Dagger team for all the awesome work 👏 😎

[chilly arch]

Community call starting now! See you on youtube or X!

[chilly arch]

You are such an inspiration @sand gorge! Thank you so much for spreading the word of Dagger and all your work on the PHP SDK!

[chilly arch]

Thank you so much for the demo @sick monolith ! If anyone wants to check it out, here is the repo https://github.com/pjmagee/dota2-helper

[split olive]

Thank you @chilly arch ❤️

[chilly arch]

Amazing demos @hasty shale @sick monolith @wraith niche !

Thank you so much for joining the community call. If you missed it, you can check out the full recording here:
https://youtube.com/live/3NTsrpKt4FU

[stoic knot]

@sharp marsh did you find out any more about the oauth expiration during long builds? From what I can tell, Google has a hard 1 hour limit on the tokens, though in testing it seems to expire in less time.

[ionic plume]

I have tested dagger cloud, but I doesn't found the service logs anywhere ?

[sharp marsh]

can you share your trace please?

[stoic knot]

Is it possible to run two versions of Dagger at the same time on the same host? We have an issue that one of our release branches is now out of date, but we need to make a hotfix for a customer. The SDK in that version of the code no longer aligns with the engine we run.

[sharp marsh]

yes, the only caveat is that you'll have to manually provision the engines and set the _EXPERIMENTAL_DAGGER_RUNNER_HOST variable for the clients. Otherwise, if you use the automatic provision, they'll garbage collect each other

[stoic knot]

ah yes, we already manually provision, forgot about the ENV var. Would need to be even more mindful of the cache prunning settings we use.

[opal mist]

does any know how to https://github.com/jpadams/azure-for-dagger/blob/ce1d338fffd5e3f973a1808b10ac48569c05e104/main.go
i need to figure out how i can locate the ~/.azure/credentials on an Azure DevOps Agent

[slender charm]

azure-for-dagger/main.go at ce1d338fffd5...

[opal mist]

Has anyone tried to execute a Temporal.io workflow from within a dagger.io module?

[slender star]

Love to try with you! Been talking to some Temporal folks about how this might look.

[opal mist]

i made it work actually, though since i am running dagger engine in docker you must use the host.docker.internal:1778 to access the temporal server on the host deving locally

[frozen basin]

has the Pipeline("foo") call been replaced somehow?

seems like things are getting fairly mature with the project but I still dont see a straightforward way to name my pipeline runs or groups of steps

[slender star]

Are you using Dagger Cloud at all?

[frozen basin]

yup

[slender star]

ah, okay, you still need better grouping or custom names?

Are you using the GitHub app too?

Sorry will go into thread 🙂

[frozen basin]

👍

[ocean karma]

Updating Dagger is quite a headache at the moment.

I try moving from 0.13 to 0.15 and the new changes with the service bindings just throw me.
It was a mistake nesting services like i did

svc ReverseProxy > svc Application [ svc php-fpm, svc db1, svc db2, svc redis ]

• WithServiceBinding fails with "! no command has been set"
• ok modify AsService(dagger.ContainerAsServiceOpts{UseEntrypoint: true}) since all have proper entrypoints, but I am unsure if this is the correct resolution
• now it gets stuck when the Service is getting "bound"

I try running with the debugging flag, but this yields no hint.

At the moment I can not produce a simple example of my (mess) solution, but maybe someone has pointers how to proceed?

[sharp marsh]

if you're relying on entrypoints in your images then yes the breaking change is that you need to use UseEntrypoint as you're currentlysetting

[ocean karma]

ok thats what I am already using

[ocean karma]

is the service, in this 'yellow' state already bound or is it stuck?

[sharp marsh]

it's probably waiting for the service to become ready. If you run dagger -vvv you should have more visibility about the healthcheck status

[ocean karma]

it gets healthy, state stays 'yellow'

[ocean karma]

I will abort this upgrade and roll back for now, might return another time

[sharp marsh]

I see you have multiple services. Are you sure all of them are becoming ready? Again, a yellow withServiceBinding indicates that it's waiting for the service to become ready

[ocean karma]

The output is very hard to parse so I can't tell, it was working prior, so my assumption is that the entrypoint is not or wrongly executed due to my changes

[sharp marsh]

well..besides adding the UseEntrypoint we're also using the container's default CMD if that's set. So maybe that algo changed the behavior in your case somehow 😬

are you using custom built images for services?

[ocean karma]

They as partially custom based on common images like redis, mariadb, php-fpm, we just added some configuration.

These changes hit me pretty hard due to the current structure I've built.

I am considering throwing it all out and starting from scratch, even though I am unsure how to avoid the same pitfalls.

[sharp marsh]

since it's code, it should be "relatively easy" to isolate the problem. What I'd do iny our case for example, is to create a function that returns the public_db service in your module so you can do dagger call mymodule services public_db up. This way you can not only start each service in isolation locally, but have a straightforward way to test if they're starting correctly.

[ocean karma]

I've not yet made a dagger module out of this, I will look into that

[vital rapids]

is my DNS stale?

Resolving dl.dagger.io (dl.dagger.io)... 18.160.200.72, 18.160.200.90, 18.160.200.79, ...
% sha256sum dagger_v0.15.1_linux_amd64.tar.gz
228c046ecaaa5a1cb5bc8dea12b8341f34e1128cc834f2dcd921bcbe6e72d10c  dagger_v0.15.1_linux_amd64.tar.gz

vs

wget https://dl.dagger.io/dagger/releases/0.15.1/dagger_v0.15.1_linux_amd64.tar.gz
--2024-12-12 14:23:11--  https://dl.dagger.io/dagger/releases/0.15.1/dagger_v0.15.1_linux_amd64.tar.gz
Resolving dl.dagger.io (dl.dagger.io)... 3.171.85.102, 3.171.85.83, 3.171.85.49, ...
% sha256sum dagger_v0.15.1_linux_amd64.tar.gz
5230bfd932ecaa02266efd830c5cf793367433242aec2ff6b3c133238bf8d266  dagger_v0.15.1_linux_amd64.tar.gz

checksums.txt from both of those are the same, the 5230

[sharp marsh]

wow, not sure where the 18.160 are coming from since ours re generally 3.171 cc @heavy gazelle. @vital rapids Are you using your ISP's DNS by any chance?

[vital rapids]

yes, the 3.171 resolution is utilizing work VPN settings

[sharp marsh]

makes sense.. then your ISP is probably doing something funky 😬

[vital rapids]

makes sense.. then your ISP is probably

[stray valve]

What happened to the Dagger VSCode extension? I can't find it anymore

[sharp marsh]

that was for the old CUE version. It's not needed anymore since everything is code now

[sand gorge]

V0.15 release is looking soooooo good. I posted it on my socials

[potent lagoon]

I'm running containerd and buildkit inside a dagger container and I'm noticing that traces from OTEL traces from those tools themselves are showing up in the dagger output. I assume that's because dagger is driving itself via OTEL, but it's super weird. Thoughts on how to disable it?

the dagger.cloud UI shows them as "orphaned traces" and the github actions out shows them inline in the log

[slender charm]

I think this is a feature 😅 but seems like could use some more polish to address all cases.

For example a lot of folks instruments their test suites with OTEL and this shows up in the log output and cloud

Agree would be nice to have more control but I don't know of a simple way at the moment.

Could you please file a GH issue and include an example trace URL (if its public great! if its private only dagger team can see it but would be very helpful to help us debug and polish)

[potent lagoon]

trace url on dagger.cloud?

it's really the github actions output that is annoying, the traces show up and absolutely flood the output

here is an example

[potent lagoon]

I'll just unset the OTEL env vars I think.

inside the container.

[winter linden]

@potent lagoon maybe dagger -q will keep the output less crazy verbose?

[potent lagoon]

in the terminal locally it does

but the github actions output using the dagger/github-actions thing is verbose AF

[warm temple]

I wonder if you could set the OTEL_ variables in the container running containerd/buildkit

[stoic knot]

Are there some docs or examples on using tools like Snyk or Sysdig for container vuln scanning? It occured to me many of them expect the image to scan to be available in the local container runtime. How does this work in the Dagger paradigm where images don't end up in the local runtime?

[potent lagoon]

They are already set, I’d be unsetting them

[slender star]

We could add that as a flag or eventual default. Quieter output.
Could you make an issue for your desired behavior? 🙏 https://github.com/dagger/dagger-for-github/issues

[old lotus]

is there a pragma for default secret ref? E.g. // +defaultSecret="env:MY_API_KEY"

[slender star]

No there isn't, in keeping with the security model (scenario where if that worked and someone didn't realize it was there, and their secret got scraped and sent to pastebin), but there is a lot of discussion/thinking happening around secrets, where to get them from, and how to safely handle them: https://github.com/dagger/dagger/pull/8730 (for one). cc @cloud canyon

[old lotus]

Understood the security aspects. I mean to suggest providing a default for the secret reference, and not the secret itself.

For example, to set the default behavior to load from an environment variable.

[winter linden]

It's definitely something we're considering. The security concern exists even if you default to a secret reference: a malicious module could access a sensitive variable on your system without you knowing.

This can be mitigated with a carefully designed trust system, or with virtual env vars.

[ancient escarp]

Question: is there a way to emulate a volume mount in dagger similar to docker run -v /local/dir:/app container->withMountedDirectory() isn't it. Do we need to add a ->withMountedDirectory() to the service primative as well?

[winter linden]

There isn't yet but we are working on it, see https://github.com/dagger/dagger/issues/6990

[tired moth]

Can we have this please in 0.16?

[ancient escarp]

reading through that it seems to be solving a slightly different use case; while it would cover most things, it's only a host->container binding using a watcher to resync files when they change. I do think this would be a useful feature; but also having Service->withMountedDirectory() which emulates docker run -v would be a good thing to have as well.

[stoic knot]

Should attached services persist when we drop into an interactive terminal after an error?

[sharp marsh]

not currently: https://discord.com/channels/707636530424053791/1317571007464210452.

You can use .Terminal as a stopgap to troubleshoot

[chrome vessel]

Can you use multiple modules with dagger shell?

[winter linden]

yes. Just use the module address as a command in your pipeline. You can do this for any number of modules.

for example:

github.com/dagger/dagger/modules/wolfi | container | with-file /bin/dagger $(github.com/dagger/dagger/cmd/dagger | binary) | terminal

[cerulean wraith]

Hi @ocean karma, I worked on these changes, and I am happy to pair with you to understand the issue and help you through this upgrade.

[dusty kestrel]

Hey team,

I am working on integrating dagger with our existing CI/CD (gitlab) using k8 setup
and have notice few issue

1. We have a large mono repo, due to which directory copying operation itself take more then 1 min
   further, we have implemented the recommendation mentioned here https://docs.dagger.io/api/filters
2. We have some large jobs and this has lead to scaling issues at our end. The recommended approach is to use dind with dagger cloud. Unfortunately can not go with this option due to lengthy compliance process at our end.

Is there any other option or recommendation that I can try. Is it possible to bring my own backend for dagger cloud ?

[slender star]

Directory Filters | Dagger

[ocean karma]

I was able to get it working.

• What I learned was, DefaultArgs are removed if you set a new Entrypoint
• Migrating all my WithExec([]string{}, withentrypoint) to the propper entrypoint and default args

It is really difficult, at least for the way I have structured my code, to find the issues, the log is not enough. So I have to resort to bisecting it, which takes time.

[ocean karma]

In my own time, I will refactor it and make it a proper dagger module, so I can run services and commands in isolation

[cerulean wraith]

I am glad you sorted it out.

one other recommendation that I have heard from the team is to use args in AsService api instead of using entrypoint and defaultArgs.

that kind of makes things quite obvious IMO for someone looking at your code e.g.

[ocean karma]

When I would have to say UseEntrypoint: false and just stuff it into Args, then it would be all at the same line, should work, but I enjoyed using the builder pattern

[ocean karma]

When I now refactor it, if I am not mistaken, to be more like dagger module/functions, I'd have to configure the Entrypoints/Args anyway, then have a separate call in my regular pipeline where it makes the 'AsService'

[cerulean wraith]

yes, that should be fine. except that there are some discussions going around in dev channel on how future of Entrypoint should look like in dagger ecosystem.

[ocean karma]

If there where a way to see the active "step", and possibly where it is stuck, would be amazing.

the dagger run output is nice but this is missing for me.

[ocean karma]

Did anything change regarding Caching Volumes?

I've used them in the past to store logs and other data from containers and then mount the volume to export data.

[cerulean wraith]

Did anything change regarding Caching

[chrome vessel]

Thanks!

[sand gorge]

Fresh machine - I just installed docker and dagger latest (0.15.1) 🔥

🧑‍🍳 I'm cooking some fresh modules right now 🙂 let's see how quickly I can do it

[chilly arch]

https://tenor.com/view/kessoku-hood-gif-16583213151155611644

[sand gorge]

I wanna copy some source code from a Git repo - and load it into a Directory object .. what's best practise here ? I'm seeing some fancy behavior of CLI arg --source=<github.com URL> is this a good plan? what about writing it all in Code and making it a Directory object? any good examples of this?

[sand gorge]

UPDATE: It's kinda neat how the "Directory" type can be a path or a git url 😎 I'm digging it - kudos to whoever's idea this was 🫡

..

New module - for PHP developers to run compiler like (static analysis) checks on their project

Works first time off to the daggerverse now

[sand gorge]

Thanks to @slender star - we had a sweet video call and pairing session on some funky things I was attempting and he taught me dagger shell kung fu 🥋

....
Symfony project can now use the phpstan module to analyze its own source code

🚀

[slender star]

Always a pleasure pairing with you @sand gorge 🙌 😁

[chrome vessel]

Is there a way to configure the location of /var/lib/dagger ?

[sand gorge]

@chrome vessel
I don't have this on my machine, could be distro dependant .. what's stored in your /var/lib/dagger/ ?

[chrome vessel]

i think thats the cache

[sand gorge]

Okay, I'm not sure (yet) but there is definitely going to be an environment variable .. kind of like $DOCKER_HOST or whatnot

I wonder where my dagger cache is located? 🤔 curious to learn too

[chrome vessel]

looks like its a constant EngineDefaultStateDir

[sand gorge]

what does the code tell you? is there an ENV var to pick this up? 🙂

[chrome vessel]

no, but i think it can be configured in the builkit toml file

[slender star]

Seth, could you share where you need it and why the default doesn't work for you?

[chrome vessel]

trying to mount it to a cache dir in codefresh

[chrome vessel]

In codefresh all the caches live in /codefresh/volume so i would want dagger to use /codefresh/volume/dagger or something like that

[chrome vessel]

I may actually be better off running the engine as a service so that all the steps share the engine, otherwise i may have multiple engines writing to the same cache anyway

[sand gorge]

It's a desirable idea, if you have multiple developers or very large stuff going on 🙂 Kind of like a shared $DOCKER_HOST or something, similar concept

[chrome vessel]

How is with-default-args used?

[sand gorge]

New daggerverse module exists - for psalm "php static analyzer" - here is the Symfony project now using it with their own custom psalm ruleset

🙂

[slender star]

https://github.com/dagger/dagger/issues/9190#issuecomment-2549676262 affects the CMD aka default-args of a container

[ancient escarp]

Quick Q. If I have function A(Directory $dir) which calls function B(File $file) with a file from the directory; function A will be rerun if the contents of the directory changes between runs (eg the cache can't be used) however if the parameter to function B is the same file(s) and they haven't changed since the last run will it be able to use the cached result from the previous run?

[winter linden]

Yes 👍

[ancient escarp]

that's what I hoped; going to go build something which will look quite convoluted but will take advantage of this 😄

[ancient escarp]

hmm, would the cache still work if the function call was internal to the module. I'm assuming so, as the API calls will eventually do something like $container->withFile()->withExec() which would be cached; so not as efficient but still cacheable

[winter linden]

• calls to dagger functions themselves are never cached, although we plan to change that (see https://github.com/dagger/dagger/issues/7428)
• calls to core dagger API are always cached (unless you intentionally bust the cache by changing an input). Cache is shared by all modules, so if module A calls a particular core function, module B can get a cache hit if it makes the same call

[chrome moss]

Hey all! I've been stuck on this for a while and hoping you had suggestions. Is there a way to both print out file contents and return a file at the same time/function call? I'm trying to write a wrapper that runs a bunch of lint tests, then returns a directory of result files, but I want someone to see the content of those files while the tests run. If I try something like fmt.PrintLn you can't see that output unless you increase verbosity in dagger which leads to seeing a lot of extra things I don't want

[analog raven]

Community Calll starting now! Whether you’re just starting with Dagger or you’re already an expert, there’s something for everyone.

🔍 What’s on the agenda?

• How CivoCloud + Dagger simplify Kubernetes and caching with Terraform by Dinesh and @warm temple

• Let’s teach an AI to use Dagger by @winter linden

Youtube: https://youtube.com/live/zULzUoWHDK8
X: https://x.com/dagger_io/live

[split olive]

Hello from Switzerland 👋

[winter linden]

https://x.com/solomonstre/status/1869793118478123192

Solomon Hykes (@solomonstre) on X

We're live! Dinesh and Kyle are showing how to deploy Dagger at scale on Civo Cloud, with cache data automatically shared on an object store

🧵

Twitter · Dec 19, 2024, 5:13 PM

[chilly arch]

You can get the Civo $250 free credits that we mentioned here - https://civo.io/dagger-workshop

The Dagger team will be at Civo Navigate too, so we'd love to meet any Daggernauts there! https://www.civo.com/navigate/north-america

[chilly arch]

If you missed the community call, you can see the recording here: https://www.youtube.com/live/zULzUoWHDK8

Thanks to everyone who joined!

[ancient escarp]

no, however what you can do is return another dagger object which has a function to retrieve the directory and output as separate calls.

[round cradle]

Is it possible to nest directory filters? Doesn't seem to work but figured it would be convenient when calling functions from another function, that can also be called from the CLI, if that makes sense.

[round cradle]

Ah, found https://docs.dagger.io/api/filters/#post-call-filtering, disregard above.

[slender star]

Hey all! I've been stuck on this for a

[sand gorge]

So I'm having some "timeout" issues with my dagger Service classes, some are just not "booting up"

https://github.com/dragoonis/symfony/blob/7.3/.dagger/src/Symfony.php#L104-L117

Port 8091 is couchbase() - https://github.com/dragoonis/symfony/blob/7.3/.dagger/src/Symfony.php#L221

Port 27017 is mongodb() - https://github.com/dragoonis/symfony/blob/7.3/.dagger/src/Symfony.php#L213

What's going on here?

PS: It happens every single time .. the same thing each time.

[slender star]

might be bit by needing to say --use-entrypoint

[cunning idol]

Hi there!

I'm just starting to learn Dagger, experimenting with the Rust SDK.

One thing that I'm struggling to wrap my head around is how to structure my repo so I can access all files from within my Dagger context.

As far as I can tell, a Directory cannot access relative paths above where my dagger program runs from, so if I run cargo run from within the foo/ci/ directory, then I cannot access files in foo/bar. If I use ../bar, it complains that I'm escaping the current directory.

Can I reference the repo root, somehow, or do I have to put my Dagger program in the repo root?

Merry Christmas! 🎄

[glad jolt]

Hi there!

[sick monolith]

Grok got it wrong so many times before, this is the best one, but you all get the point 😆

[high seal]

Hello everyone, I am new to Dagger and I was wondering if following would be possible:

1. I have a Java project built with Gradle
2. Test for the App are written using Testcontainers
3. I want to build the JAR using gradle build in Dagger function

I am unable to get this working,because testcontainers can not start the needed containers.

Up to now I have tried running Dagger engine using the module provides by @winter linden, and mounting it as a service, but with no success.

Is there a template where something like this can work?

Second thread from me: I find the Dagger technology very interesting. I would like to learn more about how it works internally, and maybe try contributing something back during next year. Can someone recommend where to start learning about internals of Dagger, and things I should be acqauinted with to work on it productively?

I assume learning Buildkit and Linux container technologies is a must, but I am currently unable to find exhaustive and well written resources for learning

[plucky pawn]

do you have some example code we can look at?

[high seal]

Not at hand, but I can supplement with code in few hours. I am simply starting the dag.docker().engine() then I bind it as a service into eclipse temurin container and set DOCKER_HOST env var to point to tcp://docker.2375, where I have bound Docker service to name "docker", I mount the directory with source code and lastly run gradle build with gradle wrapper

I can provide the error messages as well in few hours

I am using Python SDK and running in Windows against Rancher Desktop

Gradle Testcontainers

Learning Dagger internals

[humble sage]

I am trying to container.publish() to a registry ran as a dagger service. Is there something that would avoid it to work ?

[sharp marsh]

yep. specifically https://github.com/dagger/dagger/issues/6411. cc @modest pier

[modest pier]

i spent a solid couple days trying to get this to work, it's fairly difficult because buildkit's understanding of images are more coupled to worker implementation than other Sources. need to go back to the drawing board. that said, everybody agrees it should work. might require some upstream bk changes. the issue marcos linked has a workaround that's ugly but at least functional.

[sharp marsh]

FWIW there's a workaround to achieve this which doesn't include calling publish but exporting the container with AsTarball and then using skopeo to publish the tarball to the service 🙏 . LMK if that's an option for you @humble sage

[sharp marsh]

oh, just realized that @fossil pine commented about that here https://github.com/dagger/dagger/issues/6411#issuecomment-2354072245

[modest pier]

lol yeah that's what i was referring to

the issue marcos linked has a workaround that's ugly but at least functional.

[winter linden]

Ugly on the outside, functional on the inside

[high seal]

Hello, a question here, if I bind some services into container, and then call build to build from Dockerfile, can I access the services in Dockerfile steps?

[high seal]

Tested this with simple echoserver, and the amswer is no.

[sharp marsh]

ref: https://github.com/dagger/dagger/issues/6089

[high seal]

Hello everyone! In my Dockerfile I would like to access the "host.docker.internal", basically the host container, and then I would like to build that container using the build command. This does not seem to work in any way. Here is some example code to demonstrate the failure:

    @function
    async def echoserver(self, context: dagger.Directory) -> str:
        echoserver = (dag.container()
                      .from_("ealen/echo-server")
                      .with_exposed_port(80)
                      .as_service(use_entrypoint=True))
        return await (dag.docker()
                .cli()
                .container()
                .with_service_binding("echoserver", echoserver)
                .with_directory("/src", context)
                .with_workdir("/src")
                .with_entrypoint(["/bin/sh", "-c"])
                .with_exec(["apk update && apk add curl bind-tools socat"], use_entrypoint=True)
                .with_exec(["socat tcp-listen:80,reuseaddr,fork tcp:echoserver:80 & curl localhost:80/params?query=mode && docker build --add-host=host.docker.internal:host-gateway ."], use_entrypoint=True)
                .stdout())

Dockerfile:

FROM alpine:latest
RUN apk update
RUN apk add bind-tools
RUN apk add netcat-openbsd
RUN nc -v host.docker.internal 80

Note that the curl command before docker command currently succeeds, meaning that socat is working, but docker build command fails.

[sharp marsh]

replied with a stopgap here: https://discordapp.com/channels/707636530424053791/1236088075507994664/1322741123122466928

[stuck wyvern]

Working on updating my dagger version.
There used to be a client.Pipeline function (and similar on other objects, IIRC), was there something that replaced this?

[sick monolith]

from my understanding (i didnt use pipelines) they were removed because now theyre basically redundant, dagger cloud is what I think replaced this functionality.

Did a bit of a search here https://github.com/dagger/dagger/releases?q=Pipeline&expanded=true, can see when it was removed after being deprecated, but cant find much depth in those release notes regarding Pipeline removal

[sharp marsh]

@stuck wyvern since a few revisions ago we switched all the event systemto use the OTEL protocol by default. So now, if you instrument your code with OTEL (the Dagger generated code has some helper functions for this), all the spans that you define will be present both in the CLI and cloud

[sharp marsh]

there's a helper function we use in our tests to create a span for each Go test https://github.com/dagger/dagger/blob/44356172c3443c9fa58b69a689d101affaa0252f/testctx/testctx.go#L55-L73

and here's how that is being displayed in cloud: https://dagger.cloud/dagger/traces/be8282f86f3caa164ec16c7708adbf9e?span=6d1cbd51d99c4335

[tired moth]

how can we add any code custom trace in dagger any docs/examples? We are using TS sdk.
let's say we ahve to do web app and we want to trace some code of that aoo through dagger then how it's possible?

[stuck wyvern]

That visualization is so much nicer than viewing stuff in jaeger, good job 😄

[winter linden]

A very happy new year to all Daggernauts!

We are thankful for this amazing community, and for everything we built together in 2024.

We have big plans for 2025, I can't wait to show you what we have in store.

This year I'm celebrating with my family in beautiful Lake Tahoe, California. How about you?

[heavy karma]

I'm working on a cookbook recipe for this, it is still WIP and needs review but it may help you: https://github.com/dagger/dagger/pull/9301/

[tired moth]

Is it going to have ts example/guide?

[heavy karma]

there is a TS example already in the PR

[tired moth]

Was driving so didn't check.. thanks bro

[cerulean wraith]

I always loved Tahoe. though this is my fav pic from one of the visit

[split olive]

Nice! I stayed home to ski and snowboard. I live near Gstaad in Switzerland.

[marsh timber]

Hey all,

I was hoping to be able to export a list of dirs from dagger.Directory, essentially I'd like to loop a folder and build each dir. If not via dagger.Directory then I guess I can use a regular go package but thought id ask if theres an inbuilt function first 🙂

Im sure someone has had to do something similar before.

[narrow nymph]

you can loop the directories, and then just assemble them together in one output directory - doing something like dag.Directory().WithDirectory("build1", ...).WithDirectory("build2", ...)

or even something like this: where we iterate over a bunch of directories and look lint each of them

https://github.com/dagger/dagger/blob/7be649d7bc0bb98c53f3d31b23037d6393daaa0f/.dagger/engine.go#L180-L205

[marsh timber]

Thanks Jed 🙂