general

[elfin frigate]

[winter linden]

packed agenda!

https://tenor.com/view/helmet-on-nick-zetta-basically-homeless-im-ready-ready-to-go-gif-17532760

[cunning jolt]

😮 whaaaaaaat I didn't know this was possible

[chilly arch]

For newcomers, feel free to ask questions, add comments, or send gifs here 🙂 I will ask your questions live to the speakers as they come in.

[winter linden]

It's not, that's a dev branch 🙂

[elfin frigate]

possible soon! ™️

[sick monolith]

this is an awesome dx improvement

[cunning jolt]

oh wow.. this is fantastic! Massive help. I've been creating separate terminal/debug functions for all my modules.

[winter linden]

YES! Been dreaming of Directory.terminal()

[cunning jolt]

Oh.. For Directory.terminal() from a corp environment I'll always have to tell it to proxy from my container registy. Not a huge deal but can get tedious.

[wispy tapir]

This is absolutely amazing, breakpoint in CI...

[elfin frigate]

🤯

[cunning jolt]

omg! Mind blown indeed!

[queen bramble]

This is super useful! 😍

[honest oasis]

Perhaps call it Breakpoint(...)?

[winter linden]

https://tenor.com/view/one-more-thing-loki-thinking-gif-18290380

[cunning jolt]

I was thinking the same..

[honest oasis]

and perhaps have dagger lint or something complain when you didn't remove it 😄

super useful, no doubt

[winter linden]

putting Andrea's demo at the beginning, rookie mistake 😛

[chilly arch]

had to wake people up :p

[elfin frigate]

feedback: can you make it less cool

[wispy tapir]

One of the most impressive thing I've seen on Dagger, this is such a killer feature and sooo useful

[elfin frigate]

the old prototype - it's a small miracle that the data is still there, since it's all in memory: https://dagviz.fly.dev/trace/907c16c11a6f05b4c1a479817b5f91b4

[winter linden]

@cloud canyon I actually have a nit on the PR, but feel bad getting in the way of the awesomeness train 😛

[cloud canyon]

For debugging, first PR (this is the Container.Terminal() and Directory.Terminal() API):

https://github.com/dagger/dagger/pull/7586

[winter linden]

I worry I'll get mobbed if I delay this PR by even 5 minutes 😅

[cloud canyon]

Yep, the default environment is overrideable by any dagger.Container

[elfin frigate]

pretty sure we swapped between WebSockets and SSE multiple times on this journey, each of them has its own cruft

https://tenor.com/bJ8zA.gif

[cloud canyon]

[chilly arch]

For those who don't know what Dagger Cloud is - https://dagger.io/cloud

[elfin frigate]

also: those titles will become shorter in the future 😅 - will get to that soon. very unfortunate that the differentiating part ends up at the end, which gets cut off in GitHub

[honest oasis]

Wachines is to Machines what Wario is to Mario?

[timber crater]

Nice example of using the "back end" for what it's best at

[honest oasis]

(dad joke alarm)

[swift inlet]

Yes to the flame view 😍

[winter linden]

If it weren't complicated you wouldn't need to look at the trace 🙂

[glad jolt]

the new trace navigation is awesome, definitely recommend trying it out

[swift inlet]

the amount of details is paying a lot 😍 I didnt even realize that line change but it feels so natural

[winter linden]

Tell us what's coming next @glad jolt @elfin frigate !

Love the shorter function title, any chance we can propagate that into the Github checks (super long as @glad jolt mentioned earlier)

[elfin frigate]

yeah that's the most important spot to fix i think, it'd be a top priority

[winter linden]

Welcome Arsh!

[timber crater]

Yay Okteto!

[cloud canyon]

Not really kidding -- the debugging demo will eventually get integrated into Cloud so you can click your way into a terminal 🙂 @elfin frigate

[chilly arch]

welcome @pine crag ! Thank you so much for taking the time to build the module and share it with us

https://daggerverse.dev/mod/github.com/okteto/dagger-module

[timber crater]

Hmm Cypress function... interesting...

[chilly arch]

Thanks for sharing @pine crag ! Very excited to see an official Okteto module in the Daggerverse

[pine crag]

Here are the repos I showed in the demo!

1. https://github.com/RinkiyaKeDad/okteto-dagger-sample (sample repo to play with the module)
2. https://github.com/okteto/dagger-module (the code for the module)

[cunning jolt]

This is interesting. I can see a use case for CI/CD pipeline development with built in documentation. Maybe even have a final form where dagger can run the markdown file e2e as a full pipeline

[timber crater]

restructured text and Sphinx could be included as a module
(actually, these days, markdown is almost as powerful

[warm temple]

Totally, I'm picturing replacing a CONTRIBUTING.md with a notebook containing dev tasks

[winter linden]

I didn't understand the part about IDs causing a problem in dagger query

[elfin frigate]

@honest oasis if you're curious about that cryptic blob error when reusing IDs: it's because somewhere in that ID it depends on an ephemeral blob that was uploaded from the local disk, and those get reaped when the originating session goes away. Is this flow fundamentally dependent on local files? Or could it be fed e.g. a git repo?

[winter linden]

LOVE the notebook use case!

[timber crater]

Yes, similar to the way doctest will run tests against code snippets marked with >>> except using dagger as the "test" runner...

[honest oasis]

i believe @elfin frigate explained the problem I ran into with dagger query. sounds like it was at my own making @winter linden.

fwiw, that explains why some of the serialized-proto IDs looked different than others depending where they originated from. sounds like I should be able to "fix" this myself.

here's a snappy video of the Runme<>Dagger integration prototype in case you missed it: https://www.youtube.com/watch?v=g2Is2zY91wc

Totally, I'm picturing replacing a

[scenic cedar]

I'm very happy to help you if you want but I don't know if i'm qualifyied for this task, BUT I WANT TO LEARNNNNNNN

[winter linden]

One idea that's been in the back of my head for a while - what if you could "compile" your pipeline like a static binary? Your module (pre-built) + all its dependencies (pre-built) + dagger engine (locked down version) + SDK (pre-built, locked down version) -> all bundled into a docker image so you can just docker run your pipeline with minimal dependencies. Not exactly what you are talking about @narrow nymph @scenic cedar but seemed relevant

[narrow nymph]

aha i have thought about this - the big issue with something like this, is that trying to capture all your dependencies is currently tricky

like. suppose that you take a version number "vX.Y" - and you use that as a paramater to download a docker hub image based on it's tag

in terms of what i'd like the offline mode to look and feel like - i think what we want is to use buildkit sourcepolicies here - that lets you replace an image ref with a local oci directory you might have

or at least, that's the base primitive we could use for that

[winter linden]

Yes it would only apply to the module dependencies, and in that way wouldn't give you a real airport mode... But it could be nicely complemented by the idea of "lookup functions", which would be special functions annotated so that the engine caches their output in the dagger.json, as a universal lock file. Then pre-building could involve calling all lookup functions, which will pre-warm the cache, and then bundling that cache in the image

[proven dune]

Love this idea and would be something I would be willing to attempt to contribute to. I am a pretty competent developer.

[winter linden]

The lookup functions + universal lock file, or the "compile everything into a docker image" ?

[tulip fiber]

anyone experiencing pipeline kill cause of memory or cpu? version 0.11.7 getting
Process finished with exit code 137 (interrupted by signal 9:SIGKILL)
on a working pipeline

[pseudo stream]

There is a thread looking into this here: https://discord.com/channels/707636530424053791/1248533001915334676 From what we know, it's an ephemeral issue and not actually necessarily that the process got killed due to too much memory usage (though that is possible to happen too and will confusingly result in the same error message).

We know at least part of the problem is that the error was missing an annotation that caused it to not be the expected type, which will be fixed in our next release v0.11.8. We are not yet 100% sure if that's the entirety of the issue or if there's more going on. We haven't seen any of these problems in our own CI after a bunch of fixes+simplifications in the last few days, but can't know for sure until we do the release and see if it goes away for everyone. Keeping a close eye on it.

[proven dune]

Compile everything

[simple condor]

Some feedback for consideration - after a few days finally trying out the new modules approach, I'm switching back to dagger run and switching back to 0.9.11 to get host back. The modules-based workflow simply isn't ready for prime-time, making too many parts of the experience worse. I expect that fixes for them are already planned, but for now upgrading from 9 to 11 means:

• Having to wait at least 20s (usually at least 50s) for setup every time I try to run a pipeline (after reverting, it's down to a few milliseconds)
  • It even takes less time to transfer the source code to the context for some reason, like 3s vs 20s
• TUI is a lot worse
  -No longer able to scroll the output while it's running (still-running steps are cut off)
  • No longer visualizes the DAG dependencies with arrows
  • Removes completed steps unless --debug is used (but that's excessively verbose compared to previous CLI)
  • Output less accurate - after reverting, actually shows 16 concurrent "exec yarn nx run foo:lint" instead of continuing to show a handful of previous mount-cache steps when it's running the lint steps in my pipeline
  • Output cuts off exec step logs, even when --debug is used, making it impossible difficult to debug failures

[sick monolith]

Badge looks pretty good

[slender star]

Thread

[winter linden]

Has anyone managed to get ~/dagger.json to work in their home directory, so they can install dependencies globally? I keep running into .Trash: permission denied (on MacOS), and can't find the right combination of excludes to make it go away. Related: https://github.com/dagger/dagger/issues/6883

[stoic knot]

looks like the Go SDK is missing the latest two version v0.11.{7,8} and py/ts are missing the latest 0.11.8

[slender charm]

I think the releases are still in flight for v0.11.8

There was an issue with go 0.11.7 - I think there was a decision made to keep pushing forward

[stoic knot]

Is BuildKit filesystem really slow? Just the exporting of the Go binary we build takes longer than the Go rebuild on the host

... it's a 380M binary ... 😣
(mostly embedded data files for testing other tools) takes more than 10s to export

[stoic knot]

Is BuildKit filesystem really slow?

[narrow nymph]

yes, correct - v0.11.8 doesn't have the sdks published yet, due to some issues with some pipelines

the sdks will make it out tomorrow, alongside the helm chart

v0.11.8 is still usable by itself, if using modules, it's just with the standalone sdks that it's not quite ready yet

[kindred olive]

Hi! Trying to do dagger init —sdk=python within a corporate network (custom ca certs) is giving me a 509 error with some github package. If i do go install to the package it works just fine. Any idea why this could be?

[sharp marsh]

Hey there! Have you checked the section about custom certs in our docs?

https://docs.dagger.io/manuals/administrator/custom-runner/#custom-ca-certs

[kindred olive]

Hi Marcos thanks for the fast reply! The thing is this seems to be for the runner but my problem (I think) is in the CLI, which is where I am not able to inizialize the project.
I’m no my first steps of evaluating dagger so please expect the worse 😅

[pine crag]

Hey! Are there any folks running e2e tests in pipelines using Dagger modules? I'm looking for recommendations on modules which could help me test a React frontend + node.js backend application in the CI

[sharp marsh]

The CLI has to talk to the runner to initialize your project. Have you tried setting the certs in the runner?

[kindred olive]

I have created a custom container and connected it to the with the experimental_dagger_runner_host, with the certs folder mapped, but it is still giving me a 509

[kindred olive]

Is there any env var needed?

[sharp marsh]

This should be enough. If you exec in the engine container and you try to access anything that requires the cert. Does it work?

[kindred olive]

Yup

If I specify the SSL_CERT_FILE i can curl the conflictive github file

[sharp marsh]

Without specifying the cert file, does it work?

[kindred olive]

No

[sharp marsh]

It should pick the certs automatically if you put it under /etc/ssl/certs

Did you put them there?

[sharp marsh]

This is before the engine starts. So you either need to build an image with the files there or mount them when running it

[kindred olive]

I have mount them with this command

docken run -name customized-dagger-engine —privileged
--volume $PWD/ca-certificates.crt: /etc/ssl/certs/ca-certificates.crt
registry.dagger.io/engine:v0.11.8

Tried also with —volume /etc/ssl/certs:/etc/ssl/certs

[sharp marsh]

The first one should work.

[kindred olive]

Dagger init —sdk=python fails

[sharp marsh]

Can you check if the engine logs show anything about certs?

Maybe it's failing to setup the certs

[kindred olive]

It is not complaining apparently

[winter linden]

dagger for e2e tests

[sharp marsh]

@kindred olive not sure if this will work but can you try mounting your certificate with a different name inside the engine?

[zinc tinsel]

Is that the correct directory for certs? I mount them to /usr/local/share/ca-certificates/ and it works for me

[kindred olive]

Okay that fixed the issue!

[sharp marsh]

thx for the tip @zinc tinsel . Seems like a dupe of #1246470396086321162 message and the docs are outdated. I'll send a PR to fix that ASAP

[frozen basin]

Are there any example repos or docs about setting custom spans with the new dagger otel integration?

[winter linden]

We use it in a few places in our own CI:

• Go linter (one span per linted package: https://github.com/dagger/dagger/blob/main/ci/std/go/main.go#L107-L108
• Go mod check (one span per checked package): https://github.com/dagger/dagger/blob/main/ci/std/go/main.go#L116-L117

[winter linden]

May I ask what you plan on using custom spans for?

[frozen basin]

Yeah I just want to group things.

I'm still in the old sdk (no functions). I'd love to be able to group stuff so it's a little more readable, steps like 'Building docker containers or Building Image etc.

We also do some steps out of dagger, things like S3 uploads etc, would be nice to show them in the ui

I thought I saw some example of this in the first demo of traces, but hadn't seen anything official

Ahh linked examples look great, I'll start there.

[fierce sandal]

Been playing around with Dagger a lot. I built an AI Agent to help my wife and I figure out how to 'pay back our debt optimally' by comparing the benefit of paying back different debts aggressively (how much would you save in interests for each portion you pay) against the potential returns from investing some of the money in high performing stocks based on historical data. I use Dagger to send the output from the Agent to either myself or wife, depending on whose data is being handled via email

Blog: https://emmanuelsibanda.hashnode.dev/getstocked-an-ai-agent-to-help-you-pay-back-debt-optimally
Demo (will share a longer version when I present this soon):https://www.youtube.com/watch?v=glta2c_mxCQ
SM: https://www.linkedin.com/posts/activity-7209673224857481216-agiz?utm_source=share&utm_medium=member_desktop

Excited to share more about this and my Daggerized budgeting AI Agent in the community call

[sand gorge]

I have a general question about output buffers and stdout stuff.

When I run a withExec() I don't see the command running or the real-time output .. I only see the result when the command finishes ... is this the default Dagger behavior? am I supposed to see the real-time output? or am I only supposed to see the result once it's done.

[sick monolith]

https://docs.dagger.io/manuals/user/tui There is a section at the bottom there

@sand gorge have you tried running with -v? I think that's what youre looking for?

[sand gorge]

Ooh. OK. I will try. Gimme 5 min

[winter linden]

Normally you should see the output streaming in the TUI, assuming you use dagger call or dagger run

[tired moth]

is this right way to orchestrate K8 cluster through this way with dagger

import * as k8s from '@kubernetes/client-node';
import * as fs from 'fs';

// Load Kubernetes configuration
const kc = new k8s.KubeConfig();
kc.loadFromDefault();

// Create a Kubernetes API client
const k8sApi = kc.makeApiClient(k8s.CoreV1Api);

// Define a Kubernetes Pod configuration
const podManifest: k8s.V1Pod = {
  apiVersion: 'v1',
  kind: 'Pod',
  metadata: {
    name: 'dagger-pod',
    labels: {
      app: 'dagger-app',
    },
  },
  spec: {
    containers: [
      {
        name: 'dagger-container',
        image: 'nginx:latest',
        ports: [{ containerPort: 80 }],
      },
    ],
  },
};

async function main() {
  try {
    // Create a Pod in the default namespace
    const createRes = await k8sApi.createNamespacedPod('default', podManifest);
    console.log('Pod created:', createRes.body);

    // Get the status of the Pod
    const statusRes = await k8sApi.readNamespacedPod('dagger-pod', 'default');
    console.log('Pod status:', statusRes.body);

    // Delete the Pod
    const deleteRes = await k8sApi.deleteNamespacedPod('dagger-pod', 'default', {});
    console.log('Pod deleted:', deleteRes.body);
  } catch (err) {
    console.error('Error:', err);
  }
}

main().catch(console.error);

[tired moth]

can we manage k8 cluster? with dagger & https://github.com/kubernetes-client/javascript

[patent slate]

GitHub - kubernetes-client/javascript: J...

[tribal zealot]

Dagger and open source PaaS

Any benefits to using buildpacks with Dagger?

[sick monolith]

off topic but.... is there any reason why Puppet is still popular for automation? does anyone here work with it and is there a way that dagger could help there? for me I don't see the reason why it's even still around. I see Puppet modules to configure containers on the fly but still Puppet being used for server automation/creation. why? has Puppet ever come up before? the reason I think it's somewhat a thought for discussion is Puppet is about the factory. maybe not the factory shipping software out the door but how the internal factory may create infrastructure... through code. maybe that discussion more aligns with terraform and docker files. I'll end with one last comment. Puppet is a nightmare. especially when its abused and not written for deterministic results.

[slender charm]

off topic but.... is there any reason

[naive leaf]

Truth of any legacy software... if it's working and of any significant LoC no ones going to touch it beyond what's necessary... see also Cobol.

[chilly arch]

Hey everyone! The demos from the last community call are now in the demo forum and on YouTube. Feel free to add comments/questions for the speakers in the forum posts below.

https://discord.com/channels/707636530424053791/1255401990696206357

https://discord.com/channels/707636530424053791/1255405866958721024

https://discord.com/channels/707636530424053791/1255404085729824859

https://discord.com/channels/707636530424053791/1255401088018940009

Also, don't forget to mark your calendar for this week's community call too!

https://dagger-io.zoom.us/webinar/register/8517071155537/WN_j_xRpP1ASb6xrM4AiRKRnA

We have so many terrific speakers joining us this week: Team from Cerbos, @high crystal , @fierce sandal , and @austere maple 🎉

[sand gorge]

Hey! Conversation which started in the #php channel

the exit code in a withExec doesn't currently propagate to the CLI. Actually never thought about that and no one asked for it, but it's something we could do.

So If we building a CI pipeline and the E2E test suite fails, producing an exit code of 1

How can we ever use dagger call if it never returns the code of the withExec() .. moreover what's the best approach here to execute our test suites in CI if dagger call isn't designed to preserve exit codes like docker run does?

linting, static analysis, unit tests, functional tesrts, API tests, E2E tests .. all of these currently get run in CI (using Jenkins / github actions) .. so how can this be done in Dagger? and the pipeline fails and goes red when it should, and green when it should

Thanks 🙂

[rain oriole]

In most cases, just a non-zero exit is sufficient. So you don't see green if pipeline fails.

If you need to handle a specific exit code, that's best handled inside the function. If you need dagger call itself to exit with the same code, it means you have an additional process outside of dagger handling things based on exit code. In any case, it's trivial to add support for it.

[sand gorge]

@rain oriole okay .. we can live with a exit code of 1 .. for every non-zero exit code .. we don't need to preserve the exact we just need to make sure it's 0 or 1 (green or red)

Sorry if that wasn't clear before 🙂 I hope it is now.

[rain oriole]

Just to be absolutely clear, that's how it works today. Any error returns exit 1.

[sand gorge]

@rain oriole yes .. I thought so .. it was your reply that threw me off, but it's my miscommunication and misunderstanding here 😄

At least we've now had the chat and cleared it up.

[rain oriole]

Because it seemed like you guys were expecting the same exit code and were confused that phpunit returned 127 while dagger call returned 1 🙂

[sand gorge]

hehe 😄 PBKAC

[high crystal]

Why is it that on demo day things always break in a random order?

[cunning jolt]

Is it even a valid demo if things didn't break? 🙂

[fierce sandal]

I lean towards making a recording of the demo just in case

[chilly arch]

**🎉 Community call starting in 10 minutes! 🎉 **

Join here - https://dagger-io.zoom.us/webinar/register/8517071155537/WN_j_xRpP1ASb6xrM4AiRKRnA

[winter linden]

Good morning from San Francisco! 🌉

[slender star]

Hello from Portland, Oregon!

[slender charm]

Hello from San Francisco 😄

[magic rivet]

Good morning from Bay Area!

[winter linden]

How I'm picturing my "weekly financial advice" https://tenor.com/view/sell-trading-places-mortimer-randolph-gif-25089911

[elfin frigate]

👋 hi from Toronto!

[cunning jolt]

Hi from Fort Worth, Texas

[inland coral]

Good morning from Menlo Park

[high crystal]

Good evening from the Netherlands \o/

[winter linden]

Cerbos, you had me at "cute mascot"

[slender charm]

love the before/after slider 😍

Daggerize!

[slender star]

Nice use of https://docs.dagger.io/manuals/developer/services/#service-containers

[slender charm]

Amazing Demo, thanks so much Alex!

[winter linden]

Solid! It felt like a no brainer how I would insert that module into my pipeline

[slender star]

Full openstack API...talk about cognitive load!

[winter linden]

https://tenor.com/view/who-me-gif-14906421967915870271

The bigger the stack, the more satisfying it is to fit it in a Dagger pipeline, in my experience

[slender charm]

is it just me or are there black boxes in the zoom window?

Oh looks like its being moved 😄

I love the dagger cloud love today

[winter linden]

It looks like we have to fix your subscription Rob!

@high crystal sorry I missed your description of the blocker - modules not on github, correct?

Great demo!

[high crystal]

i3wm, so every hearth in zoom creates a temporary window 😄

[high crystal]

modules on github indeed! Seems our trial expired today but it still works thankfully for the demo 😅

[slender star]

Yep, Rob is using https://docs.gitea.com/usage/actions/overview

[high crystal]

Who hacks the dagger engine to always reply that @fierce sandal should invest in dagger?

[slender charm]

I love this creative use of dagger + modules!

[winter linden]

I'm amazed by this demo, and feel lucky that Dagger is running it 🙂 It's so original

@fierce sandal You mentioned you use Langchain also, what's the interaction between Dagger and Langchain? Do they work well together? Is there overlap?

[slender star]

@fierce sandal I'm guessing that one reason Dagger modules work for you is that you can run them locally whenever on any machine (without needing to set up special tools) or on a schedule via GitHub Actions. Is that right?

[slender charm]

Amazing @fierce sandal !

[warm temple]

That last slide almost looked like a neural net constructed of AI nodes 🤯 very cool

[fierce sandal]

Lol...I like that

[fierce sandal]

I'm running langchain in a few modules (the 'decision making' modules). Honestly, haven't really encountered any issues, they seem to work well together. I am questioning whether I actually need LangChain. It makes it easier to handle things like building agentic workflows and a memory, but things are constantly changing and they don't always update their docs. From the Dagger end I haven't encountered any major issue

[fierce sandal]

To be honest, being able to schedule via GitHub Actions is a big win for me

[slender charm]

@fierce sandal can you share those links to the blog you mentioned? Im eager to read more 🙂

[winter linden]

Cameron (from community call) what's your discord username?

[hazy flare]

Thanks all - you can find more about Cerbos at https://cerbos.dev/ and https://github.com/cerbos

[austere maple]

@austere maple

[fierce sandal]

@slender charm
Here are links to some of the modules I built:
https://daggerverse.dev/mod/github.com/EmmS21/daggerverse/fetchSpreadsheetData@b690a69b263902262741393552b29042c7bd7087
https://daggerverse.dev/mod/github.com/EmmS21/daggerverse/filterForNewTransactions@38a944be8fb2b71336dee507f6088822ec5f32f5
https://daggerverse.dev/mod/github.com/EmmS21/daggerverse/categorizeExpenses@b88155db6da916656f71b7895238f9862cf7db88
https://daggerverse.dev/mod/github.com/EmmS21/daggerverse/writeToMongo@0db57c7b48700cabb0e6e500f5af897ffd9f03e8
https://daggerverse.dev/mod/github.com/EmmS21/daggerverse/getFromMongo@93c85c21fd39b24a8691ad7a6a47f3c94fdde1d5
https://daggerverse.dev/mod/github.com/EmmS21/daggerverse/getAdvice@50e158a644959653ec0339e66cbdd7268a35f55c
https://daggerverse.dev/mod/github.com/EmmS21/daggerverse/GetStocks@2f015d063a68519c8a968d43785ec248bd14c8b3
Daggerverse Repo:
https://github.com/EmmS21/daggerverse
Blogs:
https://emmanuelsibanda.hashnode.dev/a-dagger-pipeline-sending-weekly-smss-with-financial-advice-generated-by-ai
https://emmanuelsibanda.hashnode.dev/getstocked-an-ai-agent-to-help-you-pay-back-debt-optimally
Presentation Slides:
https://docs.google.com/presentation/d/1bbgj2lKelwWtYYd-cY_OMKpk2CGyGjSOmKnDw3hhLYA/edit?usp=sharing

[winter linden]

https://tenor.com/view/ouch-all-that-nick-nickelodeon-presents-gif-19581576

Thank you Alex, Rob, Emmanuel, Cameron! Great demos

[high crystal]

Thanks all! https://www.safespring.com/blogg/2024/2024-05-openinfra-presentation/ here is some more background information about the self-service-api we are working on in Safespring and a talk+slides from my co-worker Jon about the development of it.

[austere maple]

@high crystal you spoke a little bit towards the end about your desire to migrate eventually to Github Actions for this work with Safespring - how do you think about choices of CI/CD providers in light of your focus on GDPR and EU security standards?

[winter linden]

If anyone wants to try that slick Dagger Cloud UI from the demos - dagger login and follow the instructions to create a free account 🙂

[analog raven]

For anyone that missed it, we'll be having our first Dagger meetup in Bern, Switzerland next week! This will be a hybrid event, so you can join from anywhere. Check out the agenda and register here! 🚀

https://www.meetup.com/dagger_bern/events/301341810/?slug=dagger_bern&eventId=301341810

[frozen basin]

dagger interactive looks lit. drop into shell on error... that's sick

[winter linden]

Thank you for this @rain oriole ! Can't wait to play with it https://github.com/dagger/dagger/issues/6515

[frozen basin]

Circling back on this.... are traces like this only available in functions? Still using standard go sdk

[winter linden]

You can use any standard open telemetry library or tool, as long as it runs in a container managed by Dagger, the standard otel spans will show up in dagger cloud

[slender charm]

No they should work as long as you're using the latest version of the SDK - which version are you on Steven?

[frozen basin]

🤔 hmmm ok. I tried with otel....

Just doing something like this

    ctx, span := otel.Tracer("ourskycustom").Start(k.runContext, "build-trace-start")
    defer span.End()

I tried last night with a tracer name of dagger sdk - can't recall exactly.

happy to DM you guys the dagger cloud trace, but I see nothing

moved into #1030538312508776540 here -> #1256308567586246796 message

[chilly arch]

anyone going to be at KCD Munich next week? @candid crag and Koray Oksay will be presenting about Dagger on Monday. https://www.kcdmunich.de/schedule/

[sudden ocean]

It's a silly question, how to create a new project on Dagger Cloud? I have the existing one created from sign up flow but I want a new one for my public project. T_T

[sharp marsh]

https://dagger.cloud/signup

[sudden ocean]

Oh, I didn't know that I need to signup again! Thanks @sharp marsh.

It would be nice if the Dagger Cloud has a new organization button that can do directly from the main UI.

[sand gorge]

Hey! Quick Question

I'm preparing for the Dagger workshop.

First question (more to some 🙂 )

To allow attendees to prepare .. can they pre-download docker images on their machine, without having Dagger installed?

If we have 100 on the hotel wifi at once .. then all downloading docker images, on the fly, will be no good.

Is it possible to do

docker pull x
docker pull y
docker pull z

and then during the workshop after we install Dagger, then dagger init/dagger call will pick up these pre-downloaded images.

Is this possible? LMK thanks

[narrow nymph]

Hey! Quick Question

[sand gorge]

The dagger cloud "traces" feature is fantastic! 🔥 I'm so impressed

[tired moth]

PLEASE HELP
We have a Deno project deployed on Deno Deploy, using deployctl for deployments. Our current workflow involves running basic commands like:
deno test
deno fmt
deno lint
Given this relatively simple setup, could Dagger provide benefits? If so, could you explain how Dagger could enhance our workflow, considering our specific use case of a Deno project on Deno Deploy?

[swift inlet]

deno project

[tired moth]

Is there any consideration for this feature, if yes then any ETA?
https://github.com/dagger/dagger/issues/6113

[winter linden]

Yes it's in development, weeks away

There's a PR open already I believe.

[tired moth]

Alright great.

[sand gorge]

Why does a dagger functions take like 20-30 seconds sometimes, in the initializing state?

What could be causing the slowness?

[chilly arch]

🗓️ The Dagger Switzerland Meetup starts in 1 hour!

@slender star @heavy gazelle and @split olive have some great demos planned.

This is a hybrid meetup, so there is an in person option to meet up at the Puzzle ITC office or join us virtually at the link on the meetup page.

All presentations will be in English.

https://www.meetup.com/dagger_bern/events/301341810/

See you there!

[narrow nymph]

do you have a lot of things in your current directory?

[sand gorge]

@narrow nymph it's just the "echo" function .. and I'm not mounting any code. Just

dagger call --value="HEYYYY"

[narrow nymph]

hm, do you have a trace?

[sand gorge]

Dagger cloud traces? How do I get a trace

[narrow nymph]

If you do dagger login it should just appear in your traces section in the UI.

[sand gorge]

I'm auth'd on Dagger Login now .. so i'll keep an eye on it, and when it becomes slow again, i'll trace that, thanks @narrow nymph

[rain oriole]

Even with the default template you could be uploading too much, for example composers vendor dir, if not excluded in dagger.json. The initialization step is for loading the module. Two things can happen. Uploading too much or inefficiencies in the runtime module.

[sand gorge]

Ooooh. So dagger.json can exclude dir, just like a dockerignore, for the build context..right?

[mossy edge]

Hey <@&946480760016207902> I would like to work on this issue: https://github.com/dagger/dagger/issues/6605
Is this still relevent?

[sand gorge]

Yes we have a large vendor DIR

[narrow nymph]

yeah definitely still relevant, i'm not sure if we might want a different dx around it? pinging @pseudo stream here

[rain oriole]

dagger.json exclude

[inland coral]

Hi to everyone in this awesome community. I have a favor to ask all of you. I run Marketing for Dagger and am looking for a contract Technical Marketing Engineer / asset focused Devrel person to help me take our website and other technical assets to the next level. Could I ask you to like/repost this post to your networks? https://www.linkedin.com/feed/update/urn:li:activity:7214305073785360384/

The goal is to continue to develop and publish material that helps people learn what Dagger is all about and motivate people to Daggerize their first pipelines, in a sense continue to build out the bridge between landing on our homepage and going through the Quickstart in the docs.

Of course if you have thoughts, ideas and advice for how we can improve this early part of the user journey I'm always interested in hearing them, so feel free to reach out to me. 🙏 🙏 🙏

[sand gorge]

I'm back! (discord on my laptop wasn't working - rebooted and it's ok).

I'm building out the Dagger workshop for tomorrow, so I'll likely need some quick questions answered, to unblock me, if I encounter any blockers 🙂 thanks in advance.

[sharp marsh]

sure Paul. feel free to drop the questions here and tag some of us in case you need some promptly answers 🙏

[sand gorge]

legend, ty @sharp marsh

Once I'm done .. all my work is going to be publishable as online learning materials, and I can reach out to CFP's on many other conferences, to give more workshops and talks .. so I think it's going to be a great year for learning Dagger 😎

[sand gorge]

Anyone got a good example of taking .env and looping over the entries to do dagger's ->withEnvironmentVariable() call? please.

I'm trying to replace docker-compose's feature of env_file or even this

docker-compose --env-file ./.env

    environment:
      APP_NAME: ${APP_NAME}
      APP_ENV: ${APP_ENV}
      APP_DEBUG: ${APP_DEBUG}
      APP_KEY: ${APP_KEY}
      APP_VERSION: ${APP_VERSION}
      APP_URL: ${APP_URL}

[sand gorge]

@sharp marsh (or others) why does daggerverse have a postgres module, but not a MySQL module 🤔

https://daggerverse.dev/search?q=mysql

UPDATE: I found MariaDB

[frozen basin]

Why does a dagger functions take like

[sand gorge]

Now we're cookin'! 🧑‍🍳

good night 🙂

[meager sentinel]

Hi guys, I'm really interested in using Dagger in production.
Now my question is, how breaking is it?
Can it be used in prod, or should I wait for a 1.0 release?
How do you guys handle dagger upgrades?

[sharp marsh]

Hi guys, I'm really interested in using

[elfin frigate]

👋 Hey all, we're trying to figure out what the default verbosity level for the CLI should be. In current shipped versions of Dagger, it defaults to "quiet" - meaning completed work goes away, and the final print is empty, with a -v flag to keep completed progress. The new behavior on main is to keep completed progress by default instead, with a -q flag to revert to the old behavior (also configurable via env).

There's a good chance your preference might change over time, so I'm looking into having a convenience like dagger -q or dagger -v (no other flags/args) to persist a setting locally, but we'd still want to decide on a default.

Please vote here: https://github.com/dagger/dagger/issues/7801#issuecomment-2207798136

Also worth noting no matter what the default is you'll be able to change it at runtime if you forgot the flag: https://github.com/dagger/dagger/issues/7801#issuecomment-2208151443

[naive leaf]

Wondering if there's a module out there already or some guidance on how to proceed. I want to stream daggers execution to a GitHub action console.

I have a CodeBuild test execution that does the following after the Dagger install via curl+shell pipe used from the example buildspec:

~/.local/bin/dagger develop
~/.local/bin/dagger call test --source=.

Is there some way I could stream the output from a dagger call to a github Action console? Some of the older examples I imagine I could implement a Tee to split to os.Stdout to maintain a stream in CodeBuild and another io.Writer that streams the bytes to Github actions console.

I'm guessing I'd need to look to tie into something other than .Stdout() in my Test function?

func (m *K8Sensor) Test(ctx context.Context, source *Directory) (string, error) {
    return m.BuildEnv(source).
        WithExec([]string{"sh", "-c", "go test -v -cover -covermode=atomic ./..."}).
        Stdout(ctx)
}

[tired moth]

is it fine if we run a dagger in dind container? if we want to run dagger in github action through container?

[cunning jolt]

That's how we run it in Jenkins.

[tired moth]

thanks for validation.

[chilly arch]

**Attention London Daggernauts! **

A few folks from the Dagger team will be in London on July 16th, so they're meeting up for a low-key hangout. If you are in the area, we'd love to have you join us! You can find the details at the link below:

https://discord.gg/dagger-io?event=1259937162460135474

Also, @heavy karma (awesome guy who writes all of our docs) is in London until July 20th, so if July 16th doesn't work for you and/or you want to chat about Dagger over coffee, DM me. I'd be happy to coordinate a meet and greet!

[chilly arch]

**Community Call Demos added to the demo forum. Check them out! **

https://discord.com/channels/707636530424053791/1260111280430252133

https://discord.com/channels/707636530424053791/1260111115623596053

https://discord.com/channels/707636530424053791/1260111044441804840

https://discord.com/channels/707636530424053791/1260110943057219636

[chilly arch]

Tomorrow is going to be fun. The Dagger team will take over the Dagger Community call at 9 am PT to show y'all the latest changes that are coming with the 0.12 Dagger Release!

Add it to your calendar: https://dagger-io.zoom.us/webinar/register/8517071155537/WN_j_xRpP1ASb6xrM4AiRKRnA

See you there!

[chilly arch]

https://tenor.com/view/michael-jackson-eating-popcorn-enjoy-i-like-nom-nom-gif-11040065238845078056

Community call starting now. See you there!

[elfin frigate]

[glad jolt]

hello from Argentina! 🧉🇦🇷

[cunning jolt]

hello from Fort Worth, Texas

[elfin frigate]

👋 Toronto checking in 🇨🇦

[inland coral]

Hello everyone from Menlo Park, California

[rain oriole]

👋 Hi from the Azores! 🇵🇹

[lapis slate]

Hello from sunny Algarve in Portugal

[sharp marsh]

👋 hello from Uruguay 🇺🇾

[jolly oriole]

Hello, from U.K. in San Diego o/

[chilly arch]

Hosting from Vancouver, WA 👋

[sharp marsh]

https://tenor.com/oIjBKPrrdaG.gif

[narrow nymph]

👋 heya y'all

[tired moth]

Dagger team, YOU ALL ROCKS...GO GO GO

[jolly needle]

Tada

[winter linden]

hello from Saint-Malo on the north coast of Britanny, France!🇫🇷

[slender star]

Now a lot more people know about Vancouver, WA 🤸‍♀️
Jordan Chiles
I'm across the river in Portland, Oregon 😉

[queen bramble]

Greetings from Germany! 🇩🇪 Super excited for today!

[winter linden]

🤯

[fossil pine]

hey-hey! Can't be there today, but I'll catch the recording. Can't wait to get my hands on 0.12.

[rain oriole]

Been using this a lot lately, real productivity boost! 🚀

[cunning jolt]

Can't wait!

[jolly oriole]

Debug Dagger in the Runner image. 🙂

[elfin frigate]

https://tenor.com/ZM7W.gif

[slender star]

[elfin frigate]

so nice we demoed it twice

[cunning jolt]

Deja Vu... didn't this happen last time too?

[elfin frigate]

also my second time FOLLOWINg that demo 😭

[wraith niche]

the big difference is last time was an early POC that may or may not ship. This time it's stable and shipping in .12 🙂

[cloud canyon]

Yep -- last demo was the POC, this is a shorter demo of the shipped feature

[rancid spire]

--interactive would have saved me a good hour today I think 🙂

[cunning jolt]

I meant going first with @cloud canyon 's demo :D...

[rain oriole]

-i is yet another productivity boost, makes it easy to not have to go edit code.

[winter linden]

interactive tui!

[elfin frigate]

https://github.com/dagger/dagger/issues/7801#issuecomment-2207798136

[cunning jolt]

Request: support hjkl for us vim users

[winter linden]

lol

[narrow nymph]

ahaha i asked for this yesterday too and got the same response 😆

[rain oriole]

+1 for gg and F as well (from tail & docker logs).

[narrow nymph]

F?

[rain oriole]

F == follow

[narrow nymph]

👀 👀

[winter linden]

wat

[rain oriole]

The jump to web when zoomed on a span is so great, I've used it a few times.

[narrow nymph]

"doesn't select line numbers" ❤️ ❤️ ❤️ ❤️

[rain oriole]

Oh, opening » in new tabs is new to me 👍

[winter linden]

Like you were saying @chilly arch : it's the little things 🙂

[glad jolt]

the recipe feat is

[rain oriole]

Haven't tried the hover on ID icon yet!

[winter linden]

Note that this particular trace shows our integration tests, which involves dagger building and running dagger to test dagger 😛

[cloud canyon]

That last quick demo "oh it's showing go test under withExec rather than sync" looks like a simple feature but it's actually massively hard to do (attribute timing to async operations)

[narrow nymph]

it's great fun, we run dagger-in-dagger-in-dagger

[rain oriole]

Yeah, very neat!!

Oh noes!!

[narrow nymph]

"if you see something that doesn't look right, speak to @elfin frigate - see it, say it, sorted" 🚆

[cloud canyon]

"win by a thousand paper stitches"

[chilly arch]

Seriously. And shout out to @fossil pine (I know he can't attend today). His feedback as a Daggernaut on the Cloud UI has been amazing.

[elfin frigate]

the new PR nav scheme / UI is - way easier to skim and find/follow my PR

[narrow nymph]

i just like the little jenkins logo here lol 😄

[slender charm]

its very cute

[sharp marsh]

https://tenor.com/stIeZiLf1Wf.gif

[elfin frigate]

do we show the flaming demon version if the group failed? 😄

[jolly needle]

Ask me for the Jenkins Dagger Logo

[swift inlet]

Totally agree 💯 Feels natural when done right, and impacts so much DX

[glad jolt]

lots of them here
https://www.jenkins.io/artwork/

[lapis slate]

So... when is dagger v0.12 being released?

[narrow nymph]

sooon!

(the idea is to do this later today)

[queen bramble]

And auth?

[jolly needle]

This one will need some tuning

[tired moth]

Is there any consideration to make dagger work native for distributed computing? which can drop the requirement of k8 cluster for autos calling?

[chilly arch]

If you want to follow every moment, you can do so here. I'll notify everyone as soon as it is out though!

https://discord.com/channels/707636530424053791/1254757484057464975

[winter linden]

Any public git server for now... Private modules are coming very very soon, but PR is stil in review. Should merge soon

[slender star]

dagger call -m also works, of course 💪

[elfin frigate]

so nice to have escaped the github.com requirement 🎉

[swift inlet]

And for @cunning jolt we also have in mind the auth via PAT in mind 👼

[cunning jolt]

YAY!!!! ❤️

[slender star]

https://tenor.com/view/memo-boss-office-gif-7299171

[sharp marsh]

we also support https://codeberg.org/ 🙈. Kudos to @swift inlet for adding that one

[cunning jolt]

I use the current corporate network support today and it works pretty well!

[queen bramble]

How do modules appear in daggerverse and how would this deal with a git repo that is only accessible in a private network - but reachable from the dagger engine?

[sharp marsh]

there's also experimental support for adding special vars like GOPROXY for your builds.

[slender charm]

@queen bramble great question - they wont show up in daggeverse, but you will still be able to call them because rememebr that its just a pointer to a repo, so as long as you have access to the repo (via SSH https://github.com/dagger/dagger/pull/7708) it will just work

[cunning jolt]

Can the containers opt out of the proxy?

[tired moth]

Can anyone please answer this?

[slender charm]

In the future we will also likely have a concept of a "private daggerverse" - but that is still a WIP/Dream

[elfin frigate]

what @winter linden says about corp network support complexity goes for supporting other VCS providers too - there's a ton of emergent complexity there 😭 - to do with silly details like identifying where the repo URL ends and a subpath begins (shoutout @swift inlet for getting it done!)

[cunning jolt]

Yeah some containers exclusively work internally so no need of the proxy and sometimes that might hinder

[queen bramble]

Thanks for clarifying. It's crucial that this doesn't leak.

[swift inlet]

• thanks to @narrow nymph for all the great comments / reviews and proposals on the best way to navigate this 👼

[tired moth]

PLEASE

[pseudo stream]

It's something we want to do someday, but there's a lot of other work that has to come first that we are doing now. E.g. we're working on making the cache storage mechanisms more customizable/pluggable and after that will be doing the same for deployments+connections to the engine. Those are pretty big efforts but once done everything will be in place to work on distributed builds, which will be another large effort 🙂

[inland coral]

Imminently. Solomon mentioned at the top of the call he's hoping for later today but could be tomorrow. Just tagging the last remaining small issues.

[cunning jolt]

Enums in Go too? Curious how that works

[rain oriole]

Yep. It's just like the core ones. You define them like that.

[tired moth]

RUST Official SDK support with every releases, when we can expect this?

[tired moth]

thanks you so much for answer and consideration? is there any issue where we can track this?

[pseudo stream]

I don't think there is for distributed builds right now since it's a ways out, just for some of the pre-req issues (i.e. https://github.com/dagger/dagger/issues/5583).

[narrow nymph]

when we do it, you'll hear about it 😄 we'll be quite loud about it i think 🎉

[swift inlet]

Erik ftw, solving huge issues in the back 🦸‍♂️

[slender star]

Rust is not an official SDK yet, but the community is working on it and it "could be" official in the future. Any help on issues would be appreciated.

[rain oriole]

I need to know how to profile Dagger 🙂

[tired moth]

thanks a lot

[rain oriole]

Reduced impact for modules only, because of compat mode. Still most of those are easy changes.

[cunning jolt]

Removal of core aliases from Go I think is one that may affect a lot of modules

[tired moth]

So with a 0.12 release we can have a live watch for files such as dev containers?

[narrow nymph]

no, there's still no live watch support

[tired moth]

any issue to track this feature?

[elfin frigate]

oops

❯ sudo rm -rf ~/.vscode-server/ ~/go ~/ ~/.cabal ~/.elm ~/.emacs.d/
direnv: error LoadConfig() Getwd failed: "getwd: no such file or directory"
vito@DESKTOP-59TV2AN ~/s/dagger>

[rancid spire]

🔍 bring your own storage for distributed caching?

[elfin frigate]

it's there but in black on black 😂

[rancid spire]

I am great at squinting 😉

[pseudo stream]

https://github.com/dagger/dagger/issues/6990

[rain oriole]

You get most things automatically just through codegen. So SDK maintainers don't have to keep up with new features as much.

[cloud canyon]

https://tenor.com/view/enhance-gif-10953787

[winter linden]

Thanks for joining everyone!

I kind of felt like a TV news anchor 😂

[tired moth]

thank you

[slender star]

https://tenor.com/view/do-burgundy-gif-26680869

[winter linden]

"interactive debugging: is it the bee's knees? Andrea Luzzard is live at the scene with more details. Andrea?"

ChatGPT to the rescue:

1.    “Fasten your seatbelts, developers! A new interactive debugging method for CI/CD pipelines with Dagger is making waves. We’re linking up with Andrea Luzzardi, who’s got the lowdown on this game-changer. Andrea, what’s the scoop?”
2.    “Hold onto your monitors, folks! Live visualization of Dagger pipelines in a trace view is here to revolutionize your workflow. We’re going live to Alex Suraci and Julian Cruciani, who are at the forefront of this tech marvel. Alex, Julian, show us what you’ve got!”
3.    “Get ready to push your code to the edge! Hosting Dagger modules on any git server is now a reality. We’re tuning in with Lev Lazinsky and Guillaume de Rouville, who have all the details on this groundbreaking development. Lev, Guillaume, how’s it looking?”
4.    “Buckle up, tech enthusiasts! Major reliability and performance improvements in the Dagger Engine are supercharging production environments. Gerhard Lazu is on the scene to break it all down for us. Gerhard, what’s the latest?”

[slender star]

used text above for prompt

[winter linden]

huh, text rendering is getting pretty good

[slender star]

RIght!? I love how people's names got blended.

[glad jolt]

https://tenor.com/view/thats-it-yes-thats-it-that-right-there-omg-that-thats-what-i-mean-gif-17579879

[slender star]

Julián, for sure, your name is dominant 🙂

[winter linden]

For those of you who have successfully adopted Dagger in your project: we're working on a new documentation page called "Adopting Dagger", with guidelines for successful adoption. I would love your review! https://github.com/dagger/dagger/pull/7893

To run a local instance:

dagger call -m github.com/dagger/dagger/dev \
 --source https://github.com/dagger/dagger#pull/7893/head \
 docs \
 server \
 as-service \
 up

Then navigate to http://localhost:8000/adopting

Thank you!

[loud briar]

Docs: Adopting Dagger by shykes · Pull R...

[jolly oriole]

Great documentation -- I have no other comments. Only the very subjective comment that the documentation felt like adoption starts with an application developer. Adopting in an enterprise can also start with centralized DevOps looking to bring pipeline logic together as an internal product/service across multiple applications.

[winter linden]

Thanks! Yes that is indeed a common adoption pattern. I tried to write it so that it applies to both, since the fundamentals are the same. Are there specific parts that stood out as not fitting in a central enterprise devops adoption ?

[jolly oriole]

On re-reading the docs. I think I just brought my own baggage to my first comment

[fossil pine]

Yay, 0.12 tag appeared

[winter linden]

https://tenor.com/view/hold-stonk-hold-wallace-hold-braveheart-stonks-gif-20142609

[fossil pine]

Blog post 404: https://dagger.io/blog/dagger-0-12 😛

[jolly oriole]

https://tenor.com/view/waiting-intensifies-zootopia-intensifies-waiting-impatient-gif-9739240072909568285

[fossil pine]

I'm just going to update everything and push to main without PRs. What can go wrong? 😄

[slender star]

https://tenor.com/view/lotr-lord-of-the-rings-gif-26034393

[winter linden]

https://tenor.com/view/not-yet-mr-miyagi-pat-morita-the-karate-kid-ii-not-so-far-gif-24816259

https://tenor.com/view/patience-family-feud-canada-tolerance-self-control-cbc-gif-23617733

[winter linden]

https://tenor.com/view/rocket-lift-off-gif-10300477

https://tenor.com/view/we-have-lift-off-cole-davis-fly-me-to-the-moon-here-we-go-lift-off-confirmed-gif-10284188292436584657

[misty coral]

There is a code typo here

It misses the closing paren

[slender charm]

@chilly arch ^ are you able to fix that

Thanks so much @misty coral !

[misty coral]

@slender charm No problem! I was looking for an edit button, but couldn't find it. :)

[slender charm]

Hm, could you actually point me to the section where you saw this? I don't see it on my end unless someone already fi xed it super quick 🙂

[misty coral]

It was under Harmonize Container.withNewFile and Directory.withNewFile

But I don't see it now either...

Interesting. But as you can see, it wasn't here. I've taken a screenshot.

[slender charm]

Thank you either way, I appreciate your attention to detail!

[misty coral]

It is really weird. It disappears for a while when the page is loaded, then appears back. Maybe related to JS and at my specific time, it didn't load (browsers, eh?).

[misty coral]

Oh, thanks :)

The changes are really cool! I can't wait to have them in Rust :)

[slender charm]

Me too! 🤞

[chilly arch]

If you haven't already heard 😉

🚀 Dagger Engine v0.12 is here! 🚀

After months of shipping bold features like Functions, Daggerverse, and Traces, we’ve focused this release on usability, polish, and quality. Here’s what’s new:

✨ Interactive Debugger: Debug your pipelines interactively without setting breakpoints.
🌐 UI Enhancements: Faster, more interactive terminal and web UI.
🔒 Corporate Network Support: Improved support for proxies and custom CAs.
🚀 Performance Boosts: Faster, more reliable pipelines.
🔄 Compatibility Mode: Easier upgrades with backwards compatibility.
🌍 Modules on Any Git Server: Host modules on any Git server.

Check out the blog post for more details: https://dagger.io/blog/dagger-0-12

You can watch all of the feature demos here: https://youtu.be/gUw-9kuvwDM

[winter linden]

Hello Gophers, a breaking change in the Go SDK 0.12: the dagger. prefix for Dagger types is now mandatory.

Before: func build(src *Directory) *Container

After: func build(src *dagger.Directory) *dagger.Container

This change is backwards compatible (0.11 supports it) so I recommend making the change to your modules as soon as possible.

Thanks to compatibility mode you can still upgrade to 0.12 right away, but won't get the other 0.12 features while compat mode is active.

This is a bandaid we've meant to rip off for a while, it will make the Go SDK more consistent with the other SDKs, and will improve IDE support.

[misty coral]

I'm something of Gopher myself, so I wanna ask if there's any automatic migration strategy. ^^

[winter linden]

That would be fantastic, but at the moment, sadly no. We accept pull requests though 😛

[raven stag]

Random question: If you had 5 minutes to create hype about dagger to an audience of 100's of engineers...
What use case would you show? Showing a pipeline running locally/remote is nice, but I'll cover that in a proper (longer) session.
I'm looking for something that triggers a "Wow, what is that? I want to know more... " type of reaction

[hollow arrow]

I think more details about the audience is important to decide.
Details like which company do they work for, which ci/cd system (GitLab, Jenkins, etc) do they use and how they use this ci/cd system.

If they are general audience that you don't know much about, then I think showing them performance differences between a traditional GitLab pipeline (or whatever you prefer) and a daggarized one will be a nice option.

It is the first advantage shown in Dagger engine's home page so I think it is the most notable one.

[chilly arch]

The overview deck that we built has helped a lot of people share the message - https://docs.google.com/presentation/d/1VUPTvajY-RXlByUA9hyybDve5slVvqGCGDBbwpfciM4/edit?usp=sharing

I think slide 2-3 can be told within 5 min.

Additional resources -
Airbyte case study - https://marketing.dagger.io/Airbyte Case Study.pdf
OpenMeter Case study - https://dagger.io/blog/daggerizing-production-pipelines-how-openmeter-overcame-push-and-pray

We just uploaded a shorter demo that talks about using Dagger beyond CI/CD that I think might be helpful too https://youtu.be/tXoIpioUCsA

[slender charm]

I think interactive debugging is the thing that always gets the best reaction in my experience.

Specifically when its with a complex pipeline and the audience knows how painful the iteration loop is going to be if they have to debug a pipeline using the existing flow which is normally

1. make some change
2. `git commit -m 'fix ci'
3. wait 15 minutes and hope

[chilly arch]

@raven stag if you want an idea of how to demo interactive debugging then I'd watch @cloud canyon 's demo here at 1:13 https://www.youtube.com/watch?v=gUw-9kuvwDM

[stoic knot]

OMG, you can change verbosity in the middle of a build with the +/- keys in v0.12?! SUPER COOL

[winter linden]

you can also select and zoom into a span with direction keys + enter

then if you press w you will get a web view of the current selected span

[stoic knot]

Is dagger.cloud down or slow?

[slender star]

Is dagger.cloud down or slow?

[stoic knot]

One thing that is confusing in the new TUI is that it looks like certain steps are done (with the ✅ ) even though they have not run yet, and also with the last step being yellow and looking like it's what is currently running. Also, the tasks like yarn build, which have not run yet, have times next to them

note, the yellow withExec was not showing when I started the screenshot

[winter linden]

Also I would love a hotkey to hide/show completed spans (hard to see what's in progress in very long and complex pipelines)

[stoic knot]

Is Dagger SOC2 now? Last I heard, things were close

[narrow nymph]

Yup 🎉 https://dagger.io/blog/soc2-audit

[elfin frigate]

isn't that - (0 = hide completed) and + (1 = show completed, default)?

[winter linden]

Indeed it is! Fantastic. I thought reducing verbosity just changed the depth

[narrow nymph]

🎉 I added for gg/G at least in https://github.com/dagger/dagger/pull/7958

[cunning jolt]

Maybe also a key to open the current log in the EDITOR?

[sharp marsh]

moar verbosity. cc @elfin frigate 🤣

[slender charm]

Are you just curious, or are you thinking about exploring for your clients?

[stoic knot]

They are interested and needed SOC2, but the issues we have with long upload times is having the tech lead rather frustrated

[sharp marsh]

https://x.com/tkdodo/status/1813578513196449899

^ good idea for a Dagger module. Any takers?

Dominik 🔮 (@TkDodo) on X

Is there a GitHub Action that will just run prettier on the whole repo and commit the changes to the PR?
I'm so tired of the feedback loop where prettier fails on PRs. We even have a pre-commit hook to validate but it's not helping. It's insane how much time is wasted here ...

Twitter · Jul 17, 2024, 2:16 PM

[stoic knot]

This step is killing me. The idea here is:

1. We are doing a local build, so need uncommitted changes, so cannot clone
2. Certain steps require the git directory to be around, but the check/upload time is 50% of a rebuild
   note, this is a rebuild, in that I ran Dagger, and then ran it again immediately after it completed, so no changes

Is there anything we can do to reduce this Host.directory() time?

✔ Host.directory(exclude: ["lfs/"], path: ".git"): Directory! 23.4s
  ✔ cache request: upload .git from dev-tony-r8stg (client id: mntfx0zm3xg0fdcupbeo47afh) (exclude: lfs/) 0.0s
  ✔ upload .git from dev-tony-r8stg (client id: mntfx0zm3xg0fdcupbeo47afh) (exclude: lfs/) 8.7s
  ✔ copy upload .git from dev-tony-r8stg (client id: mntfx0zm3xg0fdcupbeo47afh) (exclude: lfs/) 2.6s
  ✔ blob(digest: "sha256:77de4573cd44bc61141656a6c6ae234aa49717d6fd6c434b8cc8ef4413bc3233", mediaType: "application/vnd.oci.image.layer.v1.tar+zstd", size: 612116258, uncompressed: "sha256:77a2823ce80bf31429c42f100a82333013ca1c4b4e1560d0a2f43e98944d5003"): Directory! 0.0s

[slender charm]

This step is killing me. The idea here

[frozen basin]

terminal is so lit.

just realized I said this like less than 2 weeks ago. but point stands

[winter linden]

It's worth repeating 🙂 Thank you @cloud canyon

[slender star]

Interactive mode was super fun while upgrading one of my Golang modules 🙂
https://youtu.be/oS2NgB1irq4

[misty coral]

We had a fantastic meetup in Istanbul. It was very informative for all of us. I 'd like to offer my gratitudes to the organizers, especially to @candid crag for withstanding against technical difficulties. ♥

[candid crag]

thanks for attending brother! I'm so glad that you liked it ❤️

[misty coral]

Is it possible if we could get some of the pictures we've taken shared ^^ @candid crag

[candid crag]

I'll send it to you for sure xoxo

[raven stag]

👋 Random question.
Is there a way to use the Container().Export() and have some tags defined for the tarball exported?
Seems like the manifest.json has an empty list of RepoTags and I can't find a way to modify them (neither at building the container or through the export functionality)

[sand gorge]

@raven stag interesting! is .Export() kind of like docker save command?

[raven stag]

"kind of", yes. But when you build a container in dagger, you don't pass any tags. Tags are only relevant when you are publishing, so exporting a container (the equivalent to docker save) will get you a local tarball that when imported, will have a non UX friendly name for your local image. e.g:

/tmp/tarball.tar.gz
$> docker import /tmp/tarball.tar.gz
sha256:237c4ebacbd1fcf5e2b3ecc44c83a956646cd9d0676b3cadf821f654e4652046
$> docker iamges
REPOSITORY                  TAG       IMAGE ID       CREATED         SIZE
<none>                      <none>    237c4ebacbd1   7 seconds ago   14.3MB <----- This is the tarball imported. 

[misty coral]

From what I understand, you're not supposed to use the export functionality like that. It is just how docker save/load works.

[raven stag]

We are building dagger functionality for many engineers to consume. One of the expectations (today's current workflow) is that they're able to build and have their resulting docker images locally. I wish I could tell them not to continue doing so, and just rely on doing something more dagger-like. Jumping into the container terminal once built, or run the image as a service if they need it runnning. But I don't think that will cut it for now, so I was looking for a way to provide them with something similar to what they have now

I may just skip this and redirect them to $> dagger call function-that-returns-container terminal or implement functionality around the services

[misty coral]

That's what your suppose to do anyway I'd say. There's no point in exporting the image. What are you gonna use it for? Testing? You can automate that with dagger. Everything that requires you to export the image is probably a design issue, at least it was always for me.

I might be wrong, of course. Someone might make use of it in a workflow. I think it could be used for distribution purposes or manual reviewing (as in debugging).

[raven stag]

Another quick question 🙂 . Is there a simple way to see from the TUI if an operation was cached?

This would be amazing on the TUI output by default 🙂

[tough lynx]

Hello, I face the gollowing error in my company x509 certificate error :

[tough lynx]

What could I do ?

[sand gorge]

@tough lynx I'm not saying this will help you, but if you update your DNS Nameserver to 8.8.8.8 it might help. I had some funky issues with similar (but not exact) problems, and this fixed it for me

On linux it means editing your /etc/resolv.conf file to use 8.8.8.8

[sharp marsh]

not yet. It's in our backlog and something that we'd like to address very soon 🙏

[sharp marsh]

here are the instructions to setup custom CA certs for environments that require it: https://docs.dagger.io/manuals/administrator/custom-ca/.

A lot of companies usually require this for security purposes

[slender charm]

👋 Random question.

[celest grove]

This was very helpful ❤️ , thanks for sharing it @slender star .

[tough lynx]

Hello, I will try this but I am not able to understand why it would work. The dagger client fail to verify external container registry server certificates (ghcr.io github container repo in this case).
Also docker pull on this layer is working

[tough lynx]

Using wsl with Ubuntu, I also try to update ca certificates without success

[kindred dove]

We had a similar issue where it's our firewall inserting it's certificates into the cert chain. We can have IT to add an exception for the affected site (in your case ghcr.io) to disable HTTPS introspection on this domain. Not sure if that's a possibility for you?

[chilly arch]

Thanks @slender star for sharing how to upgrade modules to 0.12! If anyone is having issues upgrading your Module, let us know in the help forum.

https://www.youtube.com/watch?v=oS2NgB1irq4

[chilly arch]

Looking forward to seeing everyone at the community call tomorrow!

@slender charm and @sharp marsh will demo some key use cases that we get asked about.

<@&1151022465561149450> and <@&1122942621724184576> folks, you won't want to miss this one. Join us live to ask questions or share pointers that we might miss!

Add it to your calendar -
https://dagger-io.zoom.us/webinar/register/8517071155537/WN_j_xRpP1ASb6xrM4AiRKRnA

[chilly arch]

Community Call starting in 5 minutes. See you there!

If you are in the waiting room, we'll see you there in a moment!

https://tenor.com/view/problemastecnicos-los-simpsons-technicaldifficulties-thesimpsons-gif-18336698

[inland coral]

https://tenor.com/view/puppy-cute-adorable-husky-dog-gif-1379742650646357157

Hi everyone from Menlo Park, California

[reef shale]

Helllo from San Diego, CA USA

[analog raven]

Good morning from Vancouver, WA, USA!

[chilly arch]

Everyone should be able to see something like this. Let me know if you can't!

[cunning jolt]

Hello from Fort Worth, Texas!

Is there a direct link to share the webinar with someone else? I want to invite a co-worker

they dont' have access to discord

[chilly arch]

If they go through this, they can register real quick and it will take them right there - https://dagger-io.zoom.us/webinar/register/8517071155537/WN_j_xRpP1ASb6xrM4AiRKRnA

@sand gorge suggested a quicker way last night, so I will work on that for future webinars to bypass this registration page

[lapis slate]

Hello from Porto, Portugal

[sand gorge]

Joining now.

[inland coral]

@sharp marsh Cube See-Tee-El? You had a choice of KubeKettle, KubeCuttle, or KubeCuddle and you picked that one.... 😂

[real kite]

I also say kube see-tee-el 😄

[rigid pebble]

Same!

The nice thing about this is that if you develop your pipelines using Go, you can import client-go and perform your kubernetes integration tests using the API directly, without any commands in the middle

[sick monolith]

I think it could be a good way to test how to 'deploy' helm charts to a cluster and validate it works in some kind of 'test' environment which you can configure with a helm value.

[storm plaza]

how about cubecontrol?

pronunciation-wise

[inland coral]

Yeah that's a valid option too, if a little less fun...

[rigid pebble]

Well, control is commonly shortened to ctrl, not ctl... 👀 . let the wars begin hah

[real kite]

maté driven development

[sick monolith]

if dagger pipeline = 'local env' -, spin up a k3, deploy and validate
if dagger pipeline = 'integration env', hit a real k8s cluster, deploy the chart

end result - a dagger pipeline which can be developed to simulate deployment strategies without needing a real k8s cluster

[storm plaza]

I say cube-cee-tee-el, so shrug

[rigid pebble]

The best kind of development

[queen bramble]

Team cuddle here 🤗

[storm plaza]

that's very cute 🙂

[real kite]

Follow up question:

Is the main interface with the cluster intended to just be exposing the api endpoint? I am guessing kubectl port-forward will be necessary to expose any UI's, for example, but that should be handled via ingress on the API endpoint

[rigid pebble]

Lets go!!!

[inland coral]

https://tenor.com/view/hulk-hogan-flex-flexes-flexing-wwe-gif-13189000

[queen bramble]

Serious question: you started a k3s instance and referenced it with a completely separate dagger call invocation. How is this done from a networking perspective? You passed that name argument, so the first one didn't go over the host machine, right?

[rigid pebble]

Yeah, port-forwarding could be done by combining a WithExec of kubectl port-forward + a dagger Service that can you then tunnel to

[cunning jolt]

Next level to this is I guess emulating CSP k8s offerings like EKS, AKS, GKE etc. There are nuances to those that can't be just replicated with a vanilla k8s cluster.

[real kite]

how to stage locally constructed containers into k3s? I see here that we are only pulling from remote registry? Or is that not necessary with this approach? Is local registry necessary?

[queen bramble]

Thanks!

[thick igloo]

I confirm, it is working with private registry and it is amazing I think

[tired moth]

you guys just added support to orchestrate k8 like docker from dagger?

[rigid pebble]

Pretty much anything that can be containerized can be run within dagger

[tired moth]

k8 clusters also possible now? or by running k8 cluster in container?

[rigid pebble]

You can think of it as kind but within dagger instead

[inland coral]

Couldn't you try LocalStack? I remember folks saying it's a lot lighterweight than EKS anywhere.

[cunning jolt]

There is a jenkins plugin that will colorize those ansi codes in the console log. It looks nice.

[sick monolith]

everyone here knows the true next step is deleting jenkins once everything is daggerised 😏

[tired moth]

ok can you please provide code example link or something?

[real kite]

Emulating a cloud provider stack sounds error prone and a constantly moving window, trying to stay abreast of whatever some google scale cloud provider is doing sounds.... difficult

[rigid pebble]

@sharp marsh can you share the link to your module and the sample you just demoed?

[sharp marsh]

yes, sure. The module is here https://daggerverse.dev/mod/github.com/marcosnils/daggerverse/k3s@5e572e251b4a7f1383ca8043da88b724f8f30e64. The recording will be also available afterwards @tired moth !

[cunning jolt]

I don't disagree, but companies also use those CSP native plugins in kube. Which is hard to replicate in vanilla kube

[real kite]

yeah, probably those api's are stable enough that it may not be too much work to achieve

[cunning jolt]

didn't think of EKS anywhere.. that's a potential solution

This second way (from jenkins demo) is how I run jenkins pipelines with modules as I dont' have access to the cloud for traces. Jenkins shows and handles the stages.

[tired moth]

alright thanks a lot

[reef shale]

Combining Dagger with Jenkins should eliminate the need for Groovy Jenkins code

[cunning jolt]

And it does.. just need a little DSL for structure

[sick monolith]

regarding the 'structure' piece demonstrated by lev, would it also be valid that each 'stage' exports data from the dagger call and put 'files' back into the host jenkins machine if required (i.e artifacts or test covg repors? so its still broken down, but you may stil depend on other jenkins features

[real kite]

Last Kube question, more for local iteration. If I spawn this kube process in a dagger session, does my local container that I'm staging in need to be in the same session? Consider a tilt-like workflow where I have DaggerSession1 with all the kube stuff and DaggerSession2 is MyApp, and DaggerSession2 is the one that is being constantly iterated on

[cunning jolt]

I dont' see why not.. you can do it in steps until you move to a full dagger module combining all other modules.

[queen bramble]

I see the opportunity for a module generating these CI-tool specific pipeline definitions by analyzing the wrapper functions...
Still I think using the great Dagger primitives is superior over stdout and file artifacts on the host...

[tired moth]

live reload is possible now for dev containers through dagger?

[slender charm]

Forgot to add this meme

[reef shale]

WOndering if there's value in combining OTEL output from Jenkins pipeline with Dagger OTEL output

[cunning jolt]

I think "Jenkins Groovy" is the culprit.. Vanilla Groovy is pretty groovy

[sharp marsh]

thx for your time community! If you want to keep sharing awesome ideas about Kubernetes, let's continue the chat in #kubernetes 🙌

[slender charm]

Yeah that would be really cool, I am hoping someone can do a demo soon showing off some of the fancy new OTEL features 😄

[slender charm]

No, not yet, issue tracked here: https://github.com/dagger/dagger/issues/6990

[tired moth]

haha cool...

[tired moth]

why no progress here in this issue? it's really a great opportunity to compete with docker with this feature.

[slender charm]

We're a small team, many competing priorities, I agree its important, Id love to see this soon!

[tired moth]

ok cool, any ETA like 1 month 2 month can you please share.

[slender charm]

No, we dont have any ETA for this particular feature.

[tired moth]

ok

[sand gorge]

See tee El is the only correct way to say it 😉

[slender star]

and PHP is pronounced "fipp" right? 🧌

[inland coral]

https://tenor.com/view/peanut-butter-scoop-jar-gif-5490578

[sand gorge]

Semi serious answer: you'll still need a VCS to trigger your pipeline when you push 🙂

[sick monolith]

someone add dagger-vcs to the backlog!

[winter linden]

See #1243812444527394816

[sand gorge]

You're right, however things like LocalStack are acceptable for localhost development, or debugging of stuff..

Not meant to replace a real Integration environment

[inland coral]

@sand gorge (Not sure that joke travels though as you probably don't have the brand Jif in Scotland I just realized)

[sand gorge]

If it's like bleach for cleaning your house .. then we do 🙂

[patent slate]

I'd like to make a suggestion to "clean up" the Discord server. In the "Dagger and..." category, create I believe what would need to be two forums:

• languages
• integrations

And create tags with the channel names in use now. So for instance, if someone has a "Go" question, they can start a thread and put a "Go" tag on it in the "languages"forum. Or, if someone has a "pulumi" question, they can go to the "integrations" forum and start a thread there with a "pulumi" tag.

I think that would make the server much more orderly. 😀

[winter linden]

One thing to know about Discord servers... No matter how neatly you organize it, people will post all over the place

(not disagreeing with the suggestion necessarily, but just sharing hard-earned context from past reorgs)

[late patrol]

New user and not aware of any past context, but have there been any thoughts about a different platform than Discord? At least in my Enterprise, Discord is blocked and I’m sure a lot of others have it blocked too. As fun as it is to copy code on the phone it’s not exactly effective.

[winter linden]

That particular problem has been discussed for sure. Switching platforms altogether seems unrealistic, there are few good options (slack pricing for example is a non-starter for open communities) and the friction would be huge.

One thing I would love to explore, is hosting a specialized web client within dagger.cloud. So you could login to your free dagger cloud account, connect your discord username, and join this server to copy-paste code at will 😁 That's the theory anyway...

[sick monolith]

Regarding recent dagger version change and custom defined 'type' in Go:

something changed, and ive been out of loop for some time as i was not paying full attention, i was defining a go type, to simulate an enum in some ways, i.e type lang = string and then lang = Go, lang = "Csharp" etc etc, I was then using this custom type (which still is a string) as part of the dagger function API, since upgrading it no longer works and ive just rid of that concept and used pure strings instead. I'm more of a c# person, but im still interested to know if it was expected?

the other upgrade part was easy, just ensure dagger.* prefix was used on the dagger strucs dagger.File etc

[sick monolith]

i had the same issue @late patrol , i managed to get discord unblocked on a user request level, which still isnt great.

[slender charm]

I only recently learned about discourse chat - may be something to look into

https://blog.discourse.org/2022/08/discourse-chat-is-now-available-try-it-out-today/

I like discourse for support/help/howto because you get some of that sweet SEO juice and people can find things from search

But having a place to chat as well feels like the best of both worlds.

[naive leaf]

Is there any guidance on improving a build performance?

[raven stag]

people can find things from search
It's unreal the amount of searches I do through this discord server 😅

[sharp marsh]

Is there any guidance on improving a

[chilly arch]

Thanks for the feedback! When someone is onboarded onto Discord, they can choose which "dagger and" channels that they'd like to follow. If you would like to change your channel or role options, there should be an option at the top of the server where you can update your preferences.

[patent slate]

So basically, each user can "clean up" the channels for him or herself. I see. Nice. Thanks for the tip.

[chilly arch]

Exactly. I love the idea of a forum-type experience, but the channels make it feel like you can truly drop anything relevant to the tool/integration in them.

This is a good reminder for me to add more tags to our help forum though. Thanks again for the feedback!

[cunning jolt]

Looks like a bot got in here.. It's spamming messages in the channels 😦

[sharp marsh]

thx! I'm taking care 🙏

[royal comet]

any examples of the dagger sub 'pipeline' being used? I cant find it mentioned in the docs or an example? referring to this: https://docs.dagger.io/api/reference/#query-pipeline

[rain oriole]

Hey! That's now deprecated in favor of OpenTelemetry.

[royal comet]

Ahh, i see okay, thank you! It doesnt mention deprecatedi n the docs there

Also, i think recently private git repo support was added, does that mean private modules are now easy to do? I am making some slides to share with some people and dont want to give out any false info! Cant find details about 'module publishing' specific to private repos on the docs, but i was sure something recent changed there

[narrow nymph]

hm you're right

i shall go and deprecate all the pipelines 😄 (https://github.com/dagger/dagger/pull/8064)

[royal comet]

Also, i think recently private git repo

[sand gorge]

I'm catching up on the Dagger 0.12 updates https://www.youtube.com/watch?v=gUw-9kuvwDM&list=PLyHqb4A5ee1tEgcr7KsNFzQSPN-R3fPs2&index=1

The work being done is exceptional 👏 🔥

Amazing job, team!

Where can I read more about this :default thing ?

[narrow nymph]

so these are views: https://github.com/dagger/dagger/pull/6857

we haven't really documented them, since we're actually rethinking a lot of stuff in this area

i think eventually these are gonna get removed

[blazing ginkgo]

Hello! o/ Loving using dagger so far. Quick question: is it possible to get the base image address from a dagger container? I can see it's possible to get the platform, but am not able to see if there's a way to get the address. For example, using the Go SDK:

ctr := dag.Container().From("ruby")

address := ctr.??? // is it possible to get the base image address from the container?

[rain oriole]

Yes: https://pkg.go.dev/dagger.io/dagger#Container.ImageRef

[blazing ginkgo]

That's what I was looking for. Thank you!

[misty coral]

I hope you don't get this the wrong way, but maybe we're in a "XY problem" situation here. You might be better off using a base image instead of replicating it out of a string.

I cannot verify this as I haven't inspected the relevant parts of the code, but this approach most probably has the additional benefit of caching.

[blazing ginkgo]

I hope you don't get this the wrong way

[ionic plume]

Hi, I try to use registry mirror with dagger. I follow the doc https://docs.dagger.io/manuals/administrator/custom-registry but it seems not working with dagger engine 0.12.3. I have created custom dagger engine image and put on it the engine.toml on /etc/dagger/engine.toml

debug = true
insecure-entitlements = ["security.insecure"]

[registry."docker.io"]
  mirrors = ["mirror.gcr.io"]

dagger query --progress=plain <<< '{ container { from(address:"hello-world") { stdout } } }'


 : connect
2   :   starting engine
3   :     create
4   :       exec docker run --name dagger-engine-7a62182d54361fb7 -d --restart always -v /var/lib/dagger --privileged hm-registry.hm.dm.ad/docker-etloutils/dagger-engine:v0.12.3 --debug
4   :       [0.2s] | ec18d931e2e06c58b69be87bf63b82feb5af152c16456d6fd3d8c1fcab99dc2a
4   :       exec docker run --name dagger-engine-7a62182d54361fb7 -d --restart always -v /var/lib/dagger --privileged hm-registry.hm.dm.ad/docker-etloutils/dagger-engine:v0.12.3 --debug DONE [0.5s]
3   :     create DONE [0.6s]
2   :   starting engine DONE [0.6s]
5   :   connecting to engine
Setup tracing at https://dagger.cloud/traces/setup. To hide: export STOPIT=1

5   :   | engine name=ec18d931e2e0 version=v0.12.3 client=ijb6ltufhr86whwktqnlpd3hh
5   :   | cloud url=https://dagger.cloud/traces/setup
5   :   connecting to engine DONE [2.2s]
6   :   starting session
6   :   starting session DONE [0.1s]
1   : connect DONE [2.9s]

7   : moduleSource(refString: "."): ModuleSource!
7   : moduleSource DONE [0.0s]

8   : ModuleSource.kind: ModuleSourceKind!
8   : ModuleSource.kind DONE [0.0s]

9   : container: Container!
9   : container DONE [0.0s]

10  : Container.from(address: "hello-world"): Container!
11  :   resolving docker.io/library/hello-world:latest
11  :   resolving docker.io/library/hello-world:latest DONE [0.4s]
10  : Container.from DONE [0.4s]

[ionic plume]

docker exec -ti ec18d931e2e0 cat /etc/dagger/engine.toml
debug = true
insecure-entitlements = ["security.insecure"]

[registry."docker.io"]
  mirrors = ["mirror.gcr.io"]

[sharp marsh]

@ionic plume can you check if by any chance you missed this part of the process?

you need to set that env variable in order for Dagger to use the custom engine you've started

from the logs you shared seems like dagger is creating a new engine and not connecting to an already existing one. This generally happens if you forget to set the env variable shown above

[ionic plume]

yes of course, it already the case

docker ps
CONTAINER ID   IMAGE                                                         COMMAND                  CREATED       STATUS       PORTS     NAMES
ec18d931e2e0   hm-registry.hm.dm.ad/docker-etloutils/dagger-engine:v0.12.3   "dagger-entrypoint.s…"   8 hours ago   Up 8 hours             dagger-engine-7a62182d54361fb7

like you can see it use my custom image that contain my file in /etc/dagger/engine.toml

echo $_EXPERIMENTAL_DAGGER_RUNNER_HOST
docker-image://hm-registry.hm.dm.ad/docker-etloutils/dagger-engine:v0.12.3

[sharp marsh]

ok, so you baked the engine.toml in your custom image, gotcha

let me test really quick

[sharp marsh]

seems to work in v0.12.3. Just applied the same config you have but replaced "mirror.ghcr.io" with "foo.com" and I can see this in the engine logs:

time="2024-08-01T15:45:32Z" level=info msg="trying next host" client_hostname=buildkitsandbox client_id=68wm5jznd1dzix1r29elfalue error="failed to do request: Head \"https://foo.com/v2/library/alpine/manifests/latest?ns=docker.io\": tls: failed to verify certificate: x509: certificate has expired or is not yet valid: cu
rrent time 2024-08-01T15:45:32Z is after 2022-06-22T20:05:55Z" session_id=qh74fiejdrvrhpov2v7suxziu span="resolving docker.io/library/alpine:latest"

[ionic plume]

hum I see. It not speak about mirror when all right fine. We can just look that use mirror when not working like in your sample with foo.com.
I test again with the right target

[registry."docker.io"]
  mirrors = ["hm-registry.hm.dm.ad/docker-hub"]

But it seems not working.

time="2024-08-02T07:43:26Z" level=debug msg="checked for cached auth handler namespace" cached=false key="docker.io/library/hello-world::pull" name=docker.io/library/hello-world scope=pull
time="2024-08-02T07:43:27Z" level=debug msg=Unauthorized client_hostname=workspace3be3cf09740f4756-74d848975-wzqlr client_id=qc69vzr01assucbkutwos94z9 header="Bearer realm=\"https://hm-registry.hm.dm.ad/v2/token\",service=\"hm-registry.hm.dm.ad\",scope=\"repository:library/hello-world:pull\"" session_id=b31dgaprdq09nbcxqjezvduz5 span="resolving docker.io/library/hello-world:latest"

It's like that suffix /docker-hub is lost ?

[swift barn]

Is the dagger engine constantly running, maybe doing background jobs, like a daemon? Or is the Dagger engines work-state supposed to befully dependent on requests from the Dagger client?

Put much simpler: Is the Dagger engine supposed to do work outside of the scope of a Dagger function call?

[royal comet]

Is the https://play.dagger.cloud/ still a supported method for quick hackery/checks? I see it was removed since moving to the new site but its still up.

[winter linden]

It's not supported anymore, the pure GraphQL API is too different from the generated bindings, so we deemed it too confusing and not worth the maintenance cost. BUT it's just a hosted instance of graphiql (open source) plugged into a shared engine. It could easily be made a module and run locally.

[royal comet]

ohhh, interesting! I think for heavy script folk maybe there is still some good usecases of doing these 'smoke test' kinda graphql queries

Thanks for the quick reply

[sick monolith]

Just to mention some positive things, i setup a custom engine with a custom CA, toml with some priv registry, built the custom image locally with a docker file from dagger engine base image. docs were pretty comprehensive and easy to understand and smoke tested pulling a private image and it worked first time 👊

[sharp marsh]

I should support a suffix. AFAIK. Which registry are you using as a mirror?

[chilly arch]

Making our server accessible through read-only site

[karmic sand]

Hello folks, I've been using the Python SDK for the past few days. How do I install additional Python dependencies? I see the init gives me a pyproject.toml I would like to just use Poetry to do poetry add like it says in the docs here https://docs.dagger.io/manuals/developer/language-dependencies/ but that doesn't work. I get the error. [tool.poetry] section not found in /Users/u826936/Projects/github-ai-testgen-bot/apps/dagger/modules/bot/pyproject.toml Any guidence would be greatly appreciated.

[tired moth]

Hi Dagger Team,

I'd love to propose a single-user plan for Dagger Cloud, priced at $2-5/month. This plan would retain the core features of the team tier but be specifically tailored to individuals and very small teams.

Why this is important:
Rise of AI-Powered Startups: The AI boom has led to a surge in solopreneurs and micro-startups. A single-user plan would make Dagger more accessible to this growing market.
Client Project Needs: Sometimes, we need a simple way to give clients visibility into project progress within Dagger. A single-user plan would be perfect for this without the overhead of a full team subscription.
Cost-Effective Onboarding: A lower entry point could encourage more users to try Dagger, ultimately leading to more teams adopting the platform as their projects grow.

I believe this would be a win-win: more users for Dagger and a powerful tool in the hands of those who might otherwise find it out of reach.
Thoughts?

[slender star]

Language Dependencies | Dagger

[sand gorge]

In terms of the Dagger Cloud spans visualisation .. just passing on this CloudBees Jenkins UI tool, for inspiration - https://www.cloudbees.com/blog/introducing-pipeline-visualization-with-cloudbees-pipeline-explorer?utm_campaign=CM-GLOBAL-Advocacy&utm_medium=organicsocial&utm_source=sproutsocial&utm_term=&utm_content=&blaid=6362733

[slender charm]

Hi Dagger Team,

[chilly arch]

🚨 Livestream Alert! 🎥

We frequently get questions about how and when to Daggerize pipelines. For our general advice, check out this guide: https://docs.dagger.io/adopting

One of the next questions we often hear is, “Do you have any examples?” While our case studies and community demos are great resources, we’re thrilled to announce some exciting upcoming livestreams!

Join @vestal creek as he daggerizes two different open source projects live in the coming weeks. Whether you’re new to Dagger or looking to optimize your pipelines, these sessions should be very helpful and interactive.

Upcoming Livestreams:

August 7th 9 am PT - Developing & Building Open Source with Dagger for Kargo with @vestal creek and @cunning gazelle - https://www.youtube.com/watch?v=h6AhLhkcOsw

August 14th 9 am PT - Developing & Building Open Source with Dagger for OpenUnison with @vestal creek and Marc Boorshtein - https://www.youtube.com/watch?v=hbdURAMgCF8

[loud briar]

🚨 Livestream Alert! 🎥

[lime pond]

Hi folks!
I finally started doing some demos on Dagger after seeing the team at KubeCon and having a real nice explanation of what it can bring.
While creating this demos i was wondering what is the best practice regarding calling the functions.
Do you call them one by one in your pipeline or do you chain them in a encapsulating functions which you then call?
Would be great to hear your ideas around this.

[winter linden]

Hello and welcome! It depends on the constraints of your project. We address this in https://docs.dagger.io/adopting#integrating-with-ci

[royal comet]

I like that adopt page, my interpretation is, it comes down to required step behaviour, flow control you want in the team with that CI Platform, CI CD Platform Step visibility and number of functions a pipeline supports and if they can be executed under "different stragies", i.e a function you run locally does all 3, but you want those specifically to be a number of single independant function calls in the CI platform.

[chilly arch]

**📣 Dagger Community Calls are moving to a streaming platform near you! 📹 **

Starting now, we’ll be moving from Zoom to StreamYard, which allows us to stream directly to YouTube, X (Twitter), and more! This change allows you to join from your preferred platform and allows your colleagues to ask questions during the community calls if they cannot use Discord.

To ensure you never miss a call, add the series to your calendar using this link:
https://calendar.google.com/calendar/u/0?cid=Y19mZDEyYWRjOTI3ZTQ0MDVkMGY2NmUyZjA2Yjk0OTI3YzI4ZjA2YzJkMWQzZDFiOTRlNTIwMjZkNGU0MjMzNWQxQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20

This change will start with this week's community call. We will stream live from YouTube. We have a great lineup of topics from @thick igloo, @wispy tapir and @ancient escarp. See you there!

[royal comet]

hey team, think i found a bug with the cmd: feature on the secrets arg on the Windows exe - who can i message to share more? env: works fine, and this is actualy how ive been using it. But actually, I want to use cmd: and use 1Password or some other way to retrieve the secret, though what seems to be happening is its falling back to a sh/linux way of executing the "cmd:" - a bit lik how there were some issues a while back with windows based paths or exporting to Windows didn't work. I'm wondering if a gap here exists with "cmd:". the op CLI works perfectly fine, i used it in the same terminal to validate the output was a correct string. Note this is Windows Terminal with Powershell 7.

--password=cmd:"op read 'op://OMMITED'"
translated to
Error: failed to get value for argument "password": failed to run secret command "op read 'op://OMIITED'": exec: "sh": executable file not found in %PATH%

not sure if this is saying 'op' cannot be found or seeing 'sh' hinting to me that its kind of trying to do this on a linux env.

[slender charm]

Thanks for the bug report!

Can you please open a GH issue for this?

yes I think you’re right that it seems to be trying to wrap everything inside of sh, not sure if this is intentional or an oversight in windows

[sick monolith]

Sure

[sharp marsh]

yep, it's coming from here: https://github.com/marcosnils/dagger/blob/cff0042946c68b9ef2c1128b4ed089097f443b33/cmd/dagger/flags.go?plain=1#L449 cc @sick monolith

[sick monolith]

https://github.com/dagger/dagger/issues/8119

[mystic citrus]

Hello community, we are adapting Dagger for the CNCF project Harbor. Before taking on the main project and replacing 2k LOC Makefile, we are doing it in rather in small iterations and on a subproject, the Harbor CLI.
I would like to ask this community and Dagger team if you were willing to review and comment on our approach to Dagger adoption.

The current code was contributed by students, part of the LFX mentorship program.

here is the code, --> https://github.com/goharbor/harbor-cli
thank you for your feedback!

[analog raven]

Community Call starting in a few minutes! Watch the livestream here:

https://www.youtube.com/watch?v=hekNS8GkDJw

[thick igloo]

Dagger Community Call

[winter linden]

Live demo of the new context directory feature, right now on the community call 🙂 https://www.youtube.com/watch?v=hekNS8GkDJw

[chilly arch]

<@&1166745944444899449> fans, @ancient escarp will be introducing the experimental Dagger PHP module on the livestream in a few minutes. Join the link above 👆

[fossil pine]

Hey @mystic citrus ! Nice to see you here!

I’m happy to help!

Do you have something specific in mind that I should look at?

I’m also happy to jump on a call!

[fossil pine]

Anyone knows a good (preferably open source, so I can tinker) tool that visualizes GitHub Action runs (ie. workflow run time)?

I'd like to see how build times changed over time.

(If you build a billion dollar product from my idea, I want a cut)

[fossil pine]

Eh, looks like DataDog can do this as well.

[sharp marsh]

how about a Dagger module anyone could use? 😛 https://x.com/marcosnils/status/1821035248261501435

Marcos Nils (bsky @marcosnils.com) (@marcosnils) on X

@jessfraz @jmplourde_ I'm tempted to build a @dagger_io module so you can ship this to a @SignozHQ environment and perform the analysis there 🦹

Twitter · Aug 7, 2024, 4:06 AM

[fossil pine]

Dagger module anyone can use > Everything else

[sharp marsh]

and then we combine it with my RAG module and provide an experience where you can say:

• which are the most flaky pipelines?
• which is the longest pipeline?
• what's my success rate for the "build" pipeline?
• etc

[winter linden]

I am interested in this use case (from the dagger pipeline angle) because of the state persistence / accumulation angle. I have another use case where I need to run batch pipelines, that update the same DB. Don't know how to do that fully daggerized. I'm picturing a sqlite database that gets built with the builder pattern; then output to local filesystem; then loaded back again for the next run. Then from there, maybe a new persistence primitive in the Dagger API? Kind of like a cache volume, but with stronger persistence guarantees.

[fossil pine]

Speaking of "pipelines": I have one, it's called CI. But I'd really like to see how certain, distinguished parts of it perform over time. So the definition of a "pipeline" and potential smaller units are going to need some definition from a Dagger perspective.

[winter linden]

In Dagger terms, a pipeline is a dynamic invocation of interconnected functions. That's the best I have for you 🙂

[fossil pine]

My mistake! I don't need a definition. 😄

[winter linden]

Pipelines are not static, so naming and organizing them is not as easy as organizing a few entries in a yaml file

It's a problem we're grappling with in Traces

@elfin frigate has been diving into what he calls "browse by function". Specifically to give a more stable aggregate view beyond a single trace. How did "X" perform over time?

[fossil pine]

Lemme give you some actual example:

• I'd like to see GitHub checks for specific parts of my "pipeline"
• I'd like to be able to compare their execution time over time (potentially visualizing when a specific change/commit has been made)

IDC if it's a "pipeline" feature

[winter linden]

I'm also pushing for an orthogonal view which I call "browse by artifact". If functions are the arrows in the DAG, artifacts are the boxes. There's also an opportunith to show an stable aggregate view of those boxes. "Where did this directory come from? How many times has it been used", etc

[fossil pine]

I want these as GH checks and individual graphs (to put it simply)

[winter linden]

@fossil pine I hear you and the TLDR is that we're working on it.

[fossil pine]

A.W.E.S.O.M.E.

[winter linden]

The second part of the TLDR is that it's harder in a dynamic model like Dagger, than in a static model like GHA/CircleCI

[fossil pine]

I'm sure it is, which is why I approached it from a GHA perspective for now. It's a good compromise for me while you figure out how GHA and all those other CIs should have done it in the first place. 😛

[slender charm]

This is my dream!

[patent slate]

@fossil pine - Is the "generate" portion of your "addJobs" (CI) creating artifacts - the SDKs - at the end?

And are they the "end creations" of your development process?

[fossil pine]

Well, this is a CI pipeline, so the generate jobs here don't publish anything.

But yes, generated SDKs are artifacts that we publish when a release is tagged.

[flat vapor]

Anyone knows a good (preferably open

[timber crater]

Sorry I haven´t been around much lately. You all are never far from my mind.
Given the multi lingual nature of dagger and its emphasis on developer experience, cognitive load, etc. , I thought this might be of interest to folks like @slender charm , @winter linden , @slender star etc.
Last week the folks at https://pybit.es had me answer some questions about the emerging practices behind https://CodeReading.club :
https://youtu.be/0hkwYIk9EB0
(Disclosure: I´m no expert on the subject. The cognitive science behind code reading is described in https://www.felienne.com/book )
oh, and, of course @sharp marsh !

[slender charm]

Thanks for sharing, this is cool!

[idle solstice]

Does dagger have any "watch" type of functionality for local development? Watch a file/directory and perform a task when something changes?

[celest grove]

I'm planning to develop a reloader for that type of use case. Currently, I have a bunch of just file recipes that quickly return me feedback about my ongoing development. Mostly dagger develop, dagger functions calls, and test calls to my tests/*.go functions, and ensuring that I'm making golangci-lint happy 😅

[idle solstice]

I'm looking to either replace or supplement Tilt (https://tilt.dev/) with Dagger. I dislike Starlark, and want to unify the various build tasks that I have for local with the processes for CI

anyone doing something similar?

I was also considering Skaffold. I just really like the idea of building all this stuff in Go, and unifying local dev with CI processes

[idle solstice]

What's your process for local development right now?

[sharp marsh]

It's something currently being explored . You can follow https://github.com/dagger/dagger/issues/6990 for updates and you might not want to miss our community calls 😉 (FOMO alert)

[idle solstice]

Thanks issue subscribed.

[celest grove]

I start my modules from a template that I try to keep always up to date. That template includes tests/ submodule, and examples/go. All tests pass initially since there are some built-in functions that I always include in my modules (E.g., WithSource — a common one throughout most of the Dagger modules that deal with files or source code). Then, as soon as I add a function in my actual module, it has its counterpart in the tests/ submodule that I implement and test immediately. I have a bunch of just file recipes that I use to run the golangci-lint and run the tests based on different criteria.

[chilly arch]

@vestal creek livestreaming here - https://www.youtube.com/watch?v=ZDm8e4cS8ek

He is helping @median spear daggerize OpenUnison 🎉

[winter linden]

missed it, I was in a plane... how did it go?

[honest oasis]

Hey everybody, I presented how to use Runme - Open Source DevOps Notebooks - to develop and run Dagger functions/pipelines at the community meetup a few weeks ago. We had a prototype notebook integration, which is now generally available. Here's the release blog post - feedback/bug reports are welcomed. Thx!

https://runme.dev/blog/runme-for-dagger

[winter linden]

Congratulations on releasing!

[fossil pine]

Holy crap....this looks awesome!

[chilly arch]

Hey everyone! If you are in the SF/Bay Area on August 28th, we'd love to meet in person. A few of us will be at the first CNCF SF Meetup, and Solomon will be speaking. You can RSVP here: https://community.cncf.io/events/details/cncf-cloud-native-san-francisco-presents-inaugural-cloud-native-san-francisco-community-meetup/

[ocean viper]

FYI the Discord link is wrong

The one in the header seems correct

Also getting 405 errors when running the hello world

In case this is useful to anyone, here's a Dagger Nix file: https://gist.github.com/schickling/2c947b35a11d08b33c2411709a2de8c9

[winter linden]

405 method not allowed

[ocean viper]

Feature suggestion: Would be great if local traces were directly visible in a VSC extension instead of having to go through the cloud + browser.

[winter linden]

Two things we considered:

• A VSC extension that talks to cloud (to avoid having to use the browser)
• Adding features to the CLI to browse and replay past traces from your cloud account, not just the current one (to avoid having to use the browser)

We also constantly improve the CLI output so you don't have to use Cloud. In theory we could put that same UI in an extension, but I don't really see the point.

Do any of these address what you have in mind?

[ocean viper]

Would this allow for local development without internet connectivity?

[winter linden]

Well, the current terminal UI for traces does not require internet connectivity. But there are other things that do, we would need to work on that first. Like loading latest engine version, SDKs build modules etc

[ocean viper]

No problem if it's not yet available/possible. For me and my dev workflows it would certainly important to have a setup which works locally and is self-contained.

While it's nice to have a TUI sometimes, my main workflows re tracing etc would rely on a GUI I think (similar to how I'm using Grafana with Tempo)

[winter linden]

No problem if it's not yet available/

[fossil pine]

dagger-0.12.5.nix

[chilly arch]

Thanks for pointing this out. After signing up or logging into Discord, this link takes you to the help forum.

[ocean viper]

Only if you were already invited and joined.

[tired moth]

as of now can dagger run without docker with any other laternative like LXD or anything?

[sharp marsh]

yes, dagger doesn't need docker

[tired moth]

what are steps to run dagger without docker, can you please share docs link?

[sharp marsh]

sorry, we don't have that documented as it's not straightforward process and there's a reason why we leverage on container runtimes (docker, podman, k8s, etc) to provision the engine currently

if you want to investigate that yourself, you can start by manually compiling the dagger engine from here: https://github.com/dagger/dagger/tree/main/cmd/engine

[chilly arch]

discord channel links

Hello everyone!

I'm very excited to announce that we've launched the Dagger Commanders program to highlight the exceptional Daggernauts who lead, innovate, and inspire. You can learn more about the program below, and nominate the 2025 batch. Stay tuned for the 2024 batch announcement!

https://dagger.io/blog/dagger-commander-program

[tired moth]

great news

[tired moth]

Can compute drivers support distributed builds? https://github.com/dagger/dagger/issues/5583

[tired moth]

ok cool

[deep temple]

hi folks. i am wondering where do people keep artifacts if they want to store them locally ? is there an example dagger module that does what github releases does but locally ?

i am already using registry2 to store container artifacts locally. now i need the same for storing binaries like executables

[small salmon]

Hi!
Would developing a CI service that uses Dagger pipelines as the language for automation be in any conflict with Dagger licenses or trademarks?

[slender charm]

Trademark

[chilly arch]

See you at the Dagger Community Call tomorrow! 🚀

We’ll have demos from @winter linden and @inner wind , streaming live on YouTube, X, and LinkedIn. Choose your favorite platform and join us!

You can use this link to add the series to your calendar: https://calendar.google.com/calendar/u/0?cid=Y19mZDEyYWRjOTI3ZTQ0MDVkMGY2NmUyZjA2Yjk0OTI3YzI4ZjA2YzJkMWQzZDFiOTRlNTIwMjZkNGU0MjMzNWQxQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20

For quick access tomorrow, you can use these links to watch live:

• https://x.com/dagger_io
• https://www.linkedin.com/events/daggercommunitycall7230255970570924032/comments/
• https://youtube.com/live/mhswuKrdUSI

[ancient escarp]

dag()
->container()
->from('php:8.3')
->withDirectory($mySourceCode)
->withExec('phpunit --coverage-html=./coverage/html')
->directory('./coverage/html')

This code is psudocode for running unit tests and getting the code coverage report out as a directory. Here's the problem: phpunit returns a non 0 exit code if the tests fail, which means you can never get the coverage directory from the container as any query to do anything with the container returns an error from the API which translates to an exception in code and doesn't allow further interaction with that object.

Is there a way to do a withExec and ignore the error code (ideally making it available somewhere, but not via throwing an exception) ?

[narrow nymph]

😦 not yet

it takes a little bit of engine+buildkit plumbing, which i'm planning on picking up at some point soonish, but kinda been delayed with the hell that is our flaky test suite

i would probably hack it temporarily by adding a bit of shell, and writing the exit code to a file, and then returning zero as normal

then you can get the exit code by reading the file

[ancient escarp]

that is what we've been discussing :p

building out a quick "alwaysExec" module

[lime snow]

That's actually the exact solution we were considering XD

[sharp marsh]

@ancient escarp Justin's workaround is exaplined in this cookbook https://docs.dagger.io/cookbook/#continue-using-a-container-after-command-execution-fails

[winter linden]

See #1275216033862647890

[lime snow]

@sharp marsh @winter linden Thank you, those resources are both very helpful

[winter linden]

As you'll see in this thread 👆 the shell script hack does work, but it doesn't solve the other half of the problem which is that your top-level function can either return an error, or a report (file or string) but not both. We are discussing workarounds in that thread.

@lime snow @ancient escarp

[lime snow]

Brilliant, thank you. It'll take me a bit to work through the thread but this is exactly what I needed right now 🙂

[ancient escarp]

I think dagger may need, at an API level, the concept of try, catch, finally.

[sand gorge]

---

@ancient escarp do you think the workaround will actually unblock us and allow us to have a "phpunit + coverage" functionality? or are there still things missing?

[ancient escarp]

Maybe; my sorta plan is to have a utility which returns a TestResult dagger object that gives access to the coverage, test report and exit code which the calling module eg your pipeline can do as it pleases with

[sand gorge]

@ancient escarp sounds complex, and it's a custom built thing by you .. whereas we're looking for a low-level thing that people can use for themselves. It's not only gonna be phpunit/coverage report that they are after. There will be many custom .sh scripts they'll run, and they'll wanna grab the leftover artifacts, on non-zero codes.

Do you think you can make an "alwaysExec" thing, for now, until a future Dagger release comes out and we can maybe migrate over to that?

[ancient escarp]

@lime snow is already working on an alwaysExec module

it abstracts the work around

[analog raven]

We're going live in a few minutes! 😁

[elfin frigate]

the perks of YouTube (lighting aside)

[winter linden]

The livestream is starting!

Twitter: https://x.com/i/broadcasts/1vAxROjOqLjKl
Youtube: https://www.youtube.com/live/mhswuKrdUSI

See you there 🙂

[chilly arch]

module that Solomon is using - https://daggerverse.dev/mod/github.com/shykes/gha@a8671414892009ecdcadc84c8fed9d22984209bc

Smaine's repo from his demo - https://github.com/Smana/cloud-native-ref

[soft kayak]

Demo was great! 😍

My immediate thought. Let’s say, for better or worse, I have become quite good at doing things the GHA way, but see the value in driving the setup from a programming language, is there any leg up for converting YAML to the Dagger way? I’m thinking something like “scrapping” an existing repo to turn it over to the dagger side ☺️

[winter linden]

Thanks! Here's an exa,ple PR for exactly that. It's an existing project with existing hand-crafted GHA workflows. Some call dagger, some don't.

• I replace one handrolled workflow with 3 generated pipelines
• The rest is left untouched (generated and manually created yamls just co-exist in the same directory)

https://github.com/cubzh/cubzh/pull/619

cc @cold river and @blazing sluice who run that project

[stoic knot]

Who do I setup a call with to get a business plan for the Dagger UI? (we're finally updated and got the green light)

[sharp marsh]

cc @slender charm

[chilly arch]

Hybrid Meetup starting now 🎉

Our Ahmedabad, India meetup is starting now. We have a virtual option, so join us from wherever 🙂

https://www.meetup.com/dagger-ahmedabad/events/302696785/

~~You can join us using the link below. It will require you to login using Zoom: https://dagger-io.zoom.us/j/85839079872~~ Event has now ended.

[cerulean wraith]

Hi Miranda, I tried joining on the zoom link, it seems I am the only participant?

[chilly arch]

So sorry, we just ended. We have the recording though, so I will share it soon!

[cerulean wraith]

oh ok. oops :), thank you.

[chilly arch]

We look forward to seeing you at the next one! https://www.meetup.com/dagger-ahmedabad/ I'll share the recordings in our demo forum as soon as they are ready.

[heavy karma]

Hey, sorry you missed it but like Miranda said we will have a recording soon. Are you already using dagger? Anything we can help with?

[cerulean wraith]

Hey, sorry you missed it but like

[small salmon]

I’m close to have a prototype of a Swift SDK for Dagger, and I was wondering what’d take for Dagger to support running certain system commands in the host as opposed to the container. The motivation is that developers using the Swift SDK might want to use it to automate using underlying tools like xcodebuild, which only run on macOS.

[winter linden]

That's amazing!

On the topic of host commands, as discussed in #1270825925105291346 Dagger will not allow executing arbitrary commands on the host because it would break the guarantees of sandboxing, but there are possible stopgaps today, and we could also consider allowing executing specific trusted commands via a specialized API, for example dag.MacOS().Xcodebuild() or like you said, allow commands in a trusted path that cannot be used to hack the caller

[small salmon]

The thing is that developers use tools like Fastlane, which wrap xcodebuild, so the Dagger API might not work in some cases. Can the sandbox detect when a process is being run without having an API in code?

[reef shale]

Saw this Dagger demo today from Airbyte: https://youtu.be/XmPeEomIPuU?si=z8Ss8ZVmOfgDlNOD

[slender star]

Nice! Did it resonate with you?

[reef shale]

The beginning 2/3 of the video was stuff I already knew. That the last parts like the slide on the savings I wish he went into more detail. Also need to look at the demo again.

[hasty oriole]

yo, I noticed that when I use dagger for more than a couple of hours/6 or 7 CD/CI runs I end up with a docker_data.vhdx that expands to fill my entire hard-drive. This has happened twice now. (doesn't really belong in developers, coz I'm not deving for dagger itself, and it's not something I really need 'help' with as I'm pretty sure it shouldn't be happening).

[sand gorge]

@winter linden I watched your video here - https://www.youtube.com/watch?v=kqotAzSkhjw

If we're moving from a github action module that installs Dagger, under the hood, to just bringing all the internal bash stuff to the surface .. won't this make the DX worse, going from a single clean github action call, to install dagger, to now having bucket loads of bash on their .yml file?

Wondering if I'm missing something. Thanks

[winter linden]

it's the same amount of bash as before, just not wrapped in useless abstractions that dagger is making irrelevant anyway

[sand gorge]

I got that, but what about improving the DX, especially when it's software developers we are targeting, not devops/infra people.

Majority of JS/Python/PHP devs find bash daunting and want to live in their "sandbox"..

Anyway, this is just feedback. 🙂

[sacred veldt]

Partially related to Dagger, I'm looking for a blog link that I read a few years ago, the title was something like: "CI is your next legacy codebase". I think it's a great write-up why Dagger is great (as you program your CI), but can't find the article, and would need its arguments. Does this ring a bell to anyone? Do you know of related (independent) articles? Links are welcome!

[winter linden]

Is it this one? https://www.puzzle.ch/blog/2023/09/19/the-state-of-ci-tools-in-2023-the-legacy-part-1-3

[sacred veldt]

Thanks! I was looking for a different piece, but this seems to be a good resource too.

[chilly arch]

External articles

[fossil pine]

Check out how we achieved 5x performance and cut costs by 50% using Dagger and Depot: https://openmeter.io/blog/how-we-made-our-ci-pipeline-5x-faster

[storm verge]

if a company is using a combination of github actions, jenkins on k8s and spinnaker to perform CI/CD activities , how can dagger be used independent of all these systems ,ie, as a way to replace them instead of being run through them , cos if you are running using those systems, than how can one make a case to use dagger instead of those existing systems

[slender charm]

if a company is using a combination of

[deep temple]

some weird stuff happening on the dagger.cloud ui now ... it kicked me out and now i cant see any of my traces ..

[sharp marsh]

Just sent you a DM

[deep temple]

my apologies. i was login in using my gmail account as opposed to my github account and the ui was confused but it was my fault

[sharp marsh]

got it, that's what I imagined. Since your Github and Google emails didn't match, Dagger wasn't able to link your accounts

👍

[round cradle]

General question on WithExec vs Docker's RUN; would it be best practice to collapse these per a Dockerfile or is it not relevant? E.g.

foo.WithExec([]string{"ls"}).
    WithExec([]string{"ls"})

VS

foo.WithExec([]string{"sh", "-c", "ls && ls"})

[winter linden]

Both will work of course, but if possible I recommend going with the first option. This gives you more control and visibility.

[round cradle]

Thanks @winter linden, I guess what I'm curious about is whether there are ramifications to image size? Or is it not operating the same way.

RUN wget http://example.com/software.tar.gz 
RUN wget tar -xvf software.tar.gz 
RUN mv software/binary /opt/bin/myapp
RUN rm software.tar.gz

VS

RUN wget http://example.com/software.tar.gz &&\
    wget tar -xvf software.tar.gz &&\
    mv software/binary /opt/bin/myapp &&\
    rm software.tar.gz

[winter linden]

Yeah those same rules apply to Dagger. So if those are your 2 options, the second would be better. But in this situation you're better off running tar in one container, then copying the directory over to another container. Basically the multi-stage pattern from Dockerfiles, but generalized.

func DownloadToContainer(url string, ctr *dagger.Container) *dagger.Container {
 binary := dag.
  Container().
  From("alpine").
  WithExec([]string{"apk", "add", "tar"}).
  WithFile("software.tar.gz", dag.Http("http://example.com/software.tar.gz")).
  WithExec([]string{"tar", "-xvf", "software.tar.gz"}).
  Directory("software/binary")
 return ctr.WithFile("/opt/bin/myapp", binary)
}

Note: we really should add tar primitives to the core API, that would remove the need for installing tar and running it in a container in this case

[round cradle]

Got it, appreciate the detailed response! 🙇

[winter linden]

No problem. I love using Dagger, always looking for an excuse 🙂

[small salmon]

I've got a prototype for a Swift SDK and I'll continue working on generating the code based on the GraphQL schema. I'd appreciate if anyone could chim in in this conversation and give me some pointers:
https://github.com/dagger/dagger/discussions/8280

[rain oriole]

Sure, taking a look.

[small salmon]

Thanks!

[winter linden]

Thank you @small salmon ! We were just talking about this the other day

[deep temple]

newb question. is it garanteed that dagger will call each function in sequence and fail if one of them fails ?

or do i need to call dagger seprately for each function

[unreal latch]

How to install modules from private repositories?

[lapis slate]

Hello, @unreal latch 👋
See if this comment by @swift inlet helps you
#1279115372116705422 message

[swift inlet]

Thanks @lapis slate 🙏 please keep us in touch

[unreal latch]

hi & ty.....
thought dagger use https://<gh-uname>:<gh-token>@github.com/*/** would work 😅 ...w/o populating it in dagger.json

will wait for ssh auth 👍

oh 12.6 released too

[swift inlet]

ssh auth is available in the new 12.6 😇

[sly wharf]

Hello, is there a self hosted version of dagger cloud ?

[winter linden]

Not yet available, but it's on the roadmap. Would you be willing to get on a call to discuss your use case, and possible early access?

[sly wharf]

I can only use tools that can be installed in an airgaped environment, the dagger project is really interesting and I wish being able to use it asap 😁

[tight solstice]

Hi !
Is there a way to continue the execution after a WithExec fail ?

[bitter sierra]

Hey, I have a monorepo of many go modules, I would like to run golangci-lint in all of them in parallel, does dagger support that (yet)?

[sharp marsh]

hi! yes, there's a stopgap here: https://docs.dagger.io/cookbook/#continue-using-a-container-after-command-execution-fails and a discussion to improve the design here: https://github.com/dagger/dagger/issues/5981

[sharp marsh]

yes it's definitely supported. You can use your favorite language's concurrency / parallelism capabilities (goroutines, coroutines, async io, etc) and call as many Dagger pipelines as you need

[bitter sierra]

@sharp marsh thanks for the ultra-fast reply. Do you have an example preferably in golang somewhere (can be part of a bigger project). I would like to see how output to console is handled.

[tight solstice]

func(m *Wico) UnitTesting(ctx context.Context, source *Directory)(*Directory){
    db := m.Database("dev_test")

    test := m.BuildEnvTest(source).
        WithServiceBinding("db", db).
WithExec([]string{"php","-d","memory_limit=-1", "bin/console", "doctrine:database:drop", "-n" ,"--force","--env=test", "--if-exists"}).
        WithExec([]string{"php","-d","memory_limit=-1", "bin/console", "doctrine:database:create", "-n","--env=test"}).
        WithExec([]string{"php","-d","memory_limit=-1", "bin/console", "doctrine:query:sql", "-n", "CREATE EXTENSION IF NOT EXISTS postgis;","--env=test"}).
        WithExec([]string{"php","-d","memory_limit=-1", "bin/console", "lexik:jwt:generate-keypair", "-n", "--overwrite","--env=test"}).
        WithExec([]string{"php","-d","memory_limit=-1","bin/console" ,"doctrine:schema:update" ,"-n", "--force","--env=test"}).
        WithExec([]string{"php","-d","memory_limit=-1","vendor/bin/phpunit", "tests/test", "--stop-on-failure"}).
        WithExec([]string{"pg_dump", "-h", "db", "-U", "admin" ,"-d" ,"dev_test", "-f", "dump.sql"})
        //Stdout(ctx)
    return test.Directory(".")}

I have smt like this and i want to get my dump even if my unittest fail

[sharp marsh]

the console output depends on what you need. If you use the Dagger CLI or Dagger Cloud, the ouput will be properly formatted as Dagger keeps track of each DAG path in the pipeline. If you just print everything to stdout, all the outputs will be mixed with each other

[sharp marsh]

you could also export each output to a file and store that in your filesystem if needed

[tight solstice]

WithExec([]string{"sh", "-c", "php -d memory_limit=-1 vendor/bin/phpunit tests/test --stop-on-failure; echo -n $? > /exit_code"}).

this should be woriking

[sharp marsh]

yes, correct.

[tight solstice]

I dont know if im the onyly one with the issue but after starting multiples time my dagger call my disk start to be full, i need to docker prune my volumes (more than 200gb)

[sharp marsh]

@bitter sierra here's a simplified example in Go

func (m *Lala) Test(ctx context.Context) ([]string, error) {

    ret := make([]string, 2)
    var g errgroup.Group
    g.Go(func() error {
        out, err := dag.Container().From("alpine").WithExec([]string{"sleep", "5"}).WithExec([]string{"echo", "task 1"}).Stdout(ctx)
        ret[0] = out
        return err
    })
    g.Go(func() error {
        out, err := dag.Container().From("alpine").WithExec([]string{"sleep", "3"}).WithExec([]string{"echo", "task 2"}).Stdout(ctx)
        ret[1] = out
        return err
    })

    return ret, g.Wait()
}

[bitter sierra]

ohh Stdout(ctx) writes a string return, that's nice.

[sharp marsh]

In this case, I'm just appending the results into an array and returning that as part of the function output but as I said before, you could either export it to a file or do any other things you want with it

[bitter sierra]

so I could use hashicorp/go-multierror and a channel to run x workers, nice 🙂

Not sure this is relevant, it's meant to replace https://github.com/go-orb/plugins/blob/main/scripts/test.sh goal is to run the pipeline cross platform without any host dependencies.

[sharp marsh]

nice! thx for sharing. Let us know how can we help. Looking forward to how the final pipeline will look like 🙏

[tight solstice]

In this exemple ```go
func (m *MyModule) Test(ctx context.Context) (*TestResult, error) {
ctr, err := dag.
Container().
From("alpine").
// add script with execution permission to simulate a testing tool
WithNewFile("run-tests", script, dagger.ContainerWithNewFileOpts{Permissions: 0o750}).
// if the exit code isn't needed: "run-tests; true"
WithExec([]string{"sh", "-c", "/run-tests; echo -n $? > /exit_code"}).
// the result of sync is the container, which allows continued chaining
Sync(ctx)
if err != nil {
// unexpected error, could be network failure.
return nil, fmt.Errorf("run tests: %w", err)
}
// save report for inspection.
report := ctr.File("report.txt")

// use the saved exit code to determine if the tests passed.
exitCode, err := ctr.File("/exit_code").Contents(ctx)
if err != nil {
    // exit code not found
    return nil, fmt.Errorf("get exit code: %w", err)
}

// Return custom type
return &TestResult{
    Report:   report,
    ExitCode: exitCode,
}, nil

}

Why use Sync ?

[tired moth]

Let's say if we want to orchestrate a few pipelines without a container then is it advisable to use a dagger?
It will work effectively?

[bitter sierra]

@sharp marsh this works: https://gist.github.com/jochumdev/efa1f74feb83aabc2a15518afa7fccf9

[bitter sierra]

In this exemple ```go

[sick monolith]

dagger runs in containers. I think you'll need to give an example.

the dagger engine also runs inside a container. so I'm not sure if we will understand your question unless you can expand a bit more

if you mean not publishing a container, dagger is very useful, as its not only for publishing containers. for example you could take input files and spit files back out, the result isn't a container. the process however used containers, since that's how dagger works. are you asking if you could create a dagger function that itself doesn't use containers (except for the dagger engine and session)?

[bitter sierra]

So there's no way with dagger to directly write back to the host, right?

Like "bind-mount" some directory as workingdir run a command inside the docker and have the results directly on the host.

I see why, it's to much permissions and would bring lots of other problems, still would be nice.

[lime snow]

I believe there is an export function you can call which may do what you're after

[bitter sierra]

@lime snow I found the export function, it seems to export the fs, I want to directly work on the hosts fs.

[lime snow]

I see, in that case I'm not sure I can help

[bitter sierra]

thanks a lot, still!

[sick monolith]

if a dagger engine is running on your machine, youre looking for somehow, the engine to have permissions to write files back to the machine its running on, not going thorugh the client? I don't think ive seen an example of that. since the comms is via the GraphQL API which is how i think the engine is basically executing what its supposed to. I think if it was possible (im not 100% sure), i think it kind of goes against the portability concept a bit? Since now youre dropping files outside of a 'pipeline' where right now the export function (which could be a directory) is kind of the 'end of the pipeline' and its coming back to you via the engine and the dagger client.

theres this kind of two-way communication channel happening between client <-> engine with a session

But i have seen something in a video of getting a directory without passing it via the client as an argument. I havent played around with that myself yet though

stealing this screenshot from one of the videos

https://www.youtube.com/watch?v=eoicVPLDqbU

theres a dag.host() function, but i dont think its what youre looking for behaviour wise

[bitter sierra]

Thank you, yeah what I thought. Two way access to FS breaks a lot.

https://gist.github.com/jochumdev/efa1f74feb83aabc2a15518afa7fccf9#file-main-go-L132 <-- this is what I try atm. (non working)

[sick monolith]

hmm

[bitter sierra]

Currently it fails as I don't know howto convert a dagger.Directory back to String

[sick monolith]

wait i think it should be possible, but i think it wont understand the path

[bitter sierra]

So I have to use string as "root" and resolve that?

Then I could build the hosts path on my own.

[sick monolith]

i was watching this video and there ws something about relative paths not wokring, but it wasnt regarding the export function (i dont want to lead you down a wrong path)

https://youtu.be/g2E86Ke_RME?t=1054

[bitter sierra]

well, lemme see 🙂

[sick monolith]

timestamped about something with relative/absolute paths - but its not directly answering 'export'

hmm, its about accessing parent directory with relative, so not really related to your issue

i would deffo chuck something into the 'help' channel here about using export to host

[sick monolith]

could you collect all the files into a root dir, and export that directory out to the host system as a specific location (and it should allow writing into non empty host directories that are children of that dir you export to)

[bitter sierra]

I don't know yet howto export a single file/directory yet. Everything i tried didn't work.

[tired moth]

yes exactly, is it possible right?

[sick monolith]

my usecase of dagger so far hasnt been publishing containers, its been generating files and spitting them back out

you still need a location to run dagger though, so you need an env where it can run.

[loud briar]

Any examples of the context directory uses floating around?

[tired moth]

ys that's fine after giving an env but it can run, right?

[sharp marsh]

👋 it's highlighted here: https://docs.dagger.io/cookbook/#export-a-directory-or-file-to-the-host

[sick monolith]

can you give an example because your question is still a bit confusing to me - it seems like youre asking if you can use dagger without containers (and one of the selling points of dagger is containerised pipelines).

I dont want to say yes, because i still dont entirely know for sure what youre asking 😄

[tired moth]

ok let me clear it, i want to orchestrate pipelines but those which are running without containers. let me know if still i am not clear

[sick monolith]

I wrote something that does this:

generate-kiota-clients --languages CSharp,Go,Python,Java file --path /output.tar.gz export --path ./generated.tar.gz

https://gist.github.com/pjmagee/8eca49b36dab6eda8163569ff0a1ab66

[sick monolith]

so you want to use dagger, which uses containers... for what?

you want to build a pipeline, that isnt in containers, with containers?

it doesnt make sense to me sorry lol

[tired moth]

i think i am making it more complicated, let me get back to you with more details. thanks a lot for your help and support.

[sick monolith]

now im curious though 😄

[sharp marsh]

when you use the Export method inside a function, that gets exported to the SDK runtime container, not to the host. Given that the Dagger model is to be extremely descriptive and cautios about your function's inputs and outputs, the way to export things to the host is by returning either a dagger.File or a dagger.Directory from your function

[bitter sierra]

Means this will NEVER work https://gist.github.com/jochumdev/efa1f74feb83aabc2a15518afa7fccf9#file-main-go-L132 ?

[sick monolith]

yeah, what you can do, is aggregate all those files into a dag.directory and then return the directory, and export that to the host - and any new files will appear

[sharp marsh]

correct, you need to return the files in your function.

[sharp marsh]

this

[sick monolith]

it doesnt need to be a single file or a zip, you can return a dir, and i think there is a setting around some behaviour on host dir not being empty or not and if it should continue or fail

[bitter sierra]

This is bad, yeah "go get -u -t ./..." has no need to be run in parallel but that was fun....

[sick monolith]

i vaguely remember there being some setting there, because i was generating new api clients from a new version the lib, but i didnt export to a empty host dir, i was replacing older ones and seeing the diff in git

[bitter sierra]

May I export from the SDK runtime container to the host?

[sick monolith]

joc, what you could do (maybe it defeats the point now, im not a gopher myself) - could you write a script, which you execute with bash and run many dagger function calls (each call is its own session)

i didnt entirely get what youre doing though, so ignore me if what im saying is not helpful, just trying to think of other things you can do, i think youre trying to make use of dagger for local usage only, not as a portable container pipeline. and you may be able to achieve some of that, by running many dagger function calls somehow with some scripts (obviously defeats the point of dagger in some ways though)

[sudden ocean]

👋 Just sent a patch to melange (https://github.com/chainguard-dev/melange) to make Dagger runner backend works here https://github.com/chainguard-dev/melange/pull/1470. I'd be glad if someone could help review + test the PR. ^^

[bitter sierra]

many thanks for trying to help me. Yes a script could do the same. What I'm doing:

run "go get -u -t ./..." in parallel inside a monorepo with lot's of "go.mod" files and get the resulting (go.mod|go.sum) back

This is nothing important, so i can stop here and do it on another way.

I have my lovely Taskfile.yml here: https://github.com/go-orb/workspace/blob/main/Taskfile.yml - this one does a lot in parallel but it's not cross-platform ready and has no lovely dagger caching, wanna replace everything with dagger.

Seems there's something in the pipeline: https://github.com/dagger/dagger/issues/8235

[sharp marsh]

you can definitely do this, the main difference is that currently has a single ouput per function call. So you'll have to do something like this:

// Returns a container that echoes whatever string argument is provided
func (m *Lala) Test(ctx context.Context, root *dagger.Directory) (*TestRet, error) {

    ret := make([]*GetRet, 2)
    var g errgroup.Group
    g.Go(func() error {
        c := dag.Container().From("golang").
            WithDirectory("/app", root.Directory("one")).
            WithWorkdir("/app").
            WithExec([]string{"go", "get", "-v", "-u", "./..."})
        out, err := c.Stdout(ctx)
        if err != nil {
            return err
        }
        ret[0] = &GetRet{Logs: out, Source: c.Directory("/app")}
        return err
    })
    g.Go(func() error {
        c := dag.Container().From("golang").
            WithDirectory("/app", root.Directory("two")).
            WithWorkdir("/app").
            WithExec([]string{"go", "get", "-v", "-u", "./..."})

        out, err := c.Stdout(ctx)
        if err != nil {
            return err
        }
        ret[1] = &GetRet{Logs: out, Source: c.Directory("/app")}
        return err
    })

    err := g.Wait()
    if err != nil {
        return nil, err
    }

    tr := &TestRet{
        Logs:   []string{ret[0].Logs, ret[1].Logs},
        Source: dag.Directory().WithDirectory("one", ret[0].Source).WithDirectory("two", ret[1].Source),
    }

    return tr, nil
}

this way you can call dagger call test --root . source export --path . to update your local paths and dagger call test --root . logs to get the logs

[sharp marsh]

here's TestRet and GetRet structs

type TestRet struct {
    Logs   []string
    Source *dagger.Directory
}

type GetRet struct {
    Logs   string
    Source *dagger.Directory
}

[winter linden]

💯 This pattern is even more useful now that we support context directories. I think it's a must for any codegen use case. I use it to clean up previously generated files - otherwise you end up littering the directory with "dead" generated files when you change the codegen logic, rename files etc

[deep temple]

are the old pre module apis still available and documented ?

[winter linden]

It's the same API, so yes 🙂

[deep temple]

i am trying to build a cli tool that just calls the api directly. is there an example or documented way ? trying not to go dig in the git history

[sick monolith]

https://docs.dagger.io/reference/cli/#dagger-query is this useful?

[deep temple]

    // create Dagger dagClient
    dagClient, err := dagger.Connect(ctx,
        dagger.WithLogOutput(os.Stdout),
        dagger.WithSkipCompatibilityCheck(),
    )

    if err != nil {
        return nil, err
    }

    image := os.Getenv("MY_IMAGE")
    // tcache := dagClient.CacheVolume("test-00")

    // create HTTP service container with exposed port 8080
    httpSrv := dagClient.Container().
        From(image).
        WithExposedPort(8080).
        // WithMountedCache("/root/test-00", tcache).
        AsService()

    // expose HTTP service to host
    tunnel, err := dagClient.Host().Tunnel(httpSrv).Start(ctx)
    if err != nil {
        return nil, err
    }

    // get HTTP service address
    srvAddr, err := tunnel.Endpoint(ctx, dagger.ServiceEndpointOpts{
        Scheme: "http",
    })
    if err != nil {
        return nil, err
    }

    // Create a Client to access HTTP service from host
    client, err := client.NewClient(srvAddr)
    if err != nil {
        return nil, err
    }

i was able dig this code in go i had pre modules as opposed to using the graqhql apis. the go apis were more expressif.i will see if i can reuse that code same code now

[sick monolith]

I think it was always graphQL

[deep temple]

i think so to .. but it was on top of it. sorry for not making that clear

[sick monolith]

well, it has been since i joined, but i only looked more when functions were released

[winter linden]

Yes it's all the same API, the only differences are superficial: 1) imports, 2) establishing the client connection, and 3) host access which is a special case

We do need to document those differences better, that's coming soon

You can skip the dagger.Connect() in your example, to make it closer to the in-module code. See https://dagger.io/blog/simpler-reusability-with-the-global-client

TLDR in Go you can import dag then use it the same as in modules:

import (
    "context"
    "fmt"

    "dagger.io/dagger/dag"
)

func main() {
  out, _ := dag.
    Container().
    From("alpine").
    WithExec([]string{"echo", "hello world"}).
    Stdout(context.Background())
  fmt.Println(out)
}

[deep temple]

i really liked that api !!! thank you so much for keeping it around

[winter linden]

In a perfect world there would be zero difference - we will get there

[deep temple]

the best thing i liked about that api is that i was able to debug the code much more easily. i can set a breakpoint and get to it quickly. so far i am not sure how people debug their modules but i am doing printf like debugging and it really is breaking my compile/debug cycle in my ADD head

some people like doing debugging with printf i always envied them

[loud briar]

-i and very little else at this point, that terminal has made me ultra lazy

[deep temple]

i have not tried -i yet .. but if the logic that makes a container is the one you are debugging then thats harder with -i

[loud briar]

Yeah, it takes a bit of rethinking how you create/use/return types to make the most of it, but worthwhile I think

[winter linden]

If you're writing a custom client, then debugging will work the usual way. What prevents regular debugging in Dagger Functions is not the API, but the runtime environment of the client. Dagger Functions don't run on your native host system, they run inside the Dagger Engine - so you need specialized debugging hooks, which we haven't implemented yet (except for --interactive).

TLDR: you can call the same Dagger API from a custom client, and you can debug your client the usual way.

[deep temple]

would it make sense to switch to a local execution mode while in debug mode ? since its all generated code and the dagger function has no clue where it is running i am assuming it would be pissible to generate a "debug" build mode so that all is running locally and be debuged like a normal go function ? that would be super amazing to say the least

[sick monolith]

feels like what people are saying is remote debugger tooling is needed somehow. since a dagger function call even when you run it 'locally' isn't really local, its still deployed into the dagger engine, which just happens to have dockerd/podman on your machine.

you cant step through dagger code like a normal debugged 'hello world app' because its not running on your machine with the tooling, executing in-realtime on the literal host

each dagger call is "take this code, zip it and ship it, and execute it over there"

I do think its a nice to have, not a must though. the fact that you can write and run locally already saves a lot of time from having to do this sludge cycle: change, commit, push wait, review (and repeat until it works). Now you can just do it all locally, and then that same thing you made work locally, will run anywhere

hot take: i think if you need to debug a pipeline like normal software, the pipeline is maybe overcomplicated/overengineered.

most of the time, arent we dealing with compilation, binaries, input, output, folders, zipping, running some other clis tools/images.

If we need to start 'stepping through' due to loops and hoops in our dagger code. maaybe that's a sign something could be way simpler

but i do agree, it feels a little, anxiety enducing when you write code and you 'cant debug it', it certainly isnt a natural feeling, and i would deffo say debugging would be nice

[bitter sierra]

I've tried over 2 tired hours to implement that, failed. Somehow I never get any files transfered to my machine.

[sharp marsh]

sorry about that :(. Would you like to go to #911305510882513037 and take a look really quick?

[bitter sierra]

wow, yeah 🙂

that was quick, I forgot to assign the result of a WithDirectory() back to it orginating value (like r = append(r, "b") )

[sand sierra]

Is this https://docs.dagger.io/manuals/developer/cache-volumes/ scoped on the module your are building? Or shared across all dagger calls for all projects?

[tight solstice]

I have a question about dagger cloud ?
Can i self host this solution ?

[narrow nymph]

heya @tight solstice, i'll just quote our ceo from earlier 😄 #general message

[narrow nymph]

nope, not currently scoped/namespaced, but you can track this issue in https://github.com/dagger/dagger/issues/7211 (feel free to add any feedback/ideas or reasons why it's important to you!)

[sand sierra]

Tnx!

Is there a way to clear snapshots? Or cache volumes? My method now is a shell script that removes the dagger container 😓

[narrow nymph]

ah, i think you likely want something like this? https://github.com/dagger/dagger/issues/6260
but yeah, sorry, we don't really have that just yet 😦

[sand sierra]

yes exactly. It would be nice to see cache volumes and the layers cache.

I think the idea is amazing, and will try dagger cloud soon.

[tight solstice]

WithExec([]string{"php","-d","memory_limit=-1","vendor/bin/phpunit", "tests/mytests", "--stop-on-failure"}).
        WithMountedTemp("/").

Will it clear the file on my FS (cause after few run my disk start to be full) ?

[tight solstice]

and where do you find the doc with all method that you can use ?
here ? https://pkg.go.dev/dagger.io/dagger

[sick monolith]

https://docs.dagger.io/

theresa GraphQL page somewhere here, one moment

https://docs.dagger.io/api/reference/

[tight solstice]

yeah im on it but if i need the doc of : Sync or WithExec (using go sdk)

[sick monolith]

Since Go,Python,Ts all 'wrap' GraphQL - This is the source of truth

[sick monolith]

This looks better actually

yeah so, you would usually be working off a container the most: https://pkg.go.dev/dagger.io/dagger#Container

[tight solstice]

Cause im stuck with Terminal()
i cant find the way to use it

func(m *Wico) UnitTestingW
(ctx context.Context, source *Directory)(string, error){

    test := m.BuildEnvTest(source).
        WithServiceBinding("db", m.Database("dev_test")).
        WithEnvVariable("DATABASE_URL", "postgresql://admin:admin@db:5432/dev").
        WithExec([]string{"rm", ".env"}).
        WithExec([]string{"cp", ".env.test", ".env"})
    return test.
        WithExec([]string{"php","-d","memory_limit=-1", "bin/console", "doctrine:database:drop", "-n" ,"--force","--env=test", "--if-exists"}).
        WithExec([]string{"php","-d","memory_limit=-1", "bin/console", "doctrine:database:create", "-n","--env=test"}).
        WithExec([]string{"php","-d","memory_limit=-1", "bin/console", "doctrine:query:sql", "-n", "CREATE EXTENSION IF NOT EXISTS postgis;","--env=test"}).
        WithExec([]string{"php","-d","memory_limit=-1", "bin/console", "lexik:jwt:generate-keypair", "-n", "--overwrite","--env=test"}).
        WithExec([]string{"php","-d","memory_limit=-1","bin/console" ,"doctrine:schema:update" ,"-n", "--force","--env=test"}).
}

im starting from this and i want to debbug with terminal

[sick monolith]

you can return a dagger.Container and then from the CLI, do dagger call unit-testing-W --source=. terminal

[tight solstice]

so i change the return value to *Container

but it dont seem to return a container

[sick monolith]

WithExec should return a *Container

you have an extra . at the end of your example above

[tight solstice]

ok seem to work ! it's building i have this WithMountedTemp("/") at the end

[sick monolith]

you should also be able to place Terminal() inbetween those exec statements

but sometimes its nice to return a Container, and then from the CLI/HOST you can force evaluation dagger call some-function --my-arg=. + sync|stdout|terminal

leaves you a bit more flexibility because you can then pass the container to another function, add more layers or w/e might be needed.

[tight solstice]

after ```go
WithExec([]string{"php","-d","memory_limit=-1","bin/console" ,"doctrine:schema:update" ,"-n", "--force","--env=test"}).
Termial()

but it was not working

cause Terminal return *dagger.Terminal

and i cant put WithMountedTemp after terminal

[sick monolith]

https://docs.dagger.io/manuals/developer/debugging/#use-the-terminal-method-to-inspect-the-pipeline-live

[tight solstice]

yeah i see ! But dagger returned an error about return type

[sick monolith]

oh, hmm. are you using the latest version of dagger?

[tight solstice]

Yep

[sick monolith]

strange, it even says in the docs it returns a Container

[sick monolith]

https://github.com/dagger/dagger/issues/7711

[tight solstice]

I opened a thread https://discord.com/channels/707636530424053791/1280174756091396187

[tight solstice]

./main.go:144:3: test.WithExec([]string{…}).WithExec([]string{…}).WithExec([]string{…}).WithExec([]string{…}).WithExec([]string{…}).Terminal().WithExec undefined (type *dagger.Terminal has no field or method WithExec)

func(m *Wico) UnitTestingW(ctx context.Context, source *Directory)(*Container){

    test := m.BuildEnvTest(source).
        WithServiceBinding("db", m.Database("dev_test")).
        WithEnvVariable("DATABASE_URL", "postgresql://admin:admin@db:5432/dev").
        WithExec([]string{"rm", ".env"}).
        WithExec([]string{"cp", ".env.test", ".env"})
    return test.
        WithExec([]string{"php","-d","memory_limit=-1", "bin/console", "doctrine:database:drop", "-n" ,"--force","--env=test", "--if-exists"}).
        WithExec([]string{"php","-d","memory_limit=-1", "bin/console", "doctrine:database:create", "-n","--env=test"}).
        WithExec([]string{"php","-d","memory_limit=-1", "bin/console", "doctrine:query:sql", "-n", "CREATE EXTENSION IF NOT EXISTS postgis;","--env=test"}).
        WithExec([]string{"php","-d","memory_limit=-1", "bin/console", "lexik:jwt:generate-keypair", "-n", "--overwrite","--env=test"}).
        WithExec([]string{"php","-d","memory_limit=-1","bin/console" ,"doctrine:schema:update" ,"-n", "--force","--env=test"}).
        Terminal()

[ancient escarp]

Does dagger support extending other objects (specifically ones from core, but inter module in general I guess) ? Use case is to setup a container with specific tooling and offer access to those tools via dagger function calls but also have access to the underlying container functions.

[tight solstice]

Like haivng a core container and use it in other functions ?

[ancient escarp]

More that I want a container object which also has additional functions; see: https://github.com/carnage/dagger-php-module/blob/main/dagger/src/PHPContainer.php Ideally I'd like to be able to call functions on the container object but still have it be a php container afterwards so I can mix withExtension and withComposer calls into chains of withExec

I can't just allow passing an arbitary container into the function calls as it may not have php or the extension installer added

[tight solstice]

https://pkg.go.dev/dagger.io/dagger#Container.Terminal Yeah

[loud briar]

I recall there was some discussion about merging the engine/cli into a single container a while ago, did that ever go ahead? Can I just use the Dagger container as a cli base image now?

[narrow nymph]

yup 🎉

since v0.12.5: https://github.com/dagger/dagger/blob/main/CHANGELOG.md#v0125---2024-08-15

[loud briar]

So in gitlab terms, I could use image: <wherever_its_hosted>/dagger:<engine_version> and as long as my scripts are just dagger <stuff> I'm good?

[narrow nymph]

yup 🙂

[loud briar]

I previously had an Dockerfile that had FROM <dagger> VOLUME /var/lib/dagger CMD [command with --addr tcp and --addr buildkit socket]

[narrow nymph]

it's alpine based (for now), so you can also install any apk-based things you might need (though i might not recommend that, i'm not sure how stable we might keep that particular feature - though if we did change it, it would be a minor patch)

[loud briar]

Do I still need to provide those instructions (volume/cmd)?

Oh hang on, that might be for the engine side, not the cli, ignore!

[loud briar]

@narrow nymph Quick one, if I provide _EXPERIMENTAL_DAGGER_RUNNER_HOST and use image: <daggerimage> in a Gitlab job I presume it's not going to try and start a local engine (which will fail, no way to provision it) due to the env var

[narrow nymph]

as long as you provide _EXPERIMENTAL_DAGGER_RUNNER_HOST, then yup, it should point to that

[cunning jolt]

If you provide that var it will pull the daggerimage from the registry and use that. If you want to use the actual dagger engine within the container (engine image) you will also have to run it --privileged. I am doing something similar just to get the CLI matching the engine version. I can't run but base image --privileged so I can't use the engine that comes inside the image.

[loud briar]

I'm running Dagger on a separate host, just updating this setup from last time I used it in 0.11

Follow-up: where are Dagger images hosted? Ghcr?

Found it

[loud briar]

Engine-as-cli

[blissful gull]

Hello. I need the dagger engine to spin up attached to a different docker network. Is this possible? For security reasons the default bridge network on our production hosts is completely locked down.

[sharp marsh]

yes, you can manually start the engine yourself. As long as the container name is the same, the dagger CLI will use it. Alternatively, you can also set _EXPERIMENTAL_DAGGER_RUNNER_HOST=docker-container://$name if you want to use a custom name

[stray valve]

How does dag.cacheVolume work internally? I mean, where does this volume live? In the dagger engine container or another container is spawned by the engine?

[sharp marsh]

This lives within the Dagger engine. We're working on providing better interfaces so you can have more control over this. Current discussion is happening here: https://github.com/dagger/dagger/issues/8004

[loud briar]

Small parts of this seem like the groundwork for better cache visibility in Dagger Cloud? That's probably my number one request at the moment - cache hits/misses in Cloud

[sharp marsh]

That's coming soon 🙏

[stray valve]

Thanks for the link. From what I understood, some graphql apis were added recently but is there a way to manage a cache volume right now? I'd like to delete them or just see what is being cached using either the CLI or an SDK. I found this loadCacheVolumeFromID in the GraphQL API Reference.

[winter linden]

You're looking for the daggerEngine() top-level function.

From the dagger CLI (0.12.6 or later):

dagger core dagger-engine --help

[stray valve]

oh, just came here to say that I found it

the only function right now is dagger core dagger-engine local-cache and it pruned it

okay, so from the issue above, we can expect to see more ways to interact with it, like seeing a list of cache volumes, etc?

[winter linden]

Yes that's the idea, @sipsma is the mastermind behind this API, he just left on vacation

[stray valve]

Nice!

[winter linden]

but now that the foundation is laid, if needed we can add more calls to it. Just don't want to step on Erik's plans for what to add next

The idea is to have a special admin API

not accessible from sandboxed functions for security reasons

[stray valve]

got it. Thanks

[bitter sierra]

Dagger now succefully replaced our bash scripts: https://github.com/go-orb/plugins/commit/66296afd030f710f82451c44aa76468ac29300fa

[sharp marsh]

cc @chilly arch thinking about a community call material here 🙏

[tight solstice]

Hi !
How can i pull a docker image from a private repos in dagger ?
Just by using docker login ?

[chilly arch]

🔔 Join us for our bi-weekly Dagger Community Call tomorrow from the streaming platform of your choice:

• https://twitter.com/dagger_io
• https://www.linkedin.com/events/daggercommunitycall7232102438793584640/theater/
• https://www.youtube.com/watch?v=LbUgs1UMQcE

We have two exciting talks lined up:
1️⃣ Offhand: a ZSh plugin for daily Dagger rogues - @austere maple
2️⃣ Self-corrective agentic workflow powered by Dagger - @karmic sand

📅 Date: Thursday, September 5th
🕒 Time: 9:00 am PT

Add the series to your calendar: https://calendar.google.com/calendar/u/0?cid=Y19mZDEyYWRjOTI3ZTQ0MDVkMGY2NmUyZjA2Yjk0OTI3YzI4ZjA2YzJkMWQzZDFiOTRlNTIwMjZkNGU0MjMzNWQxQGdyb3VwLmNhbGVuZGFyLmdvb2dsZS5jb20

[sick monolith]

wow, windows anti-virus quarantined dagger.exe

Under this: Program:Win32/Wacapew.C!ml

annoying it doesnt give any proper explanation, it works fine after using the restore feature.

[bitter sierra]

Seen that behaviour of Windows Defender before with Go executables.

[sick monolith]

Yeah, i submitted it as a False Positive / Incorrect Detection on the Windows Defender website.

Waiting for someone on their team to finally mark it off, might take some time though...

[sand sierra]

It it possible to export files from a dagger.Directory as your own user / group? They are created as root on my fs

[sick monolith]

i just had a look at the export ops, looks like only Wipe is a possible option to toggle true/false. Are those files set as root in the container too? maybe it's using the same user - could you change it in the container first?

there's a .WithUser you can set on a container for commands.

[sand sierra]

Yeah that is a possibility yes. I've had the same when running it with docker

Would be nice if it did not require changes in my config

[sick monolith]

If you don't get something working, raise a new post under the #1030538312508776540 section. theres a nice collection there you can see of various issues and their own threads

[sand sierra]

👍

[leaden patio]

Do you have plans in dagger to abstract the different CI environement variables? Depending on whether it is Jenkins, gitlab-CI, github, etc... it is not always the same environement variable present during the build that encompass the same information. Is there a way to have a dagger abstraction on top of it? This dagger abstraction would then have different implementation in the different CI.

[sharp marsh]

@sand sierra doesn't happen to me :

marcos:home/marcos (⎈ |N/A)$ dagger core container from --address alpine file --path /etc/alpine-release export --path ./alpine-release
Full trace at https://dagger.cloud/marcos-test/traces/b20d197804d62a08ea15166362dfd376

✔ connect 1.0s
✔ initialize 0.2s
✔ prepare 0.0s
✔ container: Container! 0.0s
✔ Container.from(address: "alpine"): Container! 0.6s
✔ Container.file(path: "/etc/alpine-release"): File! 0.0s
✔ File.export(path: "./alpine-release"): String! 0.0s

/home/marcos/alpine-release
marcos:home/marcos (⎈ |N/A)$ ls -la ./alpine-release
-rw-r--r-- 1 marcos marcos 7 sep  5 09:53 ./alpine-release

[sharp marsh]

alpine-release is owned by root in the alpine contaner

[sick monolith]

ChatGPT 4o and my compass diagram

[sand sierra]

I will check this with your code.

You are exporting a file. I am exporting a directory

maybe that is the difference?

I am not on my desktop pc so can not test your solution just yet.