general

[swift inlet]

Yes exactly

[silent agate]

I can knock that out today

[swift inlet]

If you prefer to focus on your proposal, I can take a look ify

[silent agate]

That’d be helpful thanks!!

[autumn swallow]

Welcome @stoic turtle!

[stoic turtle]

Hello all

[winter linden]

welcome 🙂

[wispy tapir]

Nice new logo 😄

[silent agate]

Love it

[turbid tulip]

I love it, too!

[lyric notch]

the new logo is awesome 😍

[autumn swallow]

Welcome @split badge!

[swift inlet]

Hi @split badge,
We talked earlier (it's Guillaume). Feel free to explore and ask any question whatsoever 🙂

[split badge]

Hello there 🖐 I'll probably start to look at Dagger this week-end (or maybe next because Hacktoberfest is waiting for me for tomorrow ^^).

[autumn swallow]

Welcome @winged oyster!

[winged oyster]

o/ @turbid tulip

[turbid tulip]

Hey @winged oyster!

Welcome. 🙂

[autumn swallow]

Welcome @still wolf!

[undone summit]

Hey everyone, had chat with @winter linden few days back, I am trying to look at cue and would love to contribute in it. I am just getting started with the language. It'd be great if someone can suggest any issue or experiments regarding performance or anything else that I can do around it please let me know! 😄

[swift inlet]

Hi @undone summit,
Are you looking to contribute to Cue specifically ? Or Dagger in a bigger extent ? We can definitely help you find an issue fitting your requirements. What are you, more specifically, looking to work on ?
The current main performance issue around Cue is being tracked here: https://github.com/cue-lang/cue/issues/804.

[undone summit]

Hey @swift inlet , I think the target definitely would be latter one. I don't have any specific requirement right now. It's mostly for my learning purpose and contributing while doing so. So whatever might be a "good" start . I think the target for this week is just understanding Cue first, thus looking forward for suggestions.
Thank you the the link to the issue, I will go through this 🙂

[swift inlet]

Awesome. Thanks for your interest in Dagger 😄 Don't hesitate to ping us anytime. We have other issues that may be better fits for onboarding 😄 (encountered a segfault once recently, didn't manage to reciprocate/isolate it precisely, but we could explain you the context for example. Isolating and fixing it could be extremely valuable for example)

[silent agate]

Would you—or anyone else for that matter!—be interested in either free recorded videos or live community talks where you could ask questions? I'm considering both but don't have a timeline for any publications atm

[undone summit]

I think that sounds like a good one to look through. Let me ping you once I clear my schedule for this week 🙂

[undone summit]

I didn't quite understand, do you want to someone to attend these events and note the learnings (or answers to the questioners?) what are the videos/live community talk about?

[silent agate]

The videos and potentially live events would be about a mix of CUE and Dagger. Just curious who would be interested in attending as a way to learn

attending live talks or watching recorded video

[undone summit]

I think that is a very good idea, there are twitter spaces dedicated for even discussions (there is performance twitter space for instance)

If you do post on twitter it might get you an idea of how many people would be interested in attending in (twitch streaming is also a good way but you might know that already)

[silent agate]

Hmmm twitch could be interesting

[undone summit]

I definitely would be interested in attending. But I would prefer if it was live as we get to hear discussions. It's usually difficult to remember what event I wanted to watch (if I miss them) sometimes and unless its being directly used in my job I wouldn't remember to watch it so easily

but live events are usually a good way to get more audience

[winter linden]

We can start simple with a community call here on discord

That’s been on my todolist for months actually..

[silent agate]

I was about to suggest live youtube or twitch and read questions from the comments, but a discord community call is a great place to start and see what interest there is. I imagine interest being huge once Cue hits critical mass

VS Code finally gets bracket pair highlights. Been waiting for that forever

oh! and static terminal dimensions and custom tab titles! I swear VS Code continues to be the best thing to ever come out of Microsoft

[autumn swallow]

Welcome @tidal cape!

[tidal cape]

Hi everyone 👋 . Just wanted to say I'm looking forward to trying out Dagger. I've used Cue a bit in the past and worked with a couple people who are already here (good to see you here @silent agate and @turbid tulip )

[silent agate]

Welcome! Glad you joined us

[turbid tulip]

Hey @tidal cape! Good to see you!

[jolly plank]

@here We are looking for a dagger developer with strong devops experience. We are open to both contract or full-time employment.
Please DM me if interested.

[autumn swallow]

Welcome @shy wyvern!

[shy wyvern]

./wave

[autumn swallow]

Welcome @sleek mist!

Welcome @blissful ginkgo!

[west patrol]

I was wondering if this could have been in #🔭universe but @winter linden it seems that the 🥖 men have a serious competitor against docker 😂 https://github.com/brouberol/marcel

[serene thistle]

talentedmrjones

[dagger] New discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

The Issues

The current version of Dagger requires imperative CLI commands that complicate the use of the tool as well as forcing the user to do the very operations Dagger sets out to eliminate. The dagger input commands are a regular source of confusion, both for developers new to and experienced with CUE. Furthermore, Dagger has overloaded some concepts such as inputs and left other concepts such as environments and plans rather vague.

This proposal will describe a version ...

[silent agate]

Hi folks! Please take a look at this proposal and consider how it would improve your experience of Dagger and how it would enable easier onboarding and adoption. Feedback of any kind is greatly appreciated!

[silent agate]

ok now I have to learn French and Rust https://github.com/bnjbvr/rouille

[serene thistle]

shykes

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

I like the general direction, as you know.

A few questions and comments in no particular order. Not an exhaustive list, but wanted to get the conversation moving ASAP.

• Will I still be able to query the state of an environment with dagger query? If so, how is that state stored and attached to the proper environment?
• You use package dagger to mean 2 different things depending on context. Even in this proposal, there are 2 code snippets, each using package dagger in a different ...

[serene thistle]

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

Answering your points in order:

• yes via dagger query <environment>. The question of state is still outstanding. We may still need a .dagger folder for that purpose
• I agree a local package dagger coexisting with alpha.dagger.io/dagger is confusing. I feel its important to be able to colocate dagger .cue files with files used by other tools. Perhaps not? In order to do that CUE needs the package identifier. For the project package name to be arbitrary, we would need to configur...

[serene thistle]

shykes

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

Answering your points in order:

• yes via dagger query <environment>. The question of state is still outstanding. We may still need a .dagger folder for that purpose

OK. Could you add a section on this topic, if only to say “this point is still outstanding”?

• I agree a local package dagger coexisting with alpha.dagger.io/dagger is confusing. I feel its important to be able to colocate dagger .cue files with files used by other tools. Perhaps not? In order to do that ...

[serene thistle]

sdboyer

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

Just wanted to pop in and say, i'm glad to see a proposal like this! I (still 😢 ) haven't had much time to really grok Dagger's codebase, so don't have an informed opinion on Dagger's current architecture, but many of the basic principles here - e.g., eliminate special input paths, express as much of the logic as possible natively in CUE - are hard-won ideas i've come to over the past six months.

With polly, we gyrated for a couple weeks on what "passing in variables" should look like, unt...

[serene thistle]

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

Glad you weighed in @sdboyer . I'm glad to hear you echo my experience that expressing as much as possible natively in CUE is the way to go.

[turbid tulip]

@winter linden and @wraith niche were on Changelog's podcast Ship It! https://changelog.com/shipit/23

[serene thistle]

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

• edited proposal to speak to state management and continuing to use .dagger/<env>/state.json
• clarified the idea of a project entrypoint and listed other possibilities besides package dagger
• added section for host-dependent vs host-independent config

[serene thistle]

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

@shykes raises an interesting concern as it relates to the loading of CUE files as the proposal is written: It would be possible to invoke Dagger commands in a number of different ways.

• Single environment specified in the project root: may run dagger up without specifying an environment
• Single environment specified in the project root: may run dagger up <environment> so long as there is a match
• Multiple environments defined in the project root: must run `dagger up <envir...

[autumn swallow]

Welcome @sand heart!

Welcome @wooden kayak!

[sand heart]

👋

[wraith niche]

Welcome! If anyone has issues accessing the github repo, please let me know (you can pm your username).

[silent agate]

<@&707664746224025688> this feels better. In fact it makes me rethink how stax does things. Curious and eager to hear feedback as you have time

[autumn swallow]

Welcome @opal solar!

[opal solar]

👋

[serene thistle]

TomChv

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

I really like that approach but I think removing #Environment isn't a good idea.

The killer feature of [name=string]: #Environment was the possibility to dynamically create environments thanks to regexp (your example with PR was really cool btw).
If we decide to use directories as environments, how would you manage that?

Instead of removing #Environment, I think we should find a way to control how users can deploy their plans.

Dagger commands in a number of different ways.
...

[serene thistle]

slumbering

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

What we basically want to avoid (tell me if I'm wrong) is the case of multiple environments split between directories because to deploy it we will need to jump to the directory and then tip dagger up <environment>

If I understand right you can execute all variations with dagger up ./...

[serene thistle]

TomChv

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

Yead this is what @talentedmrjones explained in the main message

For advanced users it would be possible to do the following from the project root:

dagger up ./... and execute all variations. In this case each variation would require a unique name. It may also be possible to specify whether the variations execute sequentially or in parallel.

And I think this is not a good idea to allow that.

[serene thistle]

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

@TomChv can you elaborate on why you think it is not a good idea to allow the user to dagger up ./... and execute all variations?

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

Also @TomChv you're right! Moving dagger.#Environment keys up to the root removes the killer feature of matching dagger up <env> to a regexp. 🤦 I'll think through this more...

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

Re: Loading CUE Files:

Another option could be to always have the CUE API load from the project root no matter what directory you are in so long as you are in or below the project root so that executing dagger anywhere within a project always produces the same result (kind of like how git always finds .git at the root).

With this we could keep dagger.#Environment and the flexibility it provides, especially as it relates to regex matching.

If we load from the root with the _i...

TomChv

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

can you elaborate on why you think it is not a good idea to allow the user to dagger up ./... and execute all variations?

I think it could lead to unexpected behavior :disappointed_relieved:

Imagine you do dagger up ./... in a project with multiple local, production and staging environments, you don't have the control over your deployment.
Also how dagger should deploy?

• Each environment are up one by one or in parallel?
• What happens if an environment fails?

I know I'm de...

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

@TomChv excellent points. Especially if my host is not setup or authorized to deploy certain environments. It's common for devs to be able to dagger up local but not remote cloud-based environments

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

@TomChv actually, in this alternate approach, with the dagger.#Environment keys were made top-level keys, there is still the name: string field which could be written as a regex like this:

name: =~"review-[a-zA-Z0-9-_]*"

And dynamic PR review environments enabled with a flag such as --name which could be used to fill that path. So from the root: dagger up ./review --name pr-toms-feature. Without the --name flag Dagger would error with No environments found

[serene thistle]

TomChv

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

Okay that solve a problem, but I think we

[serene thistle]

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

Regarding version: string there have been questions regarding how this is used. I propose that this specifies the minimum Dagger version the plan is written for. Dagger will compare this field to its internal version, and return fatal error if dagger binary version is < plan version

shykes

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

1. I agree that we should only allow dagger up on one environment at a time. It’s one less thing to worry about in the design, and it doesn’t seem to be a big problem for users since we already have this restriction in place and nobody (to my knowledge) has complained about it.

2. I think this variation has promise. The issue with environment naming seems solvable.

3. I don’t think this variation requires eliminating dagger.#Environment. As a CUE developer, wouldn’t I expect to appl...

shykes

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

That could be useful, but feels like we could chop it off the proposal for now, in the name of scope control, and discuss later without difficulty. Wdyt?

[winter linden]

Apologies for the notification spam everyone. But if you look closely, it’s not actually spam 🙂 Each notification is for a post that would otherwise be made here. If the volume becomes too much, we’ll move it to #github-feed

[serene thistle]

shykes

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

Also:

5. In this variation, is a project just a cue module?

[winter linden]

If anyone happens to be in the Bay Area and free tonight, I’ll be at Render Oakland. Come say hi and ask me about Dagger 🙂 https://www.meetup.com/renderoak/

cc @grim dawn @fallow bloom

[grim dawn]

im there 🎉🎉

[serene thistle]

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

1. I agree. Execute a single environment at a time.
2. yep
3. The schema for dagger.#Environment can be applied to top-level keys in several different ways. That's an implementation detail to solve later. The point for now being that this pattern still allows cue eval to catch errors. Another important consideration is that environments: [string]: dagger.#Environment allows multiple environments to be defined in each cue value instance and I see a lot of room for trouble there. It se...

shykes

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

4. In dagger up ./review --name pr-toms-feature ./review would be where to load CUE files and retrieve a single value instance (plan variation) which in this example does not define a concrete name but rather a regex pattern; pr-toms-feature would be used to value.Fill() which would return a concrete value for the name (assuming the string provided matched the regex)

I understand how it would work but was wondering about terminology, and specifically user-facing terminology. I...

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

It's not two types of objects (plan and environment). In this alternate approach the plan variation is the environment. Which is one object. What's happening is that the plan as an incomplete value is being completed by the provided string.

I would explain it differently:

• a plan which is the CUE configuration loaded recursively from the specified directory to the project root
• a concrete value used to complete the plan'sname field

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

I think it's simple enough to implement, and useful enough to authors that we should leave it. Especially considering that at this stage we're likely to change the API several times before official v1.

shykes

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

It's not two types of objects (plan and environment). In this alternate approach the plan variation is the environment. Which is one object. What's happening is that the plan as an incomplete value is being completed by the provided string.

I would explain it differently:

• a plan which is the CUE configuration loaded recursively from the specified directory to the project root
• a concrete value used to complete the plan'sname field

In this scenario, wouldn’t th...

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

🤦 I misread this the first time. I would refine the explanation like this (because one thing is an object, the other thing is a string)

It defines 2 things: a CUE plan and a name for the plan

• a PLAN which is a Cue configuration loaded from the specified directory recursively to the project root
• a name for the plan to use

Each plan is its own operational context / configuration profile.

shykes

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

OK, I was hoping to dodge yet another debate ;) Unfortunately I disagree that we need such a field at this time, sorry.

• At the moment there is no backwards-incompatible format to support, so developers don’t actually need this field
• If in the future we do introduce a backwards-incompatible format change (which could be in a distant future or even never, depending on how good / lucky we are), we can always introduce the field then. There is no downside to waiting. No need to burden dev...

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

Just to be clear. My thought in this alternative is that an "environment" is not a discreet entity. There is only the plan. And variations of the plan (via sub directories) allows authors to refine plans for multiple environments

[autumn swallow]

Welcome @ebon cargo!

[ebon cargo]

Hello all, Guillaume Bizet from societe général.

[wraith niche]

Welcome!

[autumn swallow]

Welcome @tender sedge!

[silent agate]

<@&707664746224025688> proposal body updated according to our last discussion: https://github.com/dagger/dagger/discussions/1058

should I remove some of the comments that are no longer quite relevant?

[winter linden]

There’s a “hide” feature for that purpose, works well, go for it

[serene thistle]

aakarim

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

Our team would be very happy with a CUE native way of expressing inputs. We do this already anyway in non-Dagger projects - we have a root env.schema.cue file that is copied to env.cue as step 1 of a setting up a dev machine. After being replaced/refined with concrete values from the developer, the schema file validates the env file since it's in the same package. Other configuration packages import this env package and use it to load in secrets etc.

To manage secrets we also have an ...

[serene thistle]

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

In this scenario, wouldn’t there be a state file named.dagger/pr-toms-feature/state.json ? Wouldn’t that file contain the state of an environment called pr-toms-feature? How exactly would you define an environment?

😄 I had the same thought later actually. That regardless of where we store the state file, it really represents the environment, there for the plan that created it represents the environment. I've edited the proposal to include this.

[serene thistle]

shykes

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

Let’s discuss a dimension that is crucially important to me: code reuse.

A major problem we want to solve with Dagger is the lack of a high-quality ecosystem of reusable CICD code. When I write a new application in Go, Python or Typescript, I can leverage the work of thousands of people via the corresponding package ecosystem. I CICD there is no such ecosystem. Dagger has the ability to change that, and we must design with that priority in mind.

To pick a counter-example: Docker Compo...

[autumn swallow]

Welcome @vast jasper!

[undone summit]

Hi, I went through the doc (got a little late due my daytime job) and I have few question. In "General Evaluation", has anyone been working on Keeping sorted list of arcs and/or Reuse of computation. Second, would this be a good issue to pick up for performance enhancement in Cue?

[swift inlet]

Hi @undone summit,
Nice, no worries ahah, we totally understand 😄 I'm not sure to fully understand the meaning of arcs here, can you please detail ?

[undone summit]

I was hoping to know it from y'all 😅 In following link
https://github.com/cue-lang/cue/issues/804
Under General Evaluation, the OP is writing about merging the sorted list so label matching can be done in < O(n) time? I am assuming by arc here they are talking about some kind of data structure?

[silent agate]

@winter linden besides the ability to script around it I’m curious what you think of the security model Deno uses https://deno.land/manual@v1.15.2/getting_started/permissions#permissions-list

[winter linden]

Will check it out, thanks!

[silent agate]

If we implemented an explicit permissions model we could write that in CUE rather than cli args which would make it harder to script around

(in cue as a separate, isolated, hermetic scope that does not import anything)

[winter linden]

Seems like a good fit for platforms like Deno that have no choice but to support posix because that’s what millions of JS developers expect. Their only choice is to bolt ACLs on top of APIs they don’t control. We have the luxury of breaking the dependency on posix because cloud engineers/SREs (our developer demographic) have been trained to expect that their pipelines will run in a distributed system, and that if they need to run posix-dependent code, they can always fire up a container.

TLDR we can do better because we have more advantageous constraints than Deno

[silent agate]

Sorry if I'm being dense, but I don't understand how posix or a dependency on it relates to the issue of preventing reusable packages from accessing host resources, or a Dagger api that uses permissions or indirection of some kind. Can you help? I generally need more low-level mechanical examples if you can

[winter linden]

Host access APIs

[winter linden]

Cue has been featured all day on the first page of hacker news 🙂 https://news.ycombinator.com/item?id=28915655

viebel

Cue: A new language for data validation

Hacker News · Oct 19, 2021, 7:21 AM

[autumn swallow]

Welcome @sacred chasm!

[winter linden]

@silent agate Starting a thread on buildkit resources 🙂 If anyone has good resources, please share!

[autumn swallow]

Welcome @oak estuary!

[swift inlet]

I'll take a look tomorrow, if you're ok with it 😄

[autumn swallow]

Welcome @timid epoch!

[undone summit]

Thanks, I did get an answer on cue channel regarding this.

it's an []* Vertex type

but I'd still need to understand the purpose of arc if I go with first performance issue (using Merge sort and matching the label in < O(n) time)

[undone summit]

which brings me to this question, should I go with issues under General Evaluation as the first things I'd be solving? Or if you think something else would be a better issue to pick/ more easy or /important

[autumn swallow]

Welcome @river rock!

[wraith niche]

Welcome Harsh!

[river rock]

Glad to finally have hands on Dagger 😄

[wraith niche]

do you mean on the dagger/dagger github repo? Can you elaborate on the use case you have in mind?

[undone summit]

@wraith niche sure, I meant probably a performance improvement on cue repository (which could help dagger in some manner?) I haven't picked up any issue as of yet. But @swift inlet directed me to future roadmap of cue in performance issue. And I want to find a good first issue to start with. Does that explain your question?

[silent agate]

CUE town hall starts in 18m! https://www.youtube.com/watch?v=GMTQcm3AwOU

[silent agate]

I just learned that you can't close the root level struct. Even though you can enforce the types for root struct keys, you cannot prevent other keys from being added. 😬

[turbid tulip]

@winter linden @silent agate Marcel talks about the Value lattice in more depth here: https://www.youtube.com/watch?v=jSRXobu1jHk

~18min

[winter linden]

Looking at the answers to registration forms, it looks like “parity between CI and local development” is a big pain point for a lot of people…

[turbid tulip]

@winter linden I like hearing that. 🙂

I've seen stuff like this to attempt to alleviate it: https://github.com/nektos/act

[swift inlet]

Downloaded the full image once, 20go+ 🤣

[turbid tulip]

Umm... what?! That's insane.

[winter linden]

Reminds me of openstack as an attempt to solve cross-cloud VM portability. In the end it’s just a bandaid

Actually this reminds me of Eucalyptus

Early attempt at an API-compatible open-source AWS clone

[swift inlet]

Sorry, exagerated a little. But If I remember well, I had to extend 2 times the Docker disk image size

[turbid tulip]

Interesting. I hadn't heard of that. That sounds like an enormous errort.

[winter linden]

Yes it failed

10 years ago

[turbid tulip]

Still rather large.

[turbid tulip]

I'm not surprised.

[winter linden]

It appears we are doing for CICD infrastructure what Docker did to app hosting infrastructure 🙂

[turbid tulip]

Yeah, I definitely see the parallels.

Not too surprising, given what you, @wraith niche, and @cloud canyon did at Docker. 🙂

[winter linden]

What’s funny is that the infrastructure water line has moved up since then

[turbid tulip]

In part because of Docker and containers, I’d say.

And systems and tools built around them.

[winter linden]

Yes exactly. Those are infrastructure now. Same problems have moved up.

[turbid tulip]

I’m very much looking forward to building amazing, solid, and valuable tooling around making that easier with you all.

[serene thistle]

aluzzardi

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

I feel like @talentedmrjones's proposal addressed the security concern of host local access by performing static analysis, preventing imported packages from accessing host resources.

As for package re-usability, I feel like that's the job of the package author. We can provide security guarantees but we cannot guarantee re-usability without collaboration. As in, a layer of indirection won't necessarily make a package reusable if it wasn't designed this way.

A package can hard...

[serene thistle]

shykes

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

I feel like @talentedmrjones's proposal addressed the security concern of host local access by performing static analysis, preventing imported packages from accessing host resources.

As for package re-usability, I feel like that's the job of the package author. We can provide security guarantees but we cannot guarantee re-usability without collaboration. As in, a layer of indirection won't necessarily make a package reusable if it wasn't designed this way.

A packag...

[serene thistle]

aluzzardi

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

That doesn’t mean we shouldn’t take every opportunity to make it as easy as possible to write secure and portable code for Dagger. Otherwise, why bother to make any improvement to security or portability at all?

I see the indirection file as an alternative UI for achieving the same result.

I'm failing to imagine a configuration where the indirection makes a package more portable than forbidding packages to access local resources (i.e. the original proposal).

It's a trade-off in UI....

[serene thistle]

shykes

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

I'm failing to imagine a configuration where the indirection makes a package more portable than forbidding packages to access local resources (i.e. the original proposal).

It’s a developer experience issue. Here’s a trivial example:

Good snippet:

// My custom transformation pipeline
#Transform: {
  // Input directory
  input: dagger.#Directory
  // Output directory
  output: dagger.#Directory
}

Bad snippet:

// My custom transformation pipeline
#Transform...

[serene thistle]

aluzzardi

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

It’s a developer experience issue. Here’s a trivial example:

100%. Either approaches are as secure/portable, the only difference is in DX.

Hopefully we agree that the first snippet is good, and the second is bad.

Yes

Where we disagree is on whether the “bad snippet” should be allowed by our API at all. I say no, and you say yes.

It's a trade-off between DX and UX. Yes, it's going to be a nuisance to learn there's a field which you cannot set in packages. To compensate, we...

aluzzardi

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

Thinking some more, perhaps an alternative would be to copy the buildkit API, something like this:

{
  plan: #Transformation & {
    input: id: "source"
    output: remote: "https://github.com/aluzzardi/fork"
  }
  local: source: "./my-test-input"
}

However we'd end up in the same situation -- there is no scenario where it is useful for the developer to specify a value for id

shykes

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

It’s a developer experience issue. Here’s a trivial example:

100%. Either approaches are as secure/portable, the only difference is in DX.

Hopefully we agree that the first snippet is good, and the second is bad.

Yes

Where we disagree is on whether the “bad snippet” should be allowed by our API at all. I say no, and you say yes.

It's a trade-off between DX and UX. Yes, it's going to be a nuisance to learn there's a field which you cannot set in packa...

[silent agate]

👀 Lots of great input here!

I think I'm on to something.... hope to have something more concrete to show tomorrow or monday latest

[serene thistle]

shykes

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

100%. Either approaches are as secure/portable, the only difference is in DX.

I disagree, but understand why you think that: you are assuming that all solutions (with or without indirection) eventually result in all information being 1) merged in a single CUE lattice, 2) from files all living in the same git repo.

I actually think the mapping of client resources should be defined in a client-specific configuration. That configuration should not be merged in the same CUE lattice as the...

[serene thistle]

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

For clarity I think we need to speak in terms of a "project author" and a the "package author".
The package author is writing a package which could could be a single component such as yarn.#Package or a custom dagger.#Plan as an entire collection of resources, and this package is intended to be imported into a project.

The project author is the engineer writing a plan to deploy their application.

Package authors should generally avoid (and be prevented) from referencing host resources....

[serene thistle]

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

I can imagine in dynamic scenarios, especially in CI pipelines (circleci, gitlab, etc) that the contents of the sandbox scope might be dynamically generated. So maybe the sandbox scope could include json and yaml as well as cue, and even unify multiple files. Perhaps those are specified as [...string] in the plan? Or by command line flag. Or both and the CLI flag overwrites the plan

shykes

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

For clarity I think we need to speak in terms of a "project author" and a the "package author". The package author is writing a package which could could be a single component such as yarn.#Package or a custom dagger.#Plan as an entire collection of resources, and this package is intended to be imported into a project.

The project author is the engineer writing a plan to deploy their application.

Package authors should generally avoid (and be prevented) from referencing host r...

[serene thistle]

talentedmrjones

[dagger/dagger] New comment on discussion #1058: PROPOSAL: An Easier\-to\-Use, More Flexible Dagger

I don't mean to say or imply that we should determine how people break up packages. That's not at all what I'm saying. I wholeheartedly agree it should be up to them.

And I should have been more clear that I'm speaking to a persons intention and perspective more than package organization or structure. If I intend for my package to be imported then I'm authoring a package and I have to keep in mind the implications and consequences of that package's scope. If I'm writing code specific t...

[mystic mantle]

We're just building abstractions on top of existing ones.
You just need to believe the previous one is already good enough ^^

[autumn swallow]

Welcome @fair scaffold!

[nimble stone]

Hi everyone, I just ran brew outdated and I get a small warning:

❯ brew outdated
Warning: Calling bottle :unneeded is deprecated! There is no replacement.
Please report this issue to the dagger/tap tap (not Homebrew/brew or Homebrew/core):
  /opt/homebrew/Library/Taps/dagger/homebrew-tap/dagger.rb:9

it's just a warning, I don't think it's massively important but there it is 🙂

[wraith niche]

thanks @nimble stone, we'll look into it

[wraith niche]

it was solved with the latest goreleaser, so this warning will go away with the next dagger release.

[autumn swallow]

Welcome @weary mantle!

[weary mantle]

Hi

[autumn swallow]

Welcome @terse tinsel!

[terse tinsel]

Hi

[winter linden]

Welcome 🙂

[weary mantle]

Thanks @winter linden

[autumn swallow]

Welcome @stone jay!

[stone jay]

Hello there 🙂

[autumn swallow]

Welcome @blissful grove!

[blissful grove]

Hey all 👋
Thanks for the invite.

[autumn swallow]

Welcome @fringe marlin!

[fringe marlin]

Hi all, thanks for opening the door. Curious to poke around, that Dagger episode of the Ship It podcast rocked 🙂

[winter linden]

Thanks 🙂 Lots of improvements underway, make sure to let us know if you encounter an issue - it’s likely to be fixed in the coming weeks

[zinc coral]

Hello there, I'm currently doing research to make a presentation of what is Dagger and why it's awesome and because of my young experience in IT, I would like to know if somebody can give me a feedback of what changed in your project development with Dagger to complete my own experience ?

[zinc coral]

Don't hesitate to DM me 😁

[winter linden]

@zinc coral that’s great!when is your presentation due? We do a lot of user interviews, we could mention your presentation and aak of they’re willing to share some of their feedback with you. Is it for a school project, or work?

[zinc coral]

it's for my school

I don't have the date atm, I will choose one when everything will be ready

[winter linden]

In case anyone has an opinion 🙂 https://twitter.com/solomonstre/status/1455319894749155332

Solomon Hykes (@solomonstre)

If you could make one problem go away in your CICD setup, what would it be? (round 1)

Twitter · Nov 1, 2021, 11:44 PM

[autumn swallow]

Welcome @wind matrix!

[autumn swallow]

Welcome @oak sand!

[autumn swallow]

Welcome @silent moon!

[silent moon]

👋 - hi folks, excited to take Dagger for a spin 🙂

[silent moon]

Not actively on twitter but my vote would be for Reusable components, I appreciate the massive ecosystem of GitHub actions, but having that available in a statically typed (this is what drew me to Dagger in the first place, down with YAML configs - its already happening for IaC, look at the CDK and Pulumi), run from anywhere interface would be amazing.

[winter linden]

Welcome @silent moon , abd appreciate the perspective 🙂

Note there has been discussion of 2-way compat betwen Dagger and Github Actions

[silent moon]

meaning that every GitHub actions author wouldn't have to re-write in Cue for us to take advantage of them using Dagger (if this happens)?

[winter linden]

Correct you could wrap an existing GH action into a Dagger action with minimal effort. You would still write the wrapper in Cue but could be a very easy path

At least that’s the theory

And in any case you can write Dagger actions in any language. Shell python typescript etc. Cue is the declarative wrapper that allows cross-language composition into a DAG

[autumn swallow]

Welcome @raw crescent!

[raw crescent]

Hello everyone!

Just landed here 🙂

[winter linden]

Welcome 🙂

[autumn swallow]

Welcome @quartz pumice!

[quartz pumice]

Hi everyone just landed here from down under

[winter linden]

Make yourself at home, if you get stuck or confused please let us know. The more feedback you send us, the faster we can improve Dagger and launch it into the world 🙂

[patent stirrup]

Hello to the person from Twitter who wants to discuss SBOMs

Feel free to ping me here or send me a pm

[winter linden]

cc @meager stream 🙂

[patent stirrup]

I may be a bit slow to respond, feeling a bit sick today

[meager stream]

Thanks @winter linden for pointing me in this direction. And hello @patent stirrup! Hope you get feeling better. I’m interested in learning about SBOMs and where things are related to them. I’ve been experimenting/playing around with some graph database action (Neo4j) that graphs the relationships between container image layers and tags. I’m also looking at adding nodes to represent what’s running in our k8s clusters. By adding in SBOM nodes to the graph, I’m interested in the ability to say “dependency X has problems, what images running in our cluster needs to be updated?” Obviously, to do that, I need to know what’s baked in. I’ve seen SBOMs mentioned over the years, but haven’t updated myself in a while. So… here I am! Where can I get started? What work is being done? Etc. etc.

[patent stirrup]

How much time do you want to spend on this topic? I can put together a playlist of things to watch from kubecon. Alternatively, I can have a talk with you tomorrow and just fill you in on context and the spaces you want to focus on.

I need to write about this stuff, so either way this will help me too

[winter linden]

SBOM

[autumn swallow]

Welcome @final peak!

[autumn swallow]

Welcome @inland smelt!

[autumn swallow]

Welcome @green spire!

[autumn swallow]

Welcome @last vector!

[autumn swallow]

Welcome @slender orbit!

[autumn swallow]

Welcome @tropic crown!

[winter linden]

Welcome everyone!

[turbid tulip]

Welcome, @final peak @inland smelt @green spire @last vector @slender orbit @tropic crown! Let us know if you have any questions, feedback, ideas, etc.

[autumn swallow]

Welcome @patent yacht!

[autumn swallow]

Welcome @swift barn!

Welcome @unborn tusk!

[autumn swallow]

Welcome @hardy plaza!

[autumn swallow]

Welcome @vast sand!

[vast sand]

Hey everyone 🙂 Thanks for the alpha invite, excited to try Dagger with my company!

[winter linden]

FYI, now there are more active developers working on Dagger, we’re experimenting with an open audio channel to have more design discussions in the open. If you see people in #dev-audio , feel free to join. Please mute yourselves to avoid disrupting conversations. But if you do want to participate in the topic, you are welcome to. If it gets too crowded, we can always adjust the tooling (moderation etc)

[silent agate]

Anyone else old enough to remember dial-up bulletin boards (BBS) before internet was widely available? I was a daily visitor to one throughout the 90s. It was a magical time of text-based graphics! Anywho I'm looking to use Dagger to deploy MajorBBS to EC2 so I can host a retro BBS and play some of those old multiplayer text games

[turbid tulip]

@silent agate You're old. That was before my time.

😉

[silent agate]

You missed out!

[turbid tulip]

I've heard it was a fun time. 🙂

[silent agate]

You'll have to join me for Cybertank and Trade Wars

[turbid tulip]

I'd do that!

[silent agate]

perhaps I'll host it in my closet with my gigabit fiber 😜

[turbid tulip]

Get an RPi 🙂

[autumn swallow]

Welcome @hollow moat!

[autumn swallow]

Welcome @fossil pine!

[autumn swallow]

Welcome @dense shell!

[autumn swallow]

Welcome @thorny juniper!

[thorny juniper]

Hello, excited after hearing the ChangeLog podcast

[winter linden]

Welcome 🙂

[thorny juniper]

Upon first glance, it seems similar to what Waypoint is trying achieve except its way more configurable and declarative using CUE where as Waypoint is limited to its plugins and supported features like Terraform. Am I way off?

[winter linden]

Not too far off. One big difference with is that Waypoint is specialized deployment tool. Dagger is a framework for developing any kind of CICD pipeline.

So you can implement any deployment logic you want on top of Dagger - including invoking Waypoint itself, or replicating its features. But the opposite is not true.

[wraith niche]

🚀 dagger v0.1.0-alpha.29 is out!

Release highlights:

• Support different platforms (OS / Architecture) when creating a new Environment
• Windows is now 100% on-par with MacOS / Linux (was missing support for input dir and input stream)
• Universe: adds support for Trivy (vulnerability scanner for container images, file systems and git repositories)
• Universe: docker package can now auto-replace local containers
• Universe: supports AWS CLI v2
• Other various improvements and reliability fixes

Complete changelog: https://github.com/dagger/dagger/releases/tag/v0.1.0-alpha.28

Install & Upgrade instructions: https://docs.dagger.io/1001/install/

As usual, if you need any help or spot any bug, please let us know.

[autumn swallow]

Welcome @lost prawn!

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @tall quiver!

[autumn swallow]

Welcome @eternal rain!

[wraith niche]

Welcome everyone! (Pascal 👋)

[wraith niche]

⚠️ Erratum from yesterday release: we just released dagger v0.1.0-alpha.30 to address a fix for the input socket on Windows. If you had issues trying the new release on Windows, please upgrade to this latest one. Other users will be unaffected.

[wraith niche]

Thanks @weary mantle for the bug report.

[autumn swallow]

Welcome @maiden cloud!

[autumn swallow]

Welcome @clear gorge!

[wraith niche]

📺 In case you prefer videos over written docs, here is a fresh one from @silent agate walking you through the Dagger tutorial:

https://www.youtube.com/watch?v=QvyB3m9Fti0

This should give you what you need in less than 20 minutes, so you can write your own Dagger plan. If not, please don't hesitate to ask question on #help-old-do-not-post!

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @swift barn!

[rain oriole]

I gotta say… I’ve enjoyed taking Dagger for a spin.

When I first saw Solomon at DockerCon 2021 announcing Dagger, at first I didn’t quite get what Dagger was, but I also tried Docker at first a couple of months after it was announced until today. I remember the learning curve, and how it got better with time. It’s such a massive part of my day to day, so with this new project… I was interested.

Even though there were some pain points, I find that most of that is being addressed. I saw CUE some time ago, but didn’t see how I’d want to use it. Dagger brought CUE to life for me and now I feel itchy to migrate when I’m using something else (YAML, Makefile, bash scripts, Helm charts!!….).

The team has been really helpful here. I’m grateful for all your work and I love to see you guys discuss things, even though I don’t understand all of it. Can’t wait to try the new Europa, and see Dagger get better and more useful with time. I’m looking forward to being able to adopt it company-wide. It has so much potential. 🙂

[winter linden]

What a nice way to start the day! Thanks @rain oriole 🙂

Yes I agree it is reminiscent of the early days of Docker… Kind of different from everything else, so it takes time to figure out when and how to use it. Personally I can feel the picture getting clearer over the last few months.

[turbid tulip]

Thanks, @rain oriole! Glad you're so excited about Dagger! And I'm glad that it brought CUE to life for you. It's... an amazing config language.

[autumn swallow]

Welcome @sacred mango!

[winter linden]

You made it @sacred mango 🙂

[sacred mango]

Thanks! Took some email digging! 😅

A bit about myself for the amazing team here.
I'm Eyal Mor, a developer based out of Tel-Aviv/Israel.

Currently working for Autodesk as a SRE/DevOps/Platform/Fullstack engineer (I can't seem to choose what I like!)

I heard of dagger through a podcast, and it immediately hit the right notes!

Incidentally, my team has been developing a similar (albeit powered down) tool similar to Dagger.
Dagger has made me wonder if we can contribute our time into a community that is aiming to solve a problem which everyone is dealing with today.
Super excited to get started with dagger, this project is what i've been dreaming of for a long time 😄

[winter linden]

That’s exciting! Yes I noticed that too, a lot of teams forced to develop their own in-house solution… I would be interested to learn more about your in-house tool, and of course how Dagger can help you 🙂 Anything we can do to help, let us know!

[autumn swallow]

Welcome @shell sparrow!

[sacred mango]

I will be happy to share details!
Let me get back with a more detailed overview, and how I think Dagger can help us (and how we can contribute to Dagger)

Any specific channel I should be checking out? Or posting in?

[winter linden]

• I’ll DM you to schedule a video call (if that’s OK) to ask you questions about your setup
• Anything related to your use of Dagger, you can use #general, #help-old-do-not-post, github issues, github discussions as you see fit
• In #dev we discuss the internals of Dagger, future improvements, possible contributions etc

[rain oriole]

How cool is this? 😃

$ alias dc="cue export compose.cue --out yaml | docker compose -f /dev/stdin"
$ dc up -d

[winter linden]

lol 🙂

nice

[rain oriole]

Still thinking how I can integrate it with dagger. The best I can think of is to have dagger save a docker-compose.yaml on the host (warning that it's generated and shouldn't be edited), and use docker compose normally. I want to use dagger to build the image, and CUE to abstract/reuse that yaml, but I still need docker compose to run its commands.

[winter linden]

Check out my Dockercon demo, there’s a docker compose integration in there

You don’t need to generate anything 🙂

[rain oriole]

I saw that, but you're doing it the other way around. Reading the compose into dagger. What I want is to reduce duplication in writing the same compose in lots of websites, so I want to use dagger/cue to write it instead.

[winter linden]

The most promising workflow IMO:

• Keep docker-compose.yaml as is, developers can continue using it AND modifying it without having to learn anything new
• In your Dagger plan, load docker-compose.yaml on the fly, then use its contents to do a variety of things:
• You can call docker-compose up from Dagger of course - even on the exact docker-compose.yaml, or after tweaking it on the fly
• You can extract parts of the docker-compose.yaml to generate other configs, for example Kub resources, ECS manifest etc. This gives you a continuous compat bridge between docker-compose and other container deployment services

[winter linden]

I see. Yeah usually the bottleneck is getting people to adopt new tools, and hard switch to new formats. If you don’t have that problem, then you can do it the other way around

[cloud canyon]

Check out the docker.#Compose package:

https://github.com/dagger/dagger/blob/main/stdlib/docker/compose/compose.cue

[winter linden]

Just make sure you don’t actually have that problem 😉 Getting people to switch to new tools is very hard, unless you can force them

[rain oriole]

Yeah, it's not a problem.

[winter linden]

Then I guess the ideal workflow for you is like your alias dc, but in Dagger 🙂

[cloud canyon]

this is roughly:

mycompose: {
   services: [... compose-y stuff ]
}

docker.#Compose & {
  // cue export --out yaml equivalent
  source: yaml.Marshal(mycompose)
  [...]
}

[rain oriole]

How do I write that source to the host system?

[cloud canyon]

oh I thought in the dc alias you were just piping it to compose, not writing it?

bah uhm

[rain oriole]

Maybe dagger query source | docker compose -f /dev/stdin?

Didn't think of that

[winter linden]

In Europa there is a context.export 🙂

So you can write to local directories

coming soon ™️

[cloud canyon]

dagger query compose.source -f yaml > docker-compose.yaml

yeah

[cloud canyon]

err

-f raw

it's already yaml

[rain oriole]

No, I didn't know how to actually pipe until I remembered query, so piping query to docker compose is great!

I'm excited about this, it's been on my mind for a while! I've told @wraith niche today that my biggest pain is having lots of websites with lots of duplicated Dockerfile, compose.yaml and Jenkinsfile. Which is a nightmare to update when something changes. And they do!

Our developers don't actually mess with any of those files. They just use how I set them.

[winter linden]

In your case, developers would keep using docker-compose up, just with a generater compose file instead of manually edited one. Right?

[rain oriole]

Yep

Which allows me to update those configs more easily across the board.

I'm talking over 150+ websites.

[rain oriole]

It works! dagger -e compose query source -f text | dc -f /dev/stdin --project-directory .

Easier to reuse without anchors.

[autumn swallow]

Welcome @storm zinc!

[autumn swallow]

Welcome @winged falcon!

[silk apex]

https://blog.cedriccharly.com/post/20210523-how-cue-wins/

Et very good read about cue

A bit older to mention dagger sadly

[turbid tulip]

@silk apex @muted gale is great! I've enjoyed having him as an advocate for CUE.

[autumn swallow]

Welcome @fossil swift!

[winter linden]

Experimenting with live coding on #dev-audio, and the new “events” feature. Let’s see if it’s helpful!

[autumn swallow]

Welcome @swift barn!

[kindred flume]

Should docs.dagger.io repo be available to edit? I'm working through the docs today and spotted some typos

[turbid tulip]

@kindred flume The docs are all here: https://github.com/dagger/dagger/tree/main/docs

[winter linden]

Thanks for spotting them @kindred flume !

[turbid tulip]

Yes, thanks!

[kindred flume]

Happy to do a little to help FYI, that YT video walkthrough finally motivated me to dive in today.

[winter linden]

That will make @silent agate happy 🙂

[turbid tulip]

@kindred flume Really glad to hear that. 🙂 That will definitely make @silent agate happy. 🙂

Let us know if you need any help, want to talk through Dagger or CUE or Buildkit.

I can't claim expertise on Dagger or Buildkit yet, but my CUE is pretty solid.

[kindred flume]

Thanks! I'm working through the Cue tutorial now. This tool alone is very cool. Never heard of it before the Dagger docs.

also, is the Cue Playground broken for others too, or is it just me?

not hard to run local, but the dagger docs link heavily to it

[turbid tulip]

Appears to be broken to me. I'm getting a WASM error:
Uncaught (in promise) LinkError: WebAssembly.instantiate(): Import #5 module="go" function="runtime.walltime" error: function import requires a callable

[kindred flume]

ok, that's the same error i'm getting

[turbid tulip]

https://github.com/cue-lang/cue/issues/1392

[autumn swallow]

Welcome @sterile canyon!

[kindred flume]

Ah, definitely not just me then 🙂

@turbid tulip Do i need to worry about that DCO error in the docs PR?

[turbid tulip]

@kindred flume I think that is necessary, yes.

If you click into the details, it has some steps you can walk through: https://github.com/dagger/dagger/pull/1154/checks?check_run_id=4269938661

<@&785926252749651978> Anything else @kindred flume should know? (this is my first time bumping into the DCO)

[kindred flume]

ok cool, i'll take care of it now

[turbid tulip]

👍🏻

[kindred flume]

all fixed up, thanks!

[autumn swallow]

Welcome @cold ravine!

[autumn swallow]

Welcome @bold fox!

[autumn swallow]

Welcome @robust canopy!

[robust canopy]

Hi there everybody! I just got the invite to early access!

Looking forward to give it a try and contribute, to best I can! 🙂

[winter linden]

Welcome 🙂

[winter linden]

Thanks @heavy gazelle for promoting Dagger 🙂 That was such a fun episode, I hope we get to record a new one together soon. Perhaps Europa release will be the excuse. We can live code something together maybe.

https://twitter.com/shipitfm/status/1463168538651340800

The "Ship It!" Podcast (@ShipItFM)

💡 maybe it's time to think more deeply about everything that Buildkit offers...

"Everybody else uses Buildkit to build. Dagger is using Buildkit for much more than build."

🗣 @solomonstre on https://t.co/QwdjgyCjwK https://t.co/DrG2vCgArL

▶ Play video

Twitter · Nov 23, 2021, 3:32 PM

[heavy gazelle]

I really like your thinking! Europa is another great reason to do a follow-up. Once the Christmas episode with @turbid tulip & @swift inlet is in the bag, let's start on this one. January is a great month for new beginnings 😉

[autumn swallow]

Welcome @barren sentinel!

Welcome @blazing lion!

[autumn swallow]

Welcome @meager fiber!

[blazing lion]

This is what brought me here

[autumn swallow]

Welcome @stoic bay!

[autumn swallow]

Welcome @sly jay!

[autumn swallow]

Welcome @torpid wyvern!

[turbid tulip]

Welcome @blazing lion @meager fiber @sly jay @torpid wyvern !

[autumn swallow]

Welcome @untold silo!

Welcome @winged wren!

[winged wren]

hey 👋

Just read What is Dagger? and wow, this is really exciting

[turbid tulip]

@winged wren I'm glad to hear that! Anything specific that you're excited about?

[winged wren]

• Cue is brilliant, I'm definetely going to look into using it to replace a JSON DSL we use.
• Replacing my Makefiles + docker-compose + gitlab-ci dev/test/ci approach that tries to achieve "dev/ci parity".
• ... but also then merge/include the actual deployments to remote/prod/etc
• a universal deployment engine why the "focus" on "deployment"? It seems capable/applicable to way more

Can I start trying/using it at work here? Share with teammates? How should I approach this :p

[winter linden]

We’ve been iterating on how best to position Dagger… It’s very flexible so you can’t cover everything it can do. You need an initial focus. “deployment” is one possible way to narrow it down

@winged wren check out dagger.io it’s a newer iteration of the positioning

Dagger is like lego. You stay because of the infinite possibilities. But you started because one day you saw a pirate ship in the store 🙂 We need to define our pirate ship.

[winter linden]

Yes absolutely. If you want to invite teammates, have them sign up and DM their usernames, we’ll fast track

[winged wren]

Any plans to integrate in some way with https://temporal.io ?

[winged wren]

Thanks!

[turbid tulip]

I'm not aware of any current plans. Curious, though, how you'd like an integration to work.

[winter linden]

Then you can contribute universe.dagger.io/temporal 🙂

[winged wren]

Hm, not sure. Maybe "dagger remote execution" backed by a temporal workflow ... to combine the benefits of both tools. What for, idk yet

[winter linden]

TBH I had never heard of temporal before, but generally speaking: if there’s a valid use case for integrating something with Dagger, we want to make it work 🙂

FYI I’ll be livecoding the upcoming Europa stdlib at 3:30PM pacific (in one hour). Bring questions and use cases, and let’s see how well they work in Europa! If it breaks, we can fix it together.

https://discord.gg/mWrVPmYE?event=916094219515478156

[winter linden]

Thanks everyone for the great feedback & questions! I’ll schedule another one soon. Europa here we come!

[autumn swallow]

Welcome @sterile tartan!

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @glacial cosmos!

[autumn swallow]

Welcome @dim sandal!

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @dreamy plover!

[winter linden]

Doing a livecoding & Q&A sessionon Europa release, later today: https://discord.gg/mWrVPmYE?event=917483021827993630

One topic I’d like to address in that session is runtime parameters, and condition execution. Starting from your use cases @rain oriole @winged wren @blissful bone 🙂

[rain oriole]

That’s 9:15 PM for me, I’ll be putting my kid to bed. I’ll join when I can.

[winter linden]

I can relate 🙂

[winged wren]

I might be able to join 🙂

[autumn swallow]

Welcome @lone salmon!

[turbid tulip]

🚀 dagger v0.1.0-alpha.31 is out (as of Monday night)!

Release highlights:

• Add support for tcp:// DOCKER_HOST in docker.#Command, docker.#Load, docker.#Run h/t @heavy gazelle
• Add tests for http.#Wait
• The first parts of @cloud canyon's work towards our Europa release.
  • runtime: support legacy Pipelines in new execution engine
  • runtime: new execution engine
  • runtime: support for context imports
  • runtime: context: support secret environment variables
  • runtime: context: support secret files
  • dagger.#FS support

Complete changelog: https://github.com/dagger/dagger/releases/tag/v0.1.0-alpha.31

Install & Upgrade instructions: https://docs.dagger.io/1001/install/

As usual, if you need any help or spot any bug, please let us know.

[winter linden]

For tomorrow’s community call we’ll try porting a real-world Dagger configuration to the new Europa standard library. Featuring special guest @heavy gazelle 🙂

As usual bring your ideas and questions.

https://discord.gg/mWrVPmYE?event=918223847910686802

[rain oriole]

I'm interested but I won't be able to attend. That's the time I pick up my kid 🙂

[winter linden]

Starting now 🙂

[winter linden]

Going for lunch. Will cleanup our notes from the community call after, thanks everyone for participating!

[turbid tulip]

@winter linden Looking forward to seeing them! Curious what you uncovered with @heavy gazelle.

[heavy gazelle]

Thank you @winter linden, that was so good, looking forward to episode 2 in the week after next 😉

[autumn swallow]

Welcome @sullen cargo!

Welcome @hollow surge!

[winter linden]

Draft PR here 🙂 https://github.com/thechangelog/changelog.com/pull/401

Completely unfinished. We’ll continue tomorrow

[turbid tulip]

Sweet! I’ll check it out tonight or tomorrow!

[turbid tulip]

I like it! It'll be fun to get the rest of the config built. 🙂

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @tame oxide!

[tame oxide]

Hey all, any admins here who could help? I applied for Dagger back in September and received an invite on Nov 23rd, I've only just seen it as it was in my junk box. The invite has now expired; would love to get a new one 🙂 or do I need to re-apply to early access?

[wraith niche]

Hello, yes sending you a PM. I'll refresh your invite.

If anyone here does not have access to the repository or the docs yet, please ping me. I'll give you access right away.

[winter linden]

Thinking about doing part 2 today, 1pm pacific. Anyone interested?

[turbid tulip]

@winter linden 👋🏻

[winter linden]

@rain oriole let me know if another time works better for you

[rain oriole]

For me I’m mostly free at 21h30/22h which is 3 pm PST.

[winter linden]

Ok I can do both 🙂 This afternoon is all Europa DX for me anyway

[winter linden]

Community call discussion

[autumn swallow]

Welcome @lilac void!

[lilac void]

Hey, thanks for the invite. Excited to learn more about Dagger. I’ll dive into docs as soon as I have some free time

[autumn swallow]

Welcome @leaden patio!

[swift inlet]

Awesome, don't hesitate to ping if you need any help

[leaden patio]

Hello 🙂 How open are you to integrate vendor in there: https://github.com/dagger/dagger/tree/main/stdlib in particular would scaleway be a good fit? Do you have plan to have out of tree plugins to support vendors?

Could you also add https://github.com/Sh4d1

[leaden patio]

Et Val ?☺️

[meager fiber]

vendors

[autumn swallow]

Welcome @swift barn!

Welcome @eager lance!

[heavy gazelle]

Dagger (via GitHub Actions) just deployed https://changelog.com . The new pipeline is 2 minutes quicker than what we had previously with CircleCI. This is a summary of the entire story & code behind it: https://github.com/thechangelog/changelog.com/pull/395

Listen to @turbid tulip & @swift inlet tell us more about it next week on https://changelog.com/shipit

Also follow @winter linden's attempts to break changelog.com production: https://github.com/thechangelog/changelog.com/pull/401

[autumn swallow]

Welcome @pine jetty!

[winter linden]

Awesome @heavy gazelle !! What a great milestone

[autumn swallow]

Welcome @zinc dawn!

Welcome @short gust!

[short gust]

ty

[autumn swallow]

Welcome @stray jolt!

[autumn swallow]

Welcome @orchid rapids!

Welcome @swift barn!

[swift barn]

Happy to be here and be able to test dagger 😀

[winter linden]

Welcome @swift barn 🙂

If you get stuck or have questions, don’t hesitate.

We also have a big release coming in January, with major UX improvements. All the user requests we couldn’t fit in smaller incremental releases basically

[swift barn]

thanks

[autumn swallow]

Welcome @manic heath!

[orchid rapids]

looking forward to trying out Dagger this week, read the docs so far and it looks really cool and intuitive!

[autumn swallow]

Welcome @naive sequoia!

[naive sequoia]

👋

[winter linden]

Welcome 🙂

[autumn swallow]

Welcome @knotty copper!

[heavy gazelle]

SPOILER: This CI/CD Lego set is ready for Christmas https://github.com/thechangelog/changelog.com/pull/395

[autumn swallow]

Welcome @rigid storm!

[autumn swallow]

Welcome @mystic bronze!

[autumn swallow]

Welcome @snow sequoia!

[snow sequoia]

👋 Hello and thanks for the invite

[wraith niche]

Welcome!

[autumn swallow]

Welcome @swift barn!

Welcome @delicate forge!

[autumn swallow]

Welcome @frank frost!

[autumn swallow]

Welcome @humble marten!

[autumn swallow]

Welcome @ashen ice!

[autumn swallow]

Welcome @cedar folio!

[autumn swallow]

Welcome @sharp oracle!

[winter linden]

Hi everyone, the whole Dagger team is taking time off the last week of the year, so we will not be as responsive. We hope you are able to take time off too. See you next year, we have a lot of excitement planned for you. First our Europa release, our largest yet! And then… launch preparations 😁

In the meantime… happy holidays !

[rain oriole]

Happy holidays! 🥳

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @latent mural!

[autumn swallow]

Welcome @vapid stag!

[silk apex]

happy new year to all the dagger community

looking forward to discover the europa release 😄

[winter linden]

Happy new year!

[bronze storm]

looking forward to trying out Dagger this week, reading the docs and trying to create CI workflow to setup home cluster with all necessary deployments

[turbid tulip]

Us too! Stoked for you all to get your hands on it and tell us how we can make it even better.

[autumn swallow]

Welcome @sick rain!

[autumn swallow]

Welcome @swift barn!

Welcome @pastel magnet!

Welcome @violet socket!

[violet socket]

Thanks for the invite, excited to poke around soon 😄

[turbid tulip]

Welcome, @violet socket! Let us know how we can help you out!

[autumn swallow]

Welcome @swift barn!

Welcome @hushed dock!

[hushed dock]

Thanks for the invite, this week I’m on PTO but definitely will give it a spin later, and I have plenty of docs to read through

[autumn swallow]

Welcome @swift barn!

[swift barn]

Hello everyone 👋
Thanks for the invite 🤗
I played with CUE before and can't wait to see how you used it to make CI/local experience awesome 💪

[winter linden]

Thanks ! It’s still work in progress 😅 Big UX improvement dropping soon.

[autumn swallow]

Welcome @feral herald!

[autumn swallow]

Welcome @scenic sequoia!

[autumn swallow]

Welcome @robust agate!

[autumn swallow]

Welcome @patent juniper!

[winter linden]

Welcome to the other side 🙂

[autumn swallow]

Welcome @ornate wave!

[autumn swallow]

Welcome @naive skiff!

[autumn swallow]

Welcome @pure holly!

[autumn swallow]

Welcome @clear oriole!

[autumn swallow]

Welcome @rotund crystal!

Welcome @steel shadow!

[autumn swallow]

Welcome @tight vigil!

[autumn swallow]

Welcome @swift barn!

Welcome @grand ravine!

[grand ravine]

hello

[heavy gazelle]

It's nice to see you @grand ravine - what brings you here? 😉

[grand ravine]

Hey there :)! I’d like to try dagger ! The idea is great :)!

[heavy gazelle]

That sounds great! What is the first thing that you want to do with it? Happy to help 🙂

[grand ravine]

Have in mind a client I wrote ..! Maybe I would start to create a ci for it !

[heavy gazelle]

That sounds great! Would you like us to kick it off together?

[grand ravine]

Let me read the basic documentation and start trying! Then I will ask for help ! Thanks !

[autumn swallow]

Welcome @dire hawk!

Welcome @buoyant solar!

Welcome @frosty tendon!

[autumn swallow]

Welcome @swift barn!

Welcome @noble tinsel!

[autumn swallow]

Welcome @short summit!

Welcome @odd oyster!

[autumn swallow]

Welcome @solemn schooner!

Welcome @inner rain!

[solemn schooner]

Hello - just starting to read docs as @grand ravine
Thanks for the invitation 🙂

[grand ravine]

😀!

[wraith niche]

Welcome @solemn schooner and @grand ravine, let us know if you need any help!

[autumn swallow]

Welcome @raven mauve!

[raven mauve]

Hi everyone and thank you fro the invitation! Let's dive into the docs

[turbid tulip]

Welcome, @raven mauve!

[autumn swallow]

Welcome @ornate cedar!

[autumn swallow]

Welcome @languid terrace!

[autumn swallow]

Welcome @brazen matrix!

Welcome @open wyvern!

[autumn swallow]

Welcome @mild scarab!

[autumn swallow]

Welcome @silent sluice!

[autumn swallow]

Welcome @uneven haven!

[autumn swallow]

Welcome @tawdry void!

Welcome @bitter granite!

[autumn swallow]

Welcome @sour imp!

[autumn swallow]

Welcome @wet wadi!

[autumn swallow]

Welcome @silk temple!

[turbid tulip]

Welcome @silk temple! Nice to have you here. Enjoyed your Klustered episode with @vestal creek and Thomas: https://www.youtube.com/watch?v=ysfUgYs4YYY

[autumn swallow]

Welcome @slate beacon!

[winter linden]

I found this comment insightful and, although the author is talking about build systems, I think it applies 100% to Dagger and we should take it seriously.

https://news.ycombinator.com/item?id=29871438

saagarjha

Considering that all build systems must keep a dependency graph internally, it’s surprising how many of them make it incredibly difficult to inspect why things are being rebuilt. I don’t really get it either, I’ve talked to a lot of people and getting feedback on how the build system is doing its task is one of the biggest complaints I see (othe...

Hacker News · Jan 10, 2022, 6:54 AM

[turbid tulip]

Does Buildkit have a way to show differences between states?

That would be killer for this.

[autumn swallow]

Welcome @woven laurel!

[autumn swallow]

Welcome @frail latch!

[frail latch]

Hello! Thank you for the invitation! I'm going to read the documentation to learn more 🙂

[autumn swallow]

Welcome @blissful gulch!

[heavy gazelle]

It's very nice to see you here JSP. Welcome!

[heavy gazelle]

Nice to see you too David. Let us know if you get stuck on anything 😉

[winter linden]

Not directly I believe, but it does provide a full resolution map of the DAG, with topology, content-addressed ID of every edge and vertice, and per-edge event stream we can hook at will. I think caching decisions are included?

[turbid tulip]

Ok. That makes sense.

[winter linden]

With the ability to connect that back to CUE values (and from there, to Dagger engine operations) we could present something very powerful to end users..

I think the first step, though, would be to present a full picture of the DAG to end users, even without details on what was changed or executed this time. Just a static map of the DAG.

[turbid tulip]

Yeah, that'd be really good.

[winter linden]

There was a discussion in universe not long ago, about the idea of generating a standard graphviz output from a given Dagger config, to then visualize with a variety of graphviz-compatible tools and libraries

Food for though if anyone is looking for a fun side project 🙂

[silent agate]

I was noodling on similar ideas recently. Wondering if there was some useful synergy between the CUE value lattice and a graph database...

perhaps dagger graph via https://github.com/goccy/go-graphviz

[autumn swallow]

Welcome @solemn harness!

Welcome @fiery anchor!

[turbid tulip]

One small concern I have about this is... if statements and how that might impact the DAG.

[fiery anchor]

Hello - thanks for the invitation! I just finished reading the non-reference docs, which provide a very nice intro. One topic I was surprised to not see addressed is multi-node coordination - a particularly tricky concern for portable CI. Did I miss something obvious or does anyone have pointers on how Dagger will think about that?

[winter linden]

It definitely is a tricky topic. We will add more content down the road. But there is excellent news in the meantime: we delegate this 100% to buildkit 🙂 So if you can find a solution in buildkit, you solved the problem for Dagger as well

Here is an example, from the buildkit repo, of distributing buildkit jobs across multiple kubernetes nodes: https://github.com/moby/buildkit/tree/master/examples/kubernetes/consistenthash

[fiery anchor]

Thanks - that helps!

[turbid tulip]

Not sure if this will be useful... but seems related: https://github.com/moby/buildkit/blob/ffe2301031c8f8bfb8d5fc5034e5e509c5624913/client/llb/diff.go#L93-L108

Note: This isn't in the latest release, only on master

[autumn swallow]

Welcome @surreal prism!

[autumn swallow]

Welcome @swift barn!

Welcome @warm flower!

[warm flower]

Hi everyone! 🙂

[autumn swallow]

Welcome @mossy turret!

[autumn swallow]

Welcome @ionic lotus!

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @digital dagger!

[autumn swallow]

Welcome @regal monolith!

[unkempt wigeon]

I think my original comment that went into how/if to visualize the DAG came from, is my wish I can understand how and what I can "stick together" to get a pipeline written and that I thought CUEs value lattice or/and then the validity of the resulting DAG helps. Now, I don't know if that makes sense at all 😅 but writing the loosely YAML "stuff" often lefts me (perhaps because I haven't read the documentation) lost.

[autumn swallow]

Welcome @oak valve!

[oak valve]

Hey! Thanks for the invite. Super excited to dig into it more (and it's great seeing CUE get more usage )

[unkempt wigeon]

this would not represent resolved state, right? it's kind of a possible/alternative DAG. I'm not that much into dagger, but I assume you have a static and "runtime" aspect regarding the resolved resulting DAG?

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @dark wave!

[turbid tulip]

🚀 dagger v0.1.0-alpha.32 is out (as of Wednesday)!

Release highlights:
- We added support for private repos for dagger mod: https://github.com/dagger/dagger/pull/1186
- Much of work since alpha-31 has been focused on building the core engine for our next major design, codenamed Europa.
- Our intent with Europa:
- Dramatically improving the developer experience and ergonomics of using Dagger, and remove some of the patterns that caused issues in configurations in the past.
- The new dagger.#Plan configuration schema is intended to simplify and consolidate inputs and outputs, secrets, and the actions that plans should take to execute CI/CD pipelines.
- The new implementation of our core types is here: https://github.com/dagger/dagger/tree/main/pkg/dagger.io
- This includes low-level primitives for defining a #Plan, filesystem states (#FS), and a number of engine tasks: https://github.com/dagger/dagger/tree/main/pkg/dagger.io/dagger/engine, including running shell commands, getting source code from git, reading and writing files, pulling and pushing containers, and building from a Dockerfile.
- We are working on a new set of universe packages that utilize these low-level types: https://github.com/dagger/dagger/tree/main/pkg/universe.dagger.io
- These are still in very active development, and some of them simply don't work yet.
- You can see examples of how the Dagger team has used Europa in our tests
- For #Plan: https://github.com/dagger/dagger/blob/main/tests/plan.bats + https://github.com/dagger/dagger/tree/main/tests/plan
- For engine Tasks: https://github.com/dagger/dagger/blob/main/tests/tasks.bats + https://github.com/dagger/dagger/tree/main/tests/tasks
- We are also in the process of converting the changelog.com dagger config (found here: https://github.com/thechangelog/changelog.com/blob/master/2021/dagger/prod_image/main.cue) to Europa, you can find our current versions here: https://github.com/dagger/dagger/tree/main/pkg/universe.dagger.io/examples/changelog.com

Complete changelog: https://github.com/dagger/dagger/releases/tag/v0.1.0-alpha.32

Install and Upgrade instructions: https://docs.dagger.io/1001/install/

As usual, if you need any help or spot any bug, please let us know.

[autumn swallow]

Welcome @light quiver!

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @scenic fiber!

[autumn swallow]

Welcome @whole apex!

[winter linden]

For anyone interested in this kind of thing: we’re discussing Dagger’s approach to open-source licensing, branding, and how to build a great community AND business. https://twitter.com/solomonstre/status/1483857267011190787

Solomon Hykes (@solomonstre)

This is the model we chose for https://t.co/8fo2WhpPce. Red Hat proved that by restricting how your trademark is used, you can build a huge open-source business and brand without restricting how your software is used. You just need to design your business model accordingly :) https://t.co/XtmlRiM9UI

Twitter · Jan 19, 2022, 5:41 PM

I suspect that we will get more questions on this topic as we get closer to launch. We have very specific opinions on this topic, and are happy to share them anytime 🙂

[autumn swallow]

Welcome @slender star!

[autumn swallow]

Welcome @zinc mirage!

[autumn swallow]

Welcome @spark stag!

[autumn swallow]

Welcome @steep shoal!

[autumn swallow]

Welcome @unreal flower!

[autumn swallow]

Welcome @flint lintel!

[autumn swallow]

Welcome @haughty lark!

[haughty lark]

Hi I am trying to learn about how to write custom CUE templates in Dagger

Could anyone tell me what this means?

#Input: {
    @dagger(input)
    _
    ...
}

#Output: {
    @dagger(output)
    _
    ...
}

I think ... already represents an open struct. Why does it have _ here?

[winter linden]

_ allows it to match non-struct values

[winter linden]

These definitions #Input and #Output are kind of special cases, they were intended as convenience wrappers for @dagger() attributes (annotation, same as Go tags). So that you could set those annotations without learning the extra syntax for attributes. We’re moving away from using CUE attributes, so these definitions will be deprecated in the next release.

If you can share more about your use case, I’m happy to help you find more useful code examples

[autumn swallow]

Welcome @swift barn!

[haughty lark]

Thanks @winter linden !
My use case is like this:

• I am trying to parse a app.yaml as a values file: https://github.com/hongchaodeng/ai-demo/blob/main/app.yaml
• It will translate into some CUE structs that generates yaml files

[autumn swallow]

Welcome @quartz arch!

[autumn swallow]

Welcome @royal drum!

[autumn swallow]

Welcome @frank sigil!

[autumn swallow]

Welcome @violet flicker!

[violet flicker]

Hey, folks 👋🏽

Hope you’re having an awesome Friday! Not sure if this is the right channel for intros/hellos. My name is Derek from South Africa. My colleague told me about Dagger and immediately it sounded like something I’d want us to use.

I’m on our company’s security committee and platform team and I’m hoping that if I can be an early contributor this will help me convince the teams to use Dagger once it’s production ready. I don’t have any experience in Go but I’m happy to learn. If there’s any bugs that need help or small features, etc. I’m happy to help out 😃 Also more than happy to help out with documentation and other important stuff that people don’t always like doing but is necessary

[autumn swallow]

Welcome @shut jay!

[turbid tulip]

Welcome, @violet flicker! We'd love to have your help. Lots to do. 🙂

There are a number of issues here: https://github.com/dagger/dagger/issues/

We're currently focused on the next milestone: https://github.com/dagger/dagger/milestone/5

And the issues in the Europa milestone will follow that: https://github.com/dagger/dagger/milestone/4

I'm not sure if we've done a good job of splitting out good starter issues, but we could probably find some stuff that would be good starters. Another thing you could do is begin to work on packages that utilize dagger that would be useful at your work. Hopefully that would help you out and it would give us feedback to make dagger even better.

[violet flicker]

Awesome! Thanks! I will check it out this weekend and see what I can do

[turbid tulip]

Sounds great! Feel free to ping us and ask about any of it.

[silent agate]

@violet flicker welcome! I think the biggest help is in implementing dagger in real world scenarios and providing feedback on bugs or usability (DX) issues

[autumn swallow]

Welcome @swift barn!

[haughty lark]

It looks like dagger relies on docker to run? What if I want to run dagger as a k8s job or pod?

[winter linden]

Everything is run in buildkit - same backend as a docker build

So any valid buildkit installation is also a valid Dagger installation. Including running buildkit on top of kubernetes 🙂

For example: https://github.com/estesp/buildkit-cluster-example

You can configure dagger to use any buildkit daemon. By default, it will bootstrap a buildkit installation automatically, on top of a Docker engine. So Docker is a requirement for easy auto-install of buildkit; but not for running Dagger itself

See https://docs.dagger.io/1011/operator-manual/#custom-buildkit-setup

[haughty lark]

Why does it run on buildkit? In my case I just want to use it to render a couple of yaml files and orchestrate applying order..

Nothing to do with buildkit

[winter linden]

Well it has nothing to do with Linux either, but you’re still running it on Linux probably 🙂

Joking aside: we use buildkit as a general-purpose VM. It allows you to run any program as a DAG of nodes running in parallel, instead of a sequential list of operations. It’s a great fit for CICD pipelines in particular.

[haughty lark]

I see. The buildkit is the workflow manager

[winter linden]

Yes. Using buildkit gives you lots of powerful features for free. For example every operation are automatically cached if poosible, so you get lots of performance optimization out of the box.

Every input and output is content-addressed, so you can reliably inspect what changed from one run to the next; get a reliable audit trail, etc

[haughty lark]

Nice!
One more question. Is there any way for Dagger to parse yaml files for inputs?

[winter linden]

Yes CUE supports yaml natively. You can do it either statically with cue import (one time conversion of your yaml file to a cue file), or at runtime with something like:

import (
  “encoding/yaml”
)

rawYAML: string
data: yaml.Unmarshal(rawYAML)

// Enforce a schema
data: {
  hello: string
  foo: int
}

Then the question is: how do you want to pass the yaml file to Dagger?

• Do you want to pass the raw contents as a string?
• Do you want to pass a directory, and have Dagger open the file itself?
• Something else?

[haughty lark]

Pass it as dagger input

currently all values are taken as dagger input xxx

[winter linden]

But you can pass either string inputs, or directory inputs

[haughty lark]

I want to take it like dagger input -f app.yaml

[winter linden]

ok

So string input

[haughty lark]

Right.

[winter linden]

So in my example below, you can just mark rawYAML as an input, with rawYAML: string & dagger.#Input

Then dagger input text rawYAML -f app.yaml

Oh, I forgot about dagger input yaml. If you use that, then your configuration gets simpler:

data: {
  hello: string
  foo: int
}

Then dagger input yaml data -f app.yaml

Then the yaml file is implicitely unmarshaled, and merged at the key data. If the contents of the file match the structure of your CUE value, then the merge succeeds

[haughty lark]

Awesome! That's what I am looking for 🙂

[haughty lark]

@cloud canyon
Nice talking to you. Here is the issue describing the use case I just mentioned:
https://github.com/dagger/dagger/issues/1481

[cloud canyon]

@haughty lark Thank you! Nice talking to you as well, will look into it

[haughty lark]

How can I get the output data of a struct? I only find dagger output list but that only shows struct

[haughty lark]

What does op.#Load mean?

    #up: [
        op.#Load & {
            from: kubernetes.#Kubectl
        },

        op.#WriteFile & {
            dest:    "/kubeconfig"
            content: TestCluster.kubeconfig
        },

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @vital brook!

[turbid tulip]

🚀 dagger v0.1.0 is out!

Important Notes

This is our first minor release. If you are an early Dagger adopter, we recommend that you lock your pipelines to this version, since we intend to start shipping more meaningful (and potentially breaking) changes in 0.2.0 pre-releases, starting with 0.2.0-alpha.1.

Based on all your feedback, we have been working on a simpler and more intuitive configuration schema which we intend to ship as 0.2.0. Internally, we refer to this as Europa. You see our work on this new schema in dagger.io and universe.dagger.io

Release highlights:
- engine.#Source- https://github.com/dagger/dagger/pull/1312
- This allows Dagger packages to use shell scripts, etc. from the filesystem, rather than needing to embed them in CUE (think go:embed for Dagger)
- rename engine.#Build to engine.#Dockerfile - https://github.com/dagger/dagger/pull/1404
- We started implementing Europa-native packages
- alpine package - https://github.com/dagger/dagger/pull/1449
- netlify package - https://github.com/dagger/dagger/pull/1462
- Add engine.#TransformSecret - https://github.com/dagger/dagger/pull/1435
- This allows secrets to be transformed using CUE functions, allowing for unmarshalling of YAML/JSON, stripping of newlines, etc.
- docker.#Build - Fixed up some issues related to CUE structural cycles and definitions.
- nested #Build - https://github.com/dagger/dagger/pull/1468
- This is working better, but we still have some issues we need to dig into and understand better for deeply nested docker.#Build structs.
- docker.#Run - https://github.com/dagger/dagger/pull/1438
- Updated the struct, so that it can be used in docker.#Build
- Fix missing auth in docker.#Push - https://github.com/dagger/dagger/pull/1448
- Re-implement docker registry parsing - https://github.com/dagger/dagger/pull/1454
- Allow hidden Tasks - https://github.com/dagger/dagger/pull/1403
- This allows package authors to hide tasks within a package.
- Port docker.#Copy to hidden fields - https://github.com/dagger/dagger/pull/1410
- Port engine.#Subdir to hidden fields - https://github.com/dagger/dagger/pull/1406
- Support for partially running a DAG - https://github.com/dagger/dagger/pull/1302
- @hoary spoke cleaned up git repository management - https://github.com/dagger/dagger/pull/1431
- Regular weekly releases h/t @heavy gazelle - https://github.com/dagger/dagger/pull/1372

Complete changelog: https://github.com/dagger/dagger/releases/tag/v0.1.0

Install and Upgrade instructions: https://docs.dagger.io/1001/install/

As usual, if you need any help or spot any bug, please let us know.

================

Also!

🚀 dagger v0.2.0-alpha.1 is out!

Important Notes

This release is identical to the v0.1.0 release, it is meant to signal potentially breaking changes as we work towards the Europa release.

Complete changelog: https://github.com/dagger/dagger/releases/tag/v0.2.0-alpha.1

Install and Upgrade instructions: https://docs.dagger.io/1001/install/

As usual, if you need any help or spot any bug, please let us know.

[heavy gazelle]

dagger v0.1.0 just pushed changelog.com into production in 72s. The non-Dagger, CircleCI equivalent, took 240s:

• https://github.com/thechangelog/changelog.com/commit/625db715953f8c509f0c8c473cc8e19610e5c1a1
• https://github.com/thechangelog/changelog.com/runs/4943752186
• https://github.com/thechangelog/changelog.com/runs/4943752102

[turbid tulip]

💥

[autumn swallow]

Welcome @timber ermine!

[autumn swallow]

Welcome @solar wharf!

Welcome @fast cargo!

[autumn swallow]

Welcome @thorn cedar!

[autumn swallow]

Welcome @dull widget!

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @fringe gale!

[autumn swallow]

Welcome @glossy cave!

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @barren axle!

[barren axle]

Hey folks! I'm tazjin from nixery.dev, tvl.fyi and some other stuff. @heavy gazelle invited me here after some followup discussions we had after recording https://changelog.com/shipit/37

I'm interested in contrasting Dagger with Nix.

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @safe burrow!

[heavy gazelle]

Hey! It's great having you here, welcome 👋🏻 I'm looking forward to your thoughts, as will @turbid tulip @silent agate @swift inlet @wispy tapir I'm sure of it 😉

[barren axle]

@heavy gazelle Thanks! Actually going to dive right in because I came up with a complicated question I didn't ask you yet. Question is long, but I'll try to put a TL;DR at the end 🙂

There is a concept in some systems about extending a computation graph from within. For example in the gg paper (good summary: https://buttondown.email/nelhage/archive/papers-i-love-gg/ ; the important section is "The compute IR"), a step that models some kind of computation can "return" a "primitive value" (e.g. bytes on disk after running some build command) or it can return another (list of) step(s) to run.

This concept is pretty powerful. Off the top of my head, here are some examples:

• Nix has this, we call it "import from derivation". It means running something in Nix that generates more Nix code and then importing it into the same execution graph. We use this in TVL as part of our Nix build system for Go (https://code.tvl.fyi/about/nix/buildGo) to have Nix first fetch third-party deps, then run an analysis tool on them, and then import the output of that analysis tool to generate the build plan for the dependency.

• Google's CI system allows checks to return more checks to run. For example, this lets you write a check which analyses the kinds of files included in your change and generate more checks to run based on that (e.g. language-specific linters).

• Buildkite has a feature for uploading more build steps from within a build step (https://buildkite.com/docs/agent/v3/cli-pipeline). We use this in our build pipeline generation (https://cs.tvl.fyi/depot/-/tree/nix/buildkite) to go from a pipeline with a single initial step to our fully dynamic CI (e.g. https://buildkite.com/tvl/depot/builds/12075)

Is there an equivalent in Dagger and what does it look like?

(TL;DR: Can a Dagger op (not sure about the term?) dynamically output more ops to run?)

[turbid tulip]

The short version is that it can't currently do that. There are a few limitations in cue/flow that make that difficult (we can't currently change the evaluation tree to add nodes/tasks dynamically). But I'd say that is more of a current limitation than a fundamental limitation. I've been digging into that code base and I'd say it is something we can likely do in the future.

All of that said, I'm curious about how you would imagine doing that. What sorts of operations would you want to have that would add additional steps? I like your example of language-specific linters. Do you have you any thoughts about the interface for that?

[winter linden]

In the case of the gg paper (great paper btw :), if I remember correctly, there are use cases specific to the compile/build use case. For example, dynamically scanning for dependencies between files in a large codebase.

[autumn swallow]

Welcome @swift barn!

Welcome @grim lance!

[barren axle]

@winter linden yeah, that's a good case! For us the key insight is that this applies at multiple different levels. We're interested in finding a good solution for distributing the graph of our CI builds (which is dynamically generated; adding a new project in our monorepo only consists of adding the source files for the project - the rest is "discovered"). Right now our solution is specific to Buildkite, because they have this feature at the right level.

I'm thinking about the idea of Dagger as "CI-system independent CI definition", in which case we'd likely "compile" our Nix evaluation graph into something Dagger can consume. We're doing this outside of Nix itself because of two reasons:

a) we have steps that are not "pure" (e.g. integration tests), which conflict with Nix's model
b) Nix itself is (currently) not very good at distributing to e.g. multiple separate build hosts - in Buildkite we can "model" that using their build agents

It's also interesting because we have customers that use our setup who are currently tied to Buildkite if they want their CI to work the same way, but it would be nice to have an abstraction level there that makes this possible on ~any CI system without losing too much of the UX we have right now

[winter linden]

That sounds like a great fit for Dagger and we would love to help make it happen 🙂

One unfair advantage of Dagger is that we rely on buildkit for the heavy lifting. 2 major benefits in this case:

1. Compatibility with the container ecosystem without the worst of its baggage (buildkit adds the useful constraints of the DAG model compared to raw docker engine)

2. We can leverage buildkit infrastructure. Clustering, storage, multi-arch nodes, caching backend.. if someone did it for buildkit it will work out of the box for Dagger

[barren axle]

Buildkit is curiously the most opaque component of this for me. Cue and the DAG engine make sense right away.

I don't fully understand how Buildkit interfaces with tools that build container images on their own. In Nixery for example I have a completely custom builder and layering logic with close to optimal content-based caching (described in the talk https://www.youtube.com/watch?v=pOI9H4oeXqA or in text here: https://tazj.in/blog/nixery-layers).

My vague feeling is that Nixery should probably "compile to" this LLB format when running standalone and not as a registry.

A couple more hours in each day would be useful 😅

[winter linden]

Buildkit takes over the entire build DAG. If you were to integrate Nixery with it, it would be kind of like moving some of your internal data structures to Redis (if that makes sense). It’s the same functionality you already have (build and layering logic, content-based caching etc) but packaged in its own daemon with its own API and infrastructure.

In practice Nixery would 1) compile LLB and 2) probably submit them on the fly to the buildkit daemon to be resolved. It theory you can decouple those two steps (llb can be serialized to a file) but I don’t know anyone who actually uses that pattern. Perhaps something to be explored

[barren axle]

Intuitively the LLB serialisation would make sense in the Nix case, since then we'd get caching for that for free. Not sure how much that is worth though unless the closure of image contents is humongous. Regardless of that, this might be interesting to experiment with.

On the other topic, I think a first sketch could be actually generating the Cue files from Nix (a bit dirty, but could work as a POC) and then invoking Dagger on the generated files.

[winter linden]

Unless Nixery’s current build & caching logic is broken or you were planning major refactoring anyway… Personally I would just leave it the way it is. Bolting in buildkit may not be worth the effort. I could be wrong.

On generating the Cue files from Nix. That’s a really interesting idea 🙂

[barren axle]

The main Nixery use-case is the ad-hoc registry at the moment (spitting out images based on the image name), but there are some people that would like to use the same build logic "standalone". I think for that second use-case Buildkit may be interesting. Not super high priority atm though ...

I was thinking re: generating Cue - does the Cue code evaluate to some data structure that is then processed, or is there tighter integration with the evaluation? In the former case, it might be easier to output that data structure directly from Nix code instead

[winter linden]

I’m not sure what your inputs are, but if it’s possible, outputting JSON would be cleaner

Nix -> JSON -> CUE

• Execute a nix-specific tool that generates json
• Execute a Dagger configuration that loads the json

[turbid tulip]

For CUE, the data structure is CUE. The evaluator behind the scenes does translate it into an abstract data type, but it's all defined as golang data structures (and, as such, not available to an end user).

[barren axle]

Okay, so it's not like cue -> ?? proto ?? -> dagger, but more direct?

the JSON generation would be very easy - that's what we do right now for Buildkite. Hmmhmm

[turbid tulip]

We're using an evaluation engine in CUE called flow, which allows tasks to be defined by CUE values (typically a field), which progressively concretizes the CUE value lattice.

So, the output from a task becomes concrete, which allows another task to start, etc. in a DAG.

[winter linden]

nix+dagger

[autumn swallow]

Welcome @lost bone!

[autumn swallow]

Welcome @coral wedge!

[coral wedge]

Hi happy to join 😉

[autumn swallow]

Welcome @tired robin!

[autumn swallow]

Welcome @twin scaffold!

[autumn swallow]

Welcome @compact magnet!

[compact magnet]

Hey All!

[autumn swallow]

Welcome @rustic shard!

[autumn swallow]

Welcome @sonic patio!

[autumn swallow]

Welcome @blazing moth!

[winter linden]

For those of you who use Twitter… Dagger now has a Twitter account 🙂 https://twitter.com/dagger_io

dagger (@dagger_io)

A portable devkit for CI/CD pipelines, by the creators of Docker. https://dagger.io

Tweets

11

Followers

11

Twitter

[umbral glacier]

Goal:

Implement a system that allows deployment of multiple services to a cluster in a micro-service / service-oriented architecture. There are a bunch of separate services running together to make a cohesive whole. Every service is an app in and of itself and it needs to fulfill the 12-factor app principles.

Build and run server could be completely separate, meaning - the build can happens on one location, while deployment or running the service can happen on another, completely separate location.

Only the build server requires access to the source code as input. The job of the build system is to produce all the needed artifacts for going to production. These artifacts include the containers of the images, the manifests for the cluster - maybe even the description of the infrastructure (as terraform source code maybe?) required to completely run it?

The deployment system takes as input the artifact from the build system and runs it - it spins up infra as needed, applies the manifests to the cluster etc.

Is this a crazy idea? If yes, tell me why? If it's not so crazy - can dagger do this?

[wispy tapir]

I’m sure on one point, it’s definitely possible with dagger 🚀
You should check our Kubernetes tutorial, it’s similar to your use case but with one service, but thanks cue, it’s not a big deal to do something like that on a micro service infrastructure 😄

[umbral glacier]

the key point here is the k8s manifests and the infra descriptions being part of the built artifact

conceptually this is like a golden image on steroids or something

[wispy tapir]

You can generate your k8s manifests directly from dagger, we also wrote a #Kustomize definition to apply kustomization on already existing yaml, this way you can dynamically update your manifest with the image tag you built previously and so on 😉

[umbral glacier]

Imagine this use-case: a developer is working on a feature for some service x but in order to implement said feature they need to add new infrastructure. Say for example they need to start using a redis or mongo database where previously they didn't. Now they write code that connects to redis into a new branch on github, they test it locally (by providing the connection params to localhost via env variables) but they now want to deploy this to a staging environment in the cloud. I want them to be able to push terraform code, app code and k8s yaml files all on the same branch and have a deployment ready in the cloud.

Basically gitops but not only limited to k8s (like flux or argocd would be) but k8s + infra that sits outside of k8s

With the twist that a build stage happens only once per commit and produces a complete deployable artifact of that commit. If you lost the entire git history and source code but still had this artifact you could deploy it to production

[wispy tapir]

With dagger, you can run any kind of commands, from a simple bash script to just anything you want, that is not limited to k8s or anything.
Everything that can be run in a container should work in Dagger.
So what I would do is create a dagger configuration to build my project -> push it to a remote registry, this way even if you lost the source code, you still have the working image in the registry.
And then from that build, I would update the k8s yaml (still from dagger) to use the latest image build and indeed we have a terraform package to apply terraform configuration so I would just pass it as input to deploy and use your package to apply it.
This way the developper can just update the terraform code and that's it.

This is how I see it, maybe @heavy gazelle can add more context or share his experience to confirm my words

You can also create dependency between task, so you can build -> push -> update yaml & deploy infra

[umbral glacier]

but that all happens as part of one process that combines build + deploy, right?

what if I want to build on monday, but deploy what I built on tuesday?

I guess I could rephrase this

[wispy tapir]

Ohhhh okay I see

[umbral glacier]

https://12factor.net/build-release-run

I want the 12 factor app separation of build release run but want it to include infra too

[wispy tapir]

I'm not sure it's available for now but we are working on the possibility to run only one part of the dag, so that should be possible

So you could run the build part on monday and the deploy part on tuesday 🚀

[wispy tapir]

Thank you for the link!

In my opinion, if it's possible to run a part of the dag, it will be easy to match your need, we are saving outputs (for example the image sha) in a JSON so we could reuse it during deployment

[umbral glacier]

so what I'm thinking about is roughly -> a BUILD is a complete specification of what an application and the infra it needs to run in a point in time. So then an imaginary release process could take as input this build artifact along with any enviroment variables it needs plus things like infra state files to apply this to and presto you got a release

you could say it would enable time travel releases

that sounds promising!

would it be that crazy of an idea to have as build output not just the image shas but also the IaC source describing the infra they need to run on?

so the build is a function of source code with some amalgam as output

build(source) -> artifact

and then release/run is a function of that artifact plus related input to a real-world deployment

run(artifact, config, infraState) -> thing is live and running

I didn't get enough sleep last night so I could be rambling semi-coherently 😂

[wispy tapir]

If it's an output of a definition, that's definitely possible, there is an issue about that : https://github.com/dagger/dagger/issues/1317
And it has been merged here : https://github.com/dagger/dagger/pull/1528

[wispy tapir]

It's okay, I'm not sleeping well too 🤣

I think you should book a call with the Dagger team to talk about this, we would be happy to help you 😄

[umbral glacier]

so the output could literally be a zip file with terraform code in it that a run phase could point terraform apply to? along with kubectl apply etc.

[umbral glacier]

would be happy to book a call!

[wispy tapir]

In theory it's possible, I've never output a zip file because that feature is new and I think it only supports string for now, but with some change, we could output zip

[umbral glacier]

I just want a sanity check first - does this thing that I'm thinking about make sense to do at all? Do you see any major issues with this approach?

[wispy tapir]

I'm pinging you back with a calendly link from one of the team later

[wispy tapir]

IMO that make sense, split the building process from the deployment isn't a bad idea and it improve the flexibility of your infrastructure, versioning etc... so yeah
I think other member of the team should give their opinion, it will maybe be different from mine

[rain oriole]

@umbral glacier You can also run different plans, one for build and another for deploy, just reference the .cue file with the plan in the CLI. As for outputting, with outputs: directories, you can save any #FS into your local disk, so that includes .zip files or any other file that's produced using a task.

[umbral glacier]

One neat thing you could do with an approach like that is the following scenario:

You have your app running in production. It is a nodejs backend that connects to a postgresql database and runs inside a k8s cluster. The code for your app is all the javascript application source code, the terraform source code that describes an EKS cluster, an RDS instance and the k8s yaml files that describe yours pods and whatnots.

Now, you want to add some async work capabilities and you add in the popular bullmq package for giving you queues and workers etc. But for that to work you need to point it to a redis instance. So you add a redis instance definition inside your terraform source code. You also need to add another pod that runs the workers connecting to redis instance so you make the changes to your manifests by adding the required yml files as well. Now the PR for adding this work is complete. You push it up. A CI pipeline finishes the build phase and produces your artifact. You now have the following options:

1. Take this artifact and apply it to your existing staging or even production environment. The run phase upon receiving it will run terraform apply and spin up a new redis cluster for you to use, will add a new pod into your running cluster and update all the others. You have now UPGRADED this environment to have this feature.

2. You can also take this artifact and apply to a brand new environment that doesn't yet exist. Say you give it an empty infra state file. Now the run phase does terraform apply but it notices that it needs to create not just a redis instance, it needs to create also an RDS instance, an EKS cluster etc. You have now spun up a completely new environment with this feature.

Why choose? You could do #2 to test things out first and if satisfied pull the trigger on #1. Or you could even do #2 multiple times for n copies of your complete app (of course they would all get random domains or some other way to differentiate them)

[rain oriole]

Sure! You can have tweaks in your plans for dev, staging and prod. Either different plans that share definitions, or by feature flags using inputs: params, or maybe with partial dags (not ready).

[umbral glacier]

what is a partial dag?

[rain oriole]

It enables you to run only part of your plan.

[umbral glacier]

what is the envisioned use case for partial dags? What i'm describing could also be implemented with different plans right?

[rain oriole]

It depends how much different the plans are. You could have, at the root of your plan's actions, keys for your environments (e.g., dev, staging, prod) and run partially the one you want, instead of the whole plan.

If they're similar apart from a few configs, you can set that in inputs.

If they're pretty different (dev vs prod maybe), I would rather have different plans.

[umbral glacier]

See I don't want hardcoded environments. I will have as many environments as the number of times I applied the run/release of a build

I guess run phase would also need as output something that a destroy phase could take as input to clean the whole thing up

[rain oriole]

By environment I mean type of configs, not instance of that config.

[umbral glacier]

oh ok, gotcha

[rain oriole]

In dev you could be running docker compose, and in prod kubernetes.

[umbral glacier]

ok, got it - what would be the difference between staging and production in that example? staging would have smaller instance sizes or something like that?

[rain oriole]

You tell me. That depends. It could just be the target cluster, which is a simple setting.

[umbral glacier]

ah but IMO the target cluster should be an input of the run phase

[rain oriole]

But your terraform configs are different between those?

[umbral glacier]

no, they wouldn't be in this case

[rain oriole]

Or is it just the difference of using something that's there (prod) vs something new everytime (staging)

[umbral glacier]

basically

or prod vs staging could just be how we name and use it

[rain oriole]

Yep, you can have simple settings in your CUE to trigger one behavior vs another in your plan.

[heavy gazelle]

I like the thought that you put into this. While I have read the rest of the comments, I am going to reply to your first one. Happy to talk more.

The short answer is no, it is not a crazy idea. Yes, Dagger can do this. There's more ⤵

We run a Phoenix app on Kubernetes. Dagger runs inside GitHub Actions.
The pipeline stops when it publishes the resulting container image to Docker Hub.

In Kubernetes, we run a reconciler - https://keel.sh - that triggers deployment updates when there is an update to this image.
In your case, I think that you would want to replace keel.sh with Dagger, and run another pipeline with its own set of rules.

Having a seam between build & deploy is a good idea in my opinion. The / in CI/CD is important, and some people respect it.
While it's OK for a single pipeline to do it all, I understand why you don't want that - I am doing the same thing.
I am also thinking about replacing keel.sh with something else, and while I would want to do it with Dagger, there are a few issues which are not yet solved.

If you decide to use ArgoCD or Flux for K8s, and integrate with Crossplane Terrajet for the Terraform part, I would totally understand that. I partially did that 2 months ago, and while I am moving towards Dagger, I don't have a proven solution to share just yet 😉

I like your thinking @umbral glacier and would be curious to learn more about your use case. A repository to look at would be great.

More context:

• https://github.com/thechangelog/changelog.com/pull/395
• https://github.com/thechangelog/changelog.com/pull/399
• https://changelog.com/shipit/33

[umbral glacier]

That's awesome @heavy gazelle . I will dig into this a little later. Glad to see I'm not completely rambling crazy thoughts. Was surprised that this kind of practice is so rare or almost non-existent. I was thinking I had to be missing something

[heavy gazelle]

It's more common among the high-performing teams, which are at the more extreme end of the scale of CI/CD. We most certainly hope that encoding these practices within Dagger will help others to benefit from this approach more.

[umbral glacier]

I like your thinking @umbral glacier and would be curious to learn more about your use case. A repository to look at would be great.

@heavy gazelle I am actually in the process of setting up a CI/CD pipeline for an older system that had none previously. The CI part is done and it stops when container images get published to the registry. The other part, CD, is just getting started and I am a little overwhelmed with all of the options since I don't have much experience in the space and it's my first time setting something like this up. All of this to say - not much of a repository to look at right now but I will definitely try to circle back once I have something

[heavy gazelle]

I understand. I'm a big fan of taking small steps towards better. It helps with the getting overwhelmed part, and focusing more on the fun, exploratory side of it. We now know the direction that you are going towards, it sounds good to me, and the next step is to look at the particulars via some code. Leaving it with you 👋🏻

[autumn swallow]

Welcome @worn lichen!

[autumn swallow]

Welcome @snow bloom!

[autumn swallow]

Welcome @scarlet crow!

[autumn swallow]

Welcome @silk carbon!

[surreal echo]

If anyone wants to run Europa on Circle via an Orb, I've made this:https://github.com/logicalhq/dagger-orb
(Please note it's a private orb, so you'll have to fork the repo if you want to use it in your own org)

[winter linden]

Wow nice! Thank you @surreal echo !

[autumn swallow]

Welcome @nova robin!

[autumn swallow]

Welcome @cosmic pawn!

[coral wedge]

Hi guys
Glad to join this mind-blowing project and this growing community 😄

I'm so excited!

[winter linden]

Welcome @coral wedge and thank you for the kind words!

Yay, https://twitter.com/dagger_io just broke 50 followers 😂 Small beginnings

dagger (@dagger_io)

A portable devkit for CI/CD pipelines, by the creators of Docker. Launching early 2022... https://dagger.io

Tweets

14

Followers

50

Twitter

[autumn swallow]

Welcome @earnest ember!

[autumn swallow]

Welcome @fierce stone!

Welcome @analog gale!

Welcome @abstract nexus!

[haughty lark]

I have made a demo based on dagger: https://github.com/hongchaodeng/ai-demo

Is there any community meeting that I can join and share it?

[swift inlet]

Hi 😇
Awesome news 🔥
We have one tonight aimed on another use case, but I'm sure you can jump and show us what you did 😁

[winter linden]

Let’s schedule another one dedicated to your demo @haughty lark . Tomorrow would be too tight IMO. Can you send me your availabilities for next week by DM? We’ll figure out scheduling. Looking forward to seeing it!

[winter linden]

If you're watching the live-coding session: use this thread to participate 🙂

[autumn swallow]

Welcome @placid grove!

[autumn swallow]

Welcome @real garden!

[winter linden]

100 followers 🥳

[winter linden]

I love that the numbers are so small 😊 It’s way more fun going from 0 to 100 than from 1000 to 10000 (but that will be fun too)

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @fierce harbor!

[fierce harbor]

Hi all, I got the email to allow early access to dagger but when I go to github I get a 404

any ideas?

[swift inlet]

Hi, there may have been an issue with our bot. I unfortunately don't have the access to check it out 😇
It will be resolved by Monday for sure, thanks for telling us 🙏

[fierce harbor]

It didnt work when I originally got the invite either ... I think it might be more than a restart ... no rush of course 🙂

[wraith niche]

Give me your GitHub username, I can probably fix it manually

[fierce harbor]

testbucket

dont judge me 🙂

[wraith niche]

@fierce harbor ok, go to https://github.com/dagger/dagger , you should see the invite pending. Once accepted, the links from the welcome email (docs) should work, let us know

[fierce harbor]

working now! thanks @wraith niche and @swift inlet

[autumn swallow]

Welcome @gloomy prawn!

[autumn swallow]

Welcome @leaden fiber!

[autumn swallow]

Welcome @green bramble!

[autumn swallow]

Welcome @pale fossil!

[pale fossil]

hey @wraith niche , can you resend my invite? it's expired. GH username is benmoss.

[wraith niche]

Done! Go to https://github.com/dagger/dagger/ - you should see it. docs.dagger.io should also work now.

[nova robin]

Hi DevOps Pros, I am trying to understand more about the world of devops and would love to learn more about your experiences. If you are passionate about devops and would like to share your knowledge with someone new to the subject, I would like to schedule a quick call with your to discuss the main challenges and problems you are facing.

This is for my own exploration purposes on devOps processes, and on dagger.io, although I will likely share any interesting insights I gain with the dagger folks.

What is in it for you? Well, 30 minutes to brag about your work, and make a new connection, just for the heck of it. 🙂

DM me if you have the time.

[autumn swallow]

Welcome @halcyon vault!

[autumn swallow]

Welcome @mental peak!

[autumn swallow]

Welcome @random crag!

[autumn swallow]

Welcome @tribal arch!

[autumn swallow]

Welcome @glacial flame!

[autumn swallow]

Welcome @sharp marsh!

[autumn swallow]

Welcome @prisma sparrow!

[autumn swallow]

Welcome @bitter skiff!

[autumn swallow]

Welcome @lofty dagger!

[winter linden]

200 🙂

[haughty lark]

FYI, I implement KubeVela on client side using Dagger: https://github.com/hongchaodeng/vela-dagger

[autumn swallow]

Welcome @wary cave!

[winter linden]

This is fantastic. I think we could probably make this into an official package in Dagger Universe, so anyone could import universe.dagger.io/kubevela and get started with kubevela in a few minutes. Would you be interested in contributing that?

[autumn swallow]

Welcome @abstract arch!

[autumn swallow]

Welcome @zenith wind!

[winter linden]

Hey there @zenith wind good to see you here 🙂

[autumn swallow]

Welcome @cedar oracle!

Welcome @vivid moat!

[autumn swallow]

Welcome @gray dew!

Welcome @swift barn!

[silent agate]

We're looking for pilots that would use dagger to deploy AWS infrastructure. Please DM me directly. I will personally help with your implementation of dagger (europa release)

[autumn swallow]

Welcome @wet locust!

[haughty lark]

Of course! It's still early. Let me chat with @swift inlet first 🙂

[winter linden]

FYI we'll be discussing design blockers for the 0.2 release in 40mn : https://discord.gg/GRRVxC48?event=946464302259376208

[autumn swallow]

Welcome @eternal ocean!

[autumn swallow]

Welcome @marble vapor!

[autumn swallow]

Welcome @lost egret!

[autumn swallow]

Welcome @shut silo!

[autumn swallow]

Welcome @neon warren!

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @soft lichen!

[autumn swallow]

Welcome @hybrid gorge!

[hybrid gorge]

hey there, thanks for the invitation!

[autumn swallow]

Welcome @sage prism!

[sage prism]

Hola everyone! This is one very interesting project and thanks for the invite

[turbid tulip]

Nice to have you here, @sage prism!

[autumn swallow]

Welcome @regal otter!

[winter linden]

Welcome 🙂

[autumn swallow]

Welcome @pseudo stream!

[pseudo stream]

Thanks!

[autumn swallow]

Welcome @hearty kestrel!

[autumn swallow]

Welcome @main trail!

Welcome @swift barn!

[swift barn]

Hello, thanks for the invite!

[winter linden]

Welcome 🙂

[autumn swallow]

Welcome @tepid horizon!

[autumn swallow]

Welcome @clear tiger!

[autumn swallow]

Welcome @fresh dagger!

[autumn swallow]

Welcome @fair plover!

[autumn swallow]

Welcome @bitter oak!

[winter linden]

Hi @everyone. Here it is, finally... 0.2.0-beta.1. The first beta release of Dagger 0.2, codename "Europa"!

In addition to a new version of the dagger CLI, it also features a new website, and new docs (still work in progress). Check it out at https://beta.dagger.io

Dagger 0.2 includes a major overhaul of our developer experience. It is more powerful, and simpler to use, than 0.1... But it's also incompatible! If you have a Dagger 0.1 plan, you will need to migrate it manually, but don't worry, we will help. Ping us here or open an issue, and we will guide you through the process.

Thanks and please send us feedback so we can keep improving!

One more note: this release is special because it kicks off our countdown to launch 🙂 Once Dagger 0.2.0 is officially finished... We will open the community and source code to everyone. Finally!

[muted gale]

Looking forward to showing dagger to more people

[haughty lark]

Looking forward to the new features!!

[distant pebble]

Congrats Dagger team!

[patent stirrup]

Do we have a way to integrate with GCP workflows?

Two areas: 1. dagger could execute a workflow. 2. Dagger could possibly run in a workflow (need to validate that this is possible)

[swift inlet]

Hi 👋
We would definitely want that ! 😄 It's currently not the case. Can you make an issue out-of-it, to keep track of your finding (2) especially + (1), the spec that you envision for the package. We had a gcp integration in the past, so (1) shall totally be possible. We may take some parts of the previous gpc integration to integrate to this new version

[patent stirrup]

Sure, will do!

[autumn swallow]

Welcome @slow mural!

[winter linden]

We have officially released 0.2.0 🙂 Look in your inbox for a detailed release post. Happy testing everyone! Thank you for your support!

[autumn swallow]

Welcome @west frost!

[autumn swallow]

Welcome @ripe crag!

[autumn swallow]

Welcome @jovial trout!

[coral wedge]

Well done guys.

[autumn swallow]

Welcome @blazing horizon!

[autumn swallow]

Welcome @drowsy briar!

[winter linden]

Thanks Lionel!

[autumn swallow]

Welcome @loud glade!

[loud glade]

Helllooo all, especially @winter linden @drifting crown 😀

[winter linden]

Yay you made it! Sorry about the glitch

[autumn swallow]

Welcome @zenith remnant!

Welcome @hazy hamlet!

[hazy hamlet]

Hello everyone
Happy to be here!!

[zenith remnant]

Hello 👋 Happy to be here too 🙌
Let's see if I can finally get my head around dagger 😄

[turbid tulip]

Let us know if we can help!

[autumn swallow]

Welcome @civic tinsel!

[autumn swallow]

Welcome @boreal fable!

[boreal fable]

Congrats on shipping dagger 0.2! What a giant payload!

[autumn swallow]

Welcome @shut shore!

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @sullen spear!

[autumn swallow]

Welcome @humble pasture!

[autumn swallow]

Welcome @scenic tree!

Welcome @final swift!

[final swift]

👋 Howdy!

[autumn swallow]

Welcome @earnest granite!

[autumn swallow]

Welcome @mighty meadow!

Welcome @fading sluice!

[neat anchor]

https://docs.dagger.io/1202/plan/?utm_source=Dagger+Early+Access&utm_campaign=c3541b50ed-EMAIL_CAMPAIGN_2021_08_17_04_28_COPY_01&utm_medium=email&utm_term=0_05e3a05bcc-c3541b50ed-453229090

==> Oups!

It seems you don't have the permission to see Dagger's documentation. But don't worry you can request an Eary Access :). You'll be redirect to Dagger website in -8 seconds

See you soon !

ah actually all links sent by mail redirect me to an error page

[last pine]

Strange... @neat anchor could you try to access to docs.dagger.io in a private window please ?

[last pine]

@neat anchor : I found the bug and opened a PR. I let you know when this will be fixed. Thanks

[autumn swallow]

Welcome @verbal canopy!

[last pine]

fixed

[vestal creek]

Freaky timing with that comment, @heavy gazelle

[autumn swallow]

Welcome @barren patrol!

[autumn swallow]

Welcome @echo palm!

[neat anchor]

@last pine sorry, didn't see your message. It works fine now! thank you!

[last pine]

my pleasure 😉

[autumn swallow]

Welcome @radiant talon!

[radiant talon]

Kind of a conceptual question, not sure if this is the right place but

Somewhere I see dagger potentially helping us is in things like setting up dev environments; fetching containers, fetching secrets, running some other jobs like populating a test database, spinning up a local isntance; is that something that makes sense?

[winter linden]

Yes absolutely. Dagger is perfect for that.

It actually started out as a tool to do only that. Then we redesigned it to be more flexible.

[autumn swallow]

Welcome @swift barn!

Welcome @muted quest!

[autumn swallow]

Welcome @chilly sonnet!

[autumn swallow]

Welcome @chilly arch!

[autumn swallow]

Welcome @orchid raven!

[autumn swallow]

Welcome @wicked badger!

[autumn swallow]

Welcome @bitter swift!

[smoky shard]

Hiya I like to develop using my local file system and Mac set up because it’s faster and more ergonomic than doing development inside a container convenient. But my CICD and Dagger all run Docker containers. Is there some way to get parity between my local file system for local development and what’s in the container that dagger and cicd uses?

[wispy tapir]

Hey 👋

Create CI/CD with Dagger required 2 things :

• A buildkit daemon (launched using docker by dagger if no one is set)
• A plan (made in cue)

So basically, you can run the daemon in a container but the plan can be in your local system

[smoky shard]

Ah yes that is what I’m doing. But it means my project dependencies are all in the Docker image. I’d prefer them on the local file system for development eg: hot reloading, running tests etc.

[wispy tapir]

What are you trying to do with Dagger?

[autumn swallow]

Welcome @stone tulip!

[winter linden]

You can access the local filesystem at will: see client.filesystem, you could for example load your dependencies directly from local fs. However you will need to handle platform differences

[autumn swallow]

Welcome @lilac thicket!

[lilac thicket]

Hey all 👋 I’ve been dagger-adjacent for a while, but I’m diving in properly this week 😀 Really looking forward to seeing how I can stop building the same blooming’ pipelines in different systems, over and over again 😂

[winter linden]

Welcome! Yes that is the dream 🙂

[sharp marsh]

general feedback for the https://docs.dagger.io/1203/client page. I would personally love for each of the sections to be fully working dagger configurations so I can fiddle with them to learn. As they are today, if I copy/paste (the copy button is there) them into a dagger project they won't work due to missing package and import definitions. Thoughts?

[winter linden]

OK it's settled then!

[verbal canopy]

Filesystems: You should only write once to any named filesystem. Multiple writes to a single filesystem can cause unexpected results. This bit of trivia cost me a least a day of troubleshooting.
Platform: This appears to build on the previous example without explaining why you used client: _ and what it means.

[rain oriole]

Multiple writes to a single filesystem shouldn't even be possible due to the way the CUE api is set. If you have:

client: filesystem: "./dir": write: contents: actions.build.one.output
client: filesystem: "./dir": write: contents: actions.build.two.output

Then those two contents are going to be unified, which should error. Can you explain a bit more how you had it set up? Did you have different paths pointing to the same place on disk?

[verbal canopy]

It wasn't possible. Every attempt to compile the config would hang CUE/Dagger indefinitely consuming 90%+ CPU. There was no error.

[mystic mantle]

Also, it would be great that those would be checked by the CI directly. A little bit like Go that runs the examples in _test.go files, so we know our documentation is up to date with the current dagger implementation: https://go.dev/blog/examples

[winter linden]

400 twitter followers today 🙂 🚀 Small beginnings

https://twitter.com/dagger_io

dagger (@dagger_io)

A portable devkit for CICD pipelines, by the creators of Docker. Powered by CUE and Buildkit. Launching early 2022... https://t.co/ojiN8fWCeI

Followers

403

Tweets

53

Twitter

[autumn swallow]

Welcome @toxic trellis!

[autumn swallow]

Welcome @timber wind!

[autumn swallow]

Welcome @analog iron!

[autumn swallow]

Welcome @viral rapids!

[autumn swallow]

Welcome @sage plank!

[autumn swallow]

Welcome @radiant cape!

[autumn swallow]

Welcome @earnest prawn!

[cloud canyon]

dagger v0.2.3 is out 🚀 🚀

This release focuses on polishing the CLI and CUE API, notable changes:

• Move core actions to a subpackage
• Rename dagger.#Service to dagger.#Socket
• Move connecting socket to client: network
• Add docker CLI package

We'll be conducting a bug bash today and tomorrow -- we'd very much appreciate if you could try out the release and the documentation, to report back anything unexpected!

@everyone

[autumn swallow]

Welcome @pastel sluice!

[autumn swallow]

Welcome @ember juniper!

[autumn swallow]

Welcome @neon ice!

Welcome @frail bronze!

[winter linden]

Tomorrow we're throwing our first virtual meetup, right here on Discord. It will be an unconference format: bring your topics and questions! The Dagger team will be there to answer your questions, discuss the roadmap, show real-world examples of Dagger, live coding, and more.

Event link: https://discord.gg/ZSnUCNYr?event=958494137127161916

See you soon @everyone!

[autumn swallow]

Welcome @swift barn!

[autumn swallow]

Welcome @steady wasp!

[winter linden]

@everyone Dagger is now officially launched! https://dagger.io/blog/public-launch-announcement

Thank you all for investing your time and energy with feedback and contributions, Dagger is improving a little bit every day thanks to you.

[autumn swallow]

Welcome @deep mesa!

Welcome @earnest patio!

Welcome @inland harness!

Welcome @median owl!

[median owl]

yay congrats on the launch!

[winter linden]

Hello everyone! Our little community call is starting in one minute. Participants are muted by default, but raise your hand here, or ask questions if you want to participate and we'll unmute you!

[autumn swallow]

Welcome @vital shoal!

[cloud canyon]

We moved to "The Stage"!

[autumn swallow]

Welcome @lime plover!

[cloud canyon]

What should we add next to the Dagger Universe?

https://github.com/dagger/dagger/discussions/1922

[autumn swallow]

Welcome @ebon meadow!

Welcome @nimble iron!

[median owl]

gotta run to a meeting but good to see you all again 👋

[verbal canopy]

Re: UIs in CI/CD. https://twitter.com/paulstovell/status/1508973078298398724

Paul Stovell 🇺🇦 (@paulstovell)

With this feature, we hope to prove something to other CI/CD tooling vendors: you can add version control without the UX suffering; without the DevOps toolchain becoming inaccessible to the rest of the team.

Twitter · Mar 30, 2022, 1:03 AM

[autumn swallow]

Welcome @reef birch!

Welcome @waxen sparrow!

[heavy gazelle]

next 5 pages cover a small portion of the Concourse topic that we were talking about on The Stage: https://gerhard.io/slides/observable-systems-beta/#/21

[autumn swallow]

Welcome @tawny notch!

[verbal canopy]

I've started testing out using binlog (https://msbuildlog.com/) for MSBuild. One awesome benefit is the Log Viewer tool generates a flame graph of your build.

[heavy gazelle]

there is more here on the Concourse subject in this repo - bte pipeline.gerhard.io no longer works - https://github.com/gerhard/pipeline

[tawny notch]

hi, saw this amazing project announced on twitter and I just had to come say hi to the community

[turbid tulip]

Welcome, @tawny notch!

[autumn swallow]

Welcome @dawn otter!

Welcome @neat rampart!

[tawny notch]

are there any high level introductory technical guides I can look through?

[neat rampart]

Hey everyone! Not sure to where to direct this, but the website currently redirects to two different Discords! The first link is this Discord, and further downthe website says

If you share our definition of fun, come say hi!

This link redirects to a discord with Cyrillic characters in the name. I'm pretty sure this wasn't on purpose 😅

[winter linden]

There's a getting started tutorial on https://docs.dagger.io . And also a section explaining core concepts.

[tawny notch]

awesome thanks!

[neat rampart]

This the other discord -> https://discord.gg/dagger

Pretty sure it's a mistake!

[turbid tulip]

@winter linden @last pine 👆🏻

[mystic mantle]

Good catch @neat rampart

[winter linden]

Thanks! We're fixing it now

[neat rampart]

Np! I'm also a Dev currently looking at new roles, so maybe this is my in 😅

[winter linden]

🙂 Well we're hiring! https://dagger.io/careers . We'll be opening new roles in the next couple weeks, too

[autumn swallow]

Welcome @wraith hull!

Welcome @tired pulsar!

[tired pulsar]

Hi - I build lots of ETL and DB schema pipelines in the microsoft stack

coming in to check what's going on here

[turbid tulip]

Welcome, @tired pulsar!

[autumn swallow]

Welcome @pale current!

Welcome @frail dagger!

Welcome @swift barn!

Welcome @worthy tiger!

Welcome @stable grove!

Welcome @fringe cargo!

Welcome @scenic saddle!

Welcome @graceful wigeon!

Welcome @swift barn!

Welcome @agile anchor!

[sharp marsh]

https://github.com/dagger/dagger/discussions/1922 > Please vote your next Universe ideas here!

[autumn swallow]

Welcome @visual crater!

Welcome @shy dust!

Welcome @calm heron!

Welcome @south sluice!

Welcome @faint vapor!