Summarizing recent discussions about

[wintry depot]

The engine OCI image should include the CLI. It should be the canonical way to distribute the CLI.
I don't understand this part, it seems like it should be the inverse: the CLI would include the engine image (so it can bootstrap an engine, etc.). If an OCI image is the canonical distribution mechanism for the CLI, installing the CLI would involve dissecting an image to extract out a layer that contains the CLI.

If the CLI instead embeds the engine image then it can operate as an engine, bootstrap an engine elsewhere, etc.

[ember storm]

(on a call but will clarify my words after. Sorry for the confusion)

[random trout]

Afk now, but there's also a multiplatform concern here - e.g. can you bootstrap onto a amd64 system if you're arm64

[ember storm]

Relevant part:

Both the native CLI, and the OCI bootstrap material, should be derived from the One True Artifact

[tender venture]

I'm guessing this won't affect existing distribution channels, right? (eg. CLI download)

[ember storm]

Correct. It would affect (mildly) the contents of those packages (instead of just the CLI, would be CLI+extra material)

[wintry depot]

Afk now, but there's also a multiplatform concern here - e.g. can you bootstrap onto a amd64 system if you're arm64
Right but that would be a problem to deal with either way (either include multiple platform CLIs in the engine image or multiple platform engines images in the CLI, or just don' t and make this a non-optimal path that results in network deps)

Correct. It would affect (mildly) the contents of those packages (instead of just the CLI, would be CLI+extra material)
Extra material being the OCI image?

[ember storm]

Yes I'm picturing the "One True Artifact" as a big fat multi-platform image with absolutely everything in it. Then all other release artifacts are derived from that

Extra material being the OCI image?

Yes, either the OCI image as-is, or pieces of it

[wintry depot]

Okay, following enough for now 😄 Still many details and all that but that's fully expected, realize this is still high-level. i.e. If we are packaging the CLI and oci image (or parts of it) together then I still wonder if the oci image should just be embedded in the binary for more simplicity, but that's closer to an implementation detail given what I'm understanding now so not important at this moment.

Otherwise, the points STGM 👍

[ember storm]

I don't understand this part, it seems like it should be the inverse: the CLI would include the engine image (so it can bootstrap an engine, etc.). If an OCI image is the canonical distribution mechanism for the CLI, installing the CLI would involve dissecting an image to extract out a layer that contains the CLI.

I didn't mean to say that all installers should always download the OCI image then dissect its contents. But, all other installers should be derived from the OCI image.

This is a significant change from today, because the OCI image is presented as just one artifact among several of equal importance. And the component packaged in that image (what I call here the "buildkit runtime") presented as an implementation detail, not to be used directly except in niche cases, involving private interfaces etc

Instead, I propose saying basically:

• The Dagger Engine is the only thing being released. It includes CLI and buildkit runtime
• The OCI image is the canonical artifact which contains all components of that release
• For convenience, we also provide native packages, all derived from the OCI image, to facilitate bootstrap
• You can also run the engine directly from the OCI image, in server mode. That is a mainstream use case, not niche

[random trout]

Aha I think I'd missed the idea of "derivative" artifacts - I think I like it

[random trout]

I particularly like the idea of grabbing the cli out of an engine image layer, but this doesn't work as well for windows/Darwin

Another idea could be to use oci artifacts? I think our registry supports them (ghcr/gcr)

Though that's not "one" artifact, but a single artifact with "attached" artifacts (though they can share binary blobs like layers)

[ember storm]

Yeah, I don't think we necessarily need to make mac users actually download the OCI image from the registry, then extract the CLI from that. If there's an easy way to do that from super-standard tools (ie. bash, curl, tar) then we should consider it. But it's also fine IMO to prepare a "normal-looking" mac installer, that happens to be 100% derived from the OCI image

Also all of this should be done with reusable Dagger functions, so that we can make these functions available to downstream rebuilders, and give them an opportunity to run them themselves. Some won't be willing or able to. But others will. Chainguard has already told me that they are OK with it.

[forest pollen]

What does this look like in CI? For platforms like gitlab, argo workflows, tekton, we typically have a privileged engine sidecar with a "runtime" container that actually executes the dagger call. The "runtime" container is usually alpine or dind, with the first step downloading the dagger cli and the second step running dagger call.

1. Does this mean that "runtime" image would be the same as the engine image?
2. Does this mean there's some slim/minimal cli image instead?
3. Does this mean we no longer need a sidecar?

[ember storm]

Does this mean that "runtime" image would be the same as the engine image?

Yes. For CI you would have 2 choices:

• Option 1: client mode. Install the Dagger CLI in the unmodified CI runner. The CLI bootstraps its server using docker-docker or, in the future, custom compute drivers. This is the best option when your not willing or able to modify the CI infrastructure to be Dagger-optimized.
• Option 2: server mode. Run the Dagger engine image on your compute - could be via kubernetes (100 different ways), docker-compose, a systemd service, or anything else that can run a privileged oci container. The dagger image is configured to run the CI runner as a dagger function at boot. This is the best option when you want to run Dagger-optimized infra, to take full advantage of the platform, and break free of CI lock-in.

[forest pollen]

Got it, so for users still in their path to total daggerization (0-99% of ci daggerized), they'll be in option 1

[ember storm]

I think there are 2 dimensions of adopting Dagger: 1) daggerizing your pipelines, 2) daggerizing your runner infra

• Makes sense that daggerizing your infra doesn't happen on day one: you need to have successfully daggerized something
• At the same time, daggerizing your infra doesn't require all pipelines to be daggerized either. It could happen as early as after the first successfully daggerized pipeline
• It could also never happen: for example if an app team successfully daggerized their pipelines, but don't control the CI infrastructure, and can't convince the infra team to daggerize their infra - they may continue using Dagger in client mode, embedded in an unmodified CI runner, forever. This should be considered an acceptable production use case also.

TLDR: everyone starts in client mode; everyone has to daggerize at least one pipeline successfully; then as you continue daggerizing more pipelines, you may continue in client mode indefinitely, or switch to server mode when it makes sense. You can also mix and match in hybrid infrastructure.

(obviously I'm using the present tense, but describing a hypothetical future state 🙂

[forest pollen]

The missing piece in my understanding of server mode is how it interacts with a ci platform. It sounds more like CI-less, which is great, but probably not realistic for most people. What would github actions + server mode look like?

[ember storm]

In server mode, the CI runner runs on dagger: it's wrapped in a dagger function. In turn the CI job can make nested dagger calls.

I call that function the "init function": it's passed to the engine image as a configuration. Either a command-line argument, or an env variable, or a mounted file.