Dagger Lock In?

[ember moss]

Hi,

I am new to the Dagger community. I tried Bazel today and just as quickly gave up. I think Dagger looks more achievable for me and much easier to read.

The only thing I am confused about is where Dagger Cloud fits in. Dagger itself is open source, and can be integrated with our own CI provider right? Are there special runners needed in order to use Dagger in CI or can basic GitHub Actions runners suffice? Can we self host runners in secure environments if we need to?

Just looking to make sure that I am not missing some form of lock in or required payment to use Dagger effectively. Things like the Earthly shutdown just make me cautious. Thank you!

[lost robin]

👋 hey there!

The only thing I am confused about is where Dagger Cloud fits in.

Dagger Cloud is a visualization and collaboration SaaS based solution that becomes quite handy as multiple teams within your org start adopting Dagger which enables faster troubleshooting as well organizational level management.

Dagger itself is open source, and can be integrated with our own CI provider right? Are there special runners needed in order to use Dagger in CI or can basic GitHub Actions runners suffice?

Standard basic GHA hosted runners work just fine.

Can we self host runners in secure environments if we need to?

Yep, several Dagger users already do this. We have some docs on how you can get started if you're using Kubernetes (https://docs.dagger.io/ci/integrations/kubernetes)

[ember moss]

Awesome thank you so much! I will keep exploring Dagger then because it looks so much easier to start than with Bazel

[ember moss]

@lost robin Is there any support for remote caching with Dagger? Either existing or planned? I see the caching section in the Docs but it is not specific if that is a local machine cache where dagger is running or remote

[lost robin]

we used to have an experimental service which we paused registration from as we're working in a revamped version with the goal that it should be easier to integrate with the default CI platform caching native capabilities. So currently, the only way to enable remote caching is via buildkit's exporters (https://github.com/moby/buildkit?tab=readme-ov-file#export-cache) until we release our solution. cc @plain dirge @karmic vapor

[ember moss]

Very cool thank you for the quick reply. As long as it is somewhere in the pipeline I'm happy, I'm not at any kind of scale to need real remote cache yet. If there is anything I can do to help with the solution let me know, I have limited time but can help if you need me

[lost robin]

sweet, thx for the offer. We'd like to know more about what you're planning to use Dagger with and you can count on our help towards reaching a successful PoC 🙌

[ember moss]

I needed a monorepo, multi language build system that wasn't completely cryptic. Nx and Turborepo are great for JS only repos, but not for multi languages. Bazel is extremely heavy, confusing to get started with, and a full time commitment (but its great if you put that in), and Buck2 is similar. Pants seemed okay, and I was considering Earthfiles when they led me to you guys.

My original use case is that I have a Vite project that I am building into a static site. I want to package it with my Python CLI so that users can run "cli view" or something and bring up the UI on their locals. I can build the python package. I can build the Vite project. I just needed a way to glue them together and didn't want to hack it together with scripts/Makefiles. It seems like something that should be pretty easy to accomplish with Dagger, and future builds will just be easier.

Where do you recommend storing Dagger code in a monorepo? Should each project that gets built have its own code in its own SDK language? Should I choose one language and build out a tools directory or something that handles all of the builds for the whole repo?

[karmic vapor]

There is a lot of flexibility to "follow your own adventure", but we recommend making each component in your project its own Dagger module. Then you can choose the best language for each module (the main criteria should be familiarity: whomever will be in charge of maintaining the module, should pick the language). And you can model your project's actual dependency graph as dependencies between dagger modules, cross-platform.

The downside of this approach compared to other, less general tools, is that you will get a little bit of boilerplate code in each module. You can cut down 90% of it, by moving shared logic in to common utility modules. But if your project has say, 50 micro-services with exactly the same workflow, that remaining 10% boilerplate can be annoying. We are working on ways to cut down boilerplate even more. It's a very manageable downside IMO, just something to be aware of.

[ember moss]

That is definitely okay. At least I'll be able to understand the boilerplate yknow? Can't say the same when I'm looking at Bazel rules

[karmic vapor]

To take our own project as an example:

• We have a module for our CLI: github.com/dagger/dagger/cmd/dagger
• Another for our docs: github.com/dagger/dagger/docs
• Yet another for our (very complex) version string logic github.com/dagger/dagger/version

And a growing collection of shared utility modules, like our base image builder, go builder etc. that are imported by the others:

• github.com/dagger/dagger/modules/go
• github.com/dagger/dagger/modules/wolfi
• etc.

So for example you can get a build of our docs server for any version, and run it locally right now:

dagger -c 'github.com/dagger/dagger/docs@v0.18.0 | server | up`

Or get an official build of the dagger CLI for any version:

dagger -c 'github.com/dagger/dagger/cmd/dagger@main | binary --platform=current | export ./dagger-dev'

If you look at the dependencies for github.com/dagger/dagger/docs, you can see that it has a dep on the CLI module github.com/dagger/dagger/cmd/dagger. Why would the docs depend on the CLI? Well it's to generate the CLI reference 🙂 The CLI module has a function for that. And the docs module calls that function to get the up-to-date CLI reference as part of the docs build

That kind of dependency ("docs depend on the CLI") are impossible to model in a regular build tool because they're completely cross-platform. So we kind of got used to not being able to get that information. But just because your tools don't know about a dependency doesn't mean it's not real

So this is an example of the deeper benefits of modeling your components as dagger modules

[ember moss]

Oh you weren't joking about the complex version logic 🥲

Okay so one last question

[karmic vapor]

Yeah... We do lots of version checking for managing compat between the CLI and engine

cc @acoustic hull who has suffered through a lot of this 😛

[ember moss]

I've put a lot of work into building a good .devcontainer to keep versions the same between developers for all languages and build tools. Is there some kind of interconnection where I can use the same devcontainer as the build container for operations? From what I've seen in the docs, most build pipelines start by pulling a container that has the toolchain they need, but I've already done the hard work of making sure the toolchain in the devcontainer is rock solid

[ember moss]

this is really cool

[karmic vapor]

Mmm, we don't have an actual compat bridge with devcontainer, the way we do with Dockerfiles... But in my experience it shouldn't be an issue to do get the compat you need where you need it.

--> Just to clarify, you want to continue using the .devcontainer as source of truth, and have Dagger reuse it as much as possible to avoid duplication - you're not looking for a one-time migration from .devcontainer to Dagger, correct?

[ember moss]

Correct. My .devcontainer actually uses a Dockerfile as the base, all of the other features I add are vscode extensions and are not part of the toolchain. So do you think that while inside the Devcontainer, I could use Dagger and use the devcontainers Dockerfile as the build container? I have setup a docker-in-docker setup so it would probably just build another version inside the container right?

[lost robin]

have you checked the Dockerfile native integration? https://docs.dagger.io/cookbook/#build-image-from-dockerfile. You can re-use your existing Dockerfile and get a *dagger.Container type to run your commands there. That might be a sweet spot so you don't have to make any changes to your current setup as you get familiarized with Dagger 🙏

[ember moss]

That's what I'm working on doing now :). I think this is going to work out super well and I appreciate all of the speedy responses guys

[lost robin]

sure! let us know in case you run into anything. We're around

[karmic vapor]

We also really want to allow the inverse: running a devcontainer inside Dagger. But Dagger is still missing a few features to really be viable as an alternative to Docker there. Starting with live filesync.

[ember moss]

How would I specify a specific file name in a module? Are filenames relative to the top of the module, the file I am currently in, or where dagger is being run from? So right now I have this:

from typing import Annotated

import dagger
from dagger import Doc, dag, function, object_type


@object_type
class Devcontainer:
    @function
    async def build_dockerfile(
        self,
        dockerfile: Annotated[
            dagger.File,
            Doc("location of Dockerfile"),
        ],
    ) -> dagger.Container:
        """Builds the Dockerfile provided"""
        container = (
            dag.container()
            .from_("alpine")
            .with_file("/src/Dockerfile", dockerfile)
            .directory("/src")
            .docker_build()
        )

        return container
    
    @function
    def devcontainer_dockerfile(
        self
    ) -> dagger.File:
        """Returns the Dockerfile for the devcontainer"""
        pass
        

and I want to have the devcontainer_dockerfile function return the Dockerfile as a File object. The Dockerfile is located a couple directories up and over from this specific file. What is the most robust way to do that kind of path operation?

[karmic vapor]

You can give a default path to your dockerfile argument

https://docs.dagger.io/api/default-paths/

A relative path means "relative to the root of the current module". An absolute path means "relative to the git repo containing the current module"

[ember moss]

Relative to the git repo you say? Awesome. Thank you @karmic vapor that is exactly what I was looking for

[ember moss]

dagger develop command is genius. I hate doing anything in my IDE without autocomplete (I'm literally useless without it) and the fact that it generates the whole SDK into my local folder including dependent modules is possibly the best thing ever

[ember moss]

How can I get bash to do the execution of a command in a Docker container instead of sh? My devcontainer Dockerfile is building, but when I try to run commands in it like pnpm or uv, I get errors that those aren't recognized. When I pull up a terminal, I can see that it is loading to sh. When I use bash, then uv and pnpm are recognized. I've tried the things I can think of including: setting the entrypoint of the container to bash, doing a with_exec command with /usr/bin/bash, and even editing my pnpm statement to be bash -c "pnpm build", and none of them have given me the right behavior. Whats the protocol?

[lost robin]

what's the current entrypoint of your container? By default, when running things with with_exec in Dagger, it won't use the container's entrypoint by default. There's a use_entrypoint argument you need to pass to that method for it to execute the entrypoint automatically 🙏

[ember moss]

I'm running into a few different errors depending on how I am executing commands. I think the most interesting is that simply pulling up the built devcontainer Dockerfile with a terminal and attempting to run pnpm install is resulting in a permissions error. The dockerfile looks like this:

FROM mcr.microsoft.com/devcontainers/base:ubuntu-22.04

# change user
ARG USERNAME=vscode
USER vscode

# install uv and python
RUN curl -LsSf https://astral.sh/uv/0.6.17/install.sh | sh
RUN ~/.local/bin/uv python install 3.13

# install homebrew
RUN NONINTERACTIVE=1 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
ENV PATH=/home/linuxbrew/.linuxbrew/bin:$PATH

# install dagger
RUN brew install dagger/tap/dagger

# install nvm
ENV NODE_VERSION=22.15.0
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash

# install node
RUN echo "source ~/.nvm/nvm.sh && \
    nvm install $NODE_VERSION && \
    nvm alias default $NODE_VERSION && \
    nvm use default" | bash

# install pnpm
RUN brew install pnpm

ENV DO_NOT_TRACK=1

and my dagger function looks like this:

@object_type
class Devcontainer:
    @function
    async def build(
        self,
        dockerfile: Annotated[
            dagger.File,
            Doc("location of Dockerfile"),
            DefaultPath("/.devcontainer/Dockerfile.devcontainer")
        ],
    ) -> dagger.Container:
        """Builds the Dockerfile provided"""
        container = (
            dag.container()
            .from_("alpine")
            .with_file("/src/Dockerfile", dockerfile)
            .directory("/src")
            .docker_build()
            .with_user("vscode")
            .with_workdir("/home/vscode")
            .with_entrypoint(["/usr/bin/bash"])
        )

        return container

When I run build | terminal

1. I still get a sh terminal even though the entrypoint is bash
2. After I enter bash manually, if I'm set as the vscode user I can see the tools I have installed (pnpm --version works) but using pnpm install tries to execute postInstall scripts from the modules and I get a permissions error.
3. If I try to run the container as root instead, I can no longer see the tools (pnpm --version no longer works) because they were installed in the container under the VSCode user
4. I can't change that part of the Dockerfile because then that breaks the actual devcontainer which logs in as user vscode and needs the tools installed there.

[karmic vapor]

@ember moss probably unrelated but you don't need that dag.container().from_("alpine"). You can just start from an empty directory and add the dockerfile: dag.directory().with_file("Dockerfile", dockerfile).docker_build()

or even better you could just get the source directory directly as an argument, filtered to only contain the dockerfile:

@object_type
class Devcontainer:
    @function
    async def build(
        self,
        source: Annotated[
            dagger.Directory,
            Doc("source directory for the devcontainer"),
          DefaultPath("/.devcontainer")
        ],
    ) -> dagger.Container:
        """Builds the Dockerfile provided"""
        container = (
            source
            .docker_build(dockerfile="Dockerfile.devcontainer")
            .with_user("vscode")
            .with_workdir("/home/vscode")
            .with_entrypoint(["/usr/bin/bash"])
        )

        return container

@ember moss terminal has its own separate configuration, you can pass it as an optional argument:

build | terminal --cmd=bash

or, configure a default:

ctr = ctr.with_default_terminal_cmd("/bin/bash")

@ember moss any chance you could share a trace URL on dagger cloud? Would be helpful to see the actual permission errors in context

you can press w (for "web") while the TUI is showing a running pipeline

[ember moss]

I will give that a shot tomorrow, thank you for the advice on the function build I was just thinking about how I could get rid of that alpine container.

What I might do, is to change the whole dockerfile to execute as root, and then in devcontainer.json they have a spot to put what user to connect as. Might solve some of the user permission errors I've been seeing. If that doesn't work I'll try to get a trace and send it here

[lost robin]

@ember moss last question. Dagger fails on the dockerbuild function? or you're trying to execute command sin the resulting container and that's what it's currently failing?

[ember moss]

Cmds in the container is failing either through terminal or with_exec. Container builds just fine

What's weird is those same errors don't occur when I use the container as a devcontainer. There must be some special flags that the devcontainer extension loads the container with

[lost robin]

I see. Have an example of a command that fails?

[karmic vapor]

almost certainly some profile file that doesn't get loaded & as a result env variables may be missing

a trace would help sort it out

[lost robin]

yep, very likely

[lost robin]

what Solomon shared here is also relevant

docker by default inherits the cmd of the parent container when yoru Dockerfile doesn't specify one. So in your case, your devcontainer is running /bin/bash when it starts in interactive mode

[ember moss]

For the permissions error to kick in, you need a pnpm project to load that has postInstall scripts. For example esbuild has a postInstall script that pnpm asked me to enable. Running pnpm install in the project in my devcontainer works aok. Running pnpm install in the dagger terminal causes the permission issue. I will share more details tomorrow back at my computef

[lost robin]

sure, thx!

[ember moss]

@lost robin @karmic vapor I fixed it. I changed my Dockerfile to

FROM mcr.microsoft.com/devcontainers/base:ubuntu-22.04

# install uv and python
RUN curl -LsSf https://astral.sh/uv/0.6.17/install.sh | sh
RUN ~/.local/bin/uv python install 3.13

# install homebrew
# ARG USERNAME=vscode
# USER vscode
# RUN NONINTERACTIVE=1 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
# ENV PATH=/home/linuxbrew/.linuxbrew/bin:$PATH

# install dagger
RUN curl -fsSL https://dl.dagger.io/dagger/install.sh | BIN_DIR=/usr/local/bin sh

# install nvm
ENV NODE_VERSION=22.15.0
RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash

# install node
RUN echo "source ~/.nvm/nvm.sh && \
    nvm install $NODE_VERSION && \
    nvm alias default $NODE_VERSION && \
    nvm use default" | bash

# install pnpm
# RUN brew install pnpm
RUN wget -qO- https://get.pnpm.io/install.sh | ENV="$HOME/.bashrc" SHELL="$(which bash)" bash -

ENV DO_NOT_TRACK=1

I got rid of the user statements and the install brew statement (because homebrew refuses to install as root which is why those user statements were there), and I did script installs for everything. I also added

  "remoteUser": "root",
  "containerUser": "root",

to devcontainer.json. Now the whole toolchain is installed on root, and root is used as the user for both the devcontainer and the build container in dagger

I don't know why connecting as vscode in the devcontainer allowed me to run scripts via pnpm install but did not in the dagger container. Still assuming its some settings that the devcontainer extension runs with. But it won't block it as root so I'm good with that. If you'd like I can try to cobble together a minimal reproducible repo for you guys to see how dagger interplays with the devcontainers

[karmic vapor]

a repro would be fantastic thank you 🙏

[ember moss]

@karmic vapor @lost robin https://github.com/papaymaguire/repro Reproducible repo is here. Instructions are in the README

[lost robin]

checking

[karmic vapor]

@ember moss would you mind opening an issue describing the issue you encountered? Will help track & prioritize on our end. 🙏

Thank you for the detailed feedback, it's appreciated

[lost robin]

@ember moss found what's missing. Here's the fix:

diff --git a/typescript/ui/.dagger/src/ui/main.py b/typescript/ui/.dagger/src/ui/main.py
index ac6c3f8..4f5df38 100644
--- a/typescript/ui/.dagger/src/ui/main.py
+++ b/typescript/ui/.dagger/src/ui/main.py
@@ -20,9 +20,8 @@ class Ui:
         """Returns a container that echoes whatever string argument is provided"""
         container = dag.pnpm().workspace()
 
-        outputs = (
-            container.with_mounted_directory("./ui", src)
-            .with_workdir("./ui")
-        )
+        outputs = container.with_mounted_directory(
+            "./ui", src, owner="vscode"
+        ).with_workdir("./ui")
 
         return outputs

with* methods don't currently honor the current container username when adding files. So it's adding the files as root when it should be vscode

I recall speaking with @acoustic hull and @sonic obsidian to improve this at some point. Can't remember if I opened an issue for that. Let me check

found it: https://github.com/dagger/dagger/issues/9188

I'll bump that issue and add comment to it

[karmic vapor]

Marcos catching that sneaky bug like

https://tenor.com/view/simon-pegg-nick-frost-edgar-wright-hot-fuzz-cornetto-trilogy-gif-15351929

[ember moss]

Ohhh I see. That is definitely a gotcha thank you. So what should be the protocol then for the time being? Should I keep my container building as root/the devcontainer using it as root, or should I change it back to vscode and proliferate that user statement to all of my with* calls?

Im partial to keeping it as root so I don't have to deal with any permissions in the build or in the devcontainer. I mean giving root access to a temporary container is hardly a cardinal sin. The only problem is if I have to reuse homebrew at any point because it REFUSES to install as root

[karmic vapor]

yeah personally I would leave it alone since you got it to work. You can always tinker again later

[ember moss]

That's the spirit I agree

Still finicking with the pnpm install command. When I try to use it through with_exec it doesn't like it. Maybe because bashrc hasn't run yet the PATH isn't complete? I'm not so great at docker exec statements but I'll keep trying

[ember moss]

I'm sorry to bother but do you think I could get some help writing this with_exec? I'm just trying to get the debug-container in the repro repository to do pnpm install. I tried adding the full path to the pnpm binary, which did work and started the install, but a postInstall dependency tried to run a node command. Since the PATH variables aren't loaded it couldn't find node and failed. I've tried a lot of different methods to get the PATH to be respected like using bash -c or bash -l -c and even sourcing the bashrc before running the pnpm command. Any thoughts?

[plain dirge]

Do you want to pair for a bit? If you are around lets meet in dev-audio channel

[ember moss]

So it feels a little dirty but @plain dirge and I got something working. Using bash -c -i pnpm install worked. Something about loading bash in interactive mode properly sourced everything that needed to be and the PATHs resolved. If anyone has a more proper, cleaner method of doing this let me know, because I'm sure there are some bugs hiding in using -i. That being said it works for now

[karmic vapor]

That seems pretty clean to me.

[ember moss]

Any potential gotchas in using bash in interactive mode?

[lost robin]

nope, in fact if you use the same image you build from the Dockerfile you've shared via docker run you should see the same error

[plain dirge]

The only reason I say its dirty is because sometimes the programs you are running will pause for input if they think they are running in an interactive terminal. This can lead to some weird bugs.

[lost robin]

yeah agree. I guess the challenge here is that devcontainers are not the same as build containers IMO. DevContainers are intended to be used interactively as a developer sandbox. That's why you probably have things like nvm installed in there. When you're writing CI / Production grade Dockerfiles, you generally don't want that

[ember moss]

Yeah understood on that. I always desire one source of truth and think if I have to specify version numbers and Dockerfile scripts across two places they will fall out of sync.

[lost robin]

that's why you're here :). You can handle that in a programmatic and composable way now instead of gluing Dockerfile and bash scripts for that 😉

happy to help you designing that when that moment comes @ember moss