hi guys, i’ve been tasked with evaluating dagger to potentially move away from using github workflows for our ci/cd pipeline (mostly because debugging github workflows is not unlike pulling teeth without anesthesia). I am struggling to figure out how exactly dagger fits into this. For example, our current workflow uses push as a trigger and dorny/paths-filter@v2 to figure out what has changed and what components should be built (as opposed to building everything every time something is pushed to main branch). Can this functionality be achieved with dagger or do i keep the “skeleton” of the github workflow and replace the actual build steps only with dagger code ?
#Integration with github workflows
forest lava
solid harness
Welcome @forest lava ! At the moment we recommend the latter: keep the skeleton, replace actual pipeline logic (build, deploy, test etc) with dagger code
Over time the dagger code can "eat" more and more of the yaml. There are obstacles along the way that force to keep you more github yaml than you would like, but we are working on solutions to those
Eventually your GHA yaml becomes a simple shim that contains only the logic necessary to spawn your dagger code
dull pilot
Just to make things a little more explicit, dagger can already do filtering and only execute step of what has changed. But again, you don't have to break everything, and you can replace little by little, and get more and more advantages of using dagger the more you convert.
forest lava
Just to make things a little more explicit, dagger can already do filtering and ...
so dagger can in theory run on branch pushes, find what changed and build the required part without relying on github yml ?
dull pilot
Yes. Even in a monorepo environment, you could just decide that for a specific step, you only send a subdirectory of your source and only include/exclude some file/dir patterns.
Then, it's the dagger engine system that will figure out if anything included in this step has changed.
Some simplified example
src1 := client.Host().Directory("./subdir1", dagger.HostDirectoryOpts{Exclude: "node_modules"})
src2 := client.Host().Directory("./subdir2", dagger.HostDirectoryOpts{Include: "./src"})
nodeBuild, err := client.Container().
From("node").
WithDirectory("/nodeapp", src1).
Exec([]string{"node", "build"}).
ExitCode(ctx)
cbuild, err := client.Container().
From("cppimage").
WithDirectory("/cppapp", src1).
Exec([]string{"make"}).
ExitCode(ctx)
if nodebuild != 0 || cbuild != 0 {
os.Exit(-1)
}