github

[final heron]

I spent some time playing with gale for running a github workflow on my repo locally. It took me a while to understand what to invoke on the CLI, so I opened this issue: https://github.com/aweris/gale/issues/33

[heady stream]

exciting! (didn't know about this project)

[quasi bloom]

Hi, is there any idea in how to use docker based action within gale?

[orchid panther]

Hey, currently gale only support node based actions and bash scripts. Supporting other types in our road map.

You can create an issue at https://github.com/aweris/ghx (executor for github actions which gale use) for this scenario. We can prioritize it 🙏

[orchid panther]

This is relatively new and made early dev release recently. I'll send more details and updates to this channel

[final heron]

my 2c, I would migrate the ghx repo to a subdirectory on the gale repo. Tracking two repo is hard and is confusing (for a new user like me). IMO ghx looks more like an internal gale component than a standalone project.

[orchid panther]

I understand your concerns about the difficulty of understanding and tracking two repositories with insufficient documentation. However, the internal part of ghx, which is part of gale, needs to be executed inside the Dagger container as a binary. Keeping them in the same repository would require adjusting gale as a monorepo or something similar.

On the other hand, keeping them as separate repositories makes management and releasing easier.

I must admit, I chose a faster and easier solution for the initial kickoff. However, I'm open to reconsidering or we could discuss this further when we have dagger plugin support. I think ghx is perfect candidate for a plugin

[jovial oxide]

just to help me understand the workflow of how you set up ghx:

first you create a run (some container to hold the rest of the things)
Then you create a workflow that goes inside the run
Then you create a job that goes inside the workflow
Then you create a step that goes inside the job

Then you tell it to execute after you configure everything ghx run.

Correct?

[orchid panther]

you can think ghx as a job run stage in GHA. It focuses on sequential steps. You can configure them with workflow/job combination from repository or you can add them individually.

Yes mostly what you described is correct only difference would be how to configure the ghx. I'm adding some examples below.

1- Without using workflow or job:

• ghx with step --uses=actions/checkout@v3
• ghx with step --uses=actions/setup-go@v2 --with=go-version='1.20'
• ghx with step --run="go test ./..."
• ghx run

This commands will configure actions for checkout , step-go and executes go test ./... command without need any workflow or job

2- Importing job configuration directly

• ghx with job --workflow .github/workflows/test.yaml --job="go-test"
• ghx run

3- Combine / Override job config:

• ghx with job --workflow .github/workflows/test.yaml --job="go-test"
• ghx with step --id checkout --run="override the step with id checkout" --override
• ghx with step --run="go build ."
• ghx run

Now this example imports job config, overrides step with given id and adds final go build step at the end.

Hello everyone,

I am currently keeping weekly progress reports on my project and planning for the upcoming week. Starting this week, I will be posting these reports every Monday on this channel. Please feel free to provide feedback or ask any questions.

Last week,

• Added the aweris/ghx/pkg/expression package to resolve expressions in GitHub actions.
• To unblock actions that use expressions for default values, integrate the expression package into the custom action's default values.
• Version 0.0.2 of aweris/ghx has been released, including some fixes and changes.

This week's goal is, to make gale and ghx documents well and add examples to make it easier to understand.

[final heron]

Looking forward to see GALE's demo @orchid panther at the Dagger community call! 🚀

(if other people want to see it: https://dagger-io.zoom.us/webinar/register/4416686668409/WN_USQjVBGXT0SWhNMvqYVvCA#/registration) - starting now!

[strange knot]

That was a great demo @orchid panther , thank you

[orchid panther]

Hey everyone,

Last week was relatively relaxed for me as I focused on conducting demos and testing. There wasn't much happening in terms of new features.

highlights of the week:

• https://github.com/aweris/gale-demo This repository is a demo for Gale and includes examples of:
  • using Gale as a CLI
  • using Gale as a library with magefile
  • executing gale in a Github Action Runners with public 'ubuntu-latest' runners as well as self-hosted runners.
• We can easily run gale on Github actions. This provide us with greater flexibility for using gale.

lowlights of the week:

• Not using issues and missing documentation makes harder to communicate.
• The initial benefits of the polyrepo approach are getting smaller. The polyrepo approach is still easier and simpler to use, but the monorepo approach is getting more meaning full to keep everything in a single place.
  • https://github.com/aweris/gale/issues/37 , this issue claused by polyrepo approach

This week, I'm planning to focus on give monorepo one more chance and create more issues from roadmap. Then I would focus on executing one real world workflow with gale

[frosty idol]

Thank you again @orchid panther for the summary of Project Gale at the last community call. For anyone that missed it, you can find the demo snippet here: https://discord.com/channels/707636530424053791/1120939230680600636

[orchid panther]

Hey everyone,

I hope you have a great week ahead! Although we experienced some delays due to minor details last week, we still have a couple of promising highlights.

Summary of my week,

• I expended all of my allotted time on an experiment involving a monorepo, but unfortunately, the results did not meet my expectations. To make progress in this area, it seems prudent to break down my efforts into smaller, more manageable pieces and approach the task with smaller steps.
• The test went better than I thought, but some issues block the progress

New issues:

• https://github.com/aweris/gale/issues/42
• https://github.com/aweris/ghx/issues/14

Highlights:

• Project Zenith and the possibility of running gale as an extension
• I've found that combining Dagger Services with Github Artifact Service, which allows me to upload and download artifacts, works seamlessly and has been inspiring me with new ideas. (PR: https://github.com/aweris/gale/pull/44)

Lowlights:

• Small details and issues slowed down feature progress
• The size of our testing scope is still small. We're still working on the happiest path.

This week's plan is finalizing, cache and artifact services to resolve https://github.com/aweris/gale/issues/42 then I'll focus on improving the UX of the gale

Just created a issue with my initial thoughts on UX improvements https://github.com/aweris/gale/issues/45 to kick of the work

[orchid panther]

Hey everyone,

Unfortunately, I didn't accomplish much last week as I devoted all my time to working on integrating services support on 'gale'. I find myself preoccupied with small issues most of the time.

Highlights:

• Finally merged https://github.com/aweris/gale/pull/44
  Lowlights:
• Creating a solid picture and roadmap is harder than I thought

This week, I'm hoping to work on second half of the https://github.com/aweris/gale/issues/42 than jump on https://github.com/aweris/gale/issues/45

[uncut cove]

Good job keeping this one ticking along. Sometimes, it's the small additions that make all the difference 💪

[orchid panther]

I'm still working on UX changes on gale I'm skipping this weeks moday update and hopefully I'll come back with a PR including all changes I'm currently doing 🙏

[frosty idol]

Someone has a question on twitter from the project gale post. If possible can an expert from this channel answer? Please and thank you! https://twitter.com/dagger_io/status/1681091500019425280

Dagger (@dagger_io)

Project Gale, GitHub Action Local Executor, gives you local execution capabilities, simplifying testing and debugging of GitHub Actions. Check out the demo here: https://t.co/yWfiKznvBp

Twitter · Jul 18, 2023, 12:00 AM

[orchid panther]

Hey everyone!

After a couple of weeks of silence, I wanted to provide a quick update on where we're at. We've had some ups and downs, but we're still moving forward. During this period, we've made some structural changes to create a more robust codebase before pushing new features.

Some of the highlights:

• Version 0.0.3 has been released! This marks the first official release of Gale after the POC demo. This release can be considered our first alpha version.
• New interactive demo available at https://github.com/aweris/gale/tree/main/demo that includes several predefined scenarios.
• Now that most of the structural issues and questions have been resolved, we can shift our focus to what we want to achieve rather than how to achieve it.
• Now you can use the list and run commands to execute gale without having to check out the repository code first.

[orchid panther]

Hey everyone!

New release v0.0.4 now available! Includes updated dagger to v0.8.2 and actions/cache support in gale. Check out release notes at https://github.com/aweris/gale/releases/tag/v0.0.4.

[orchid panther]

Hey everyone!

New release v0.0.5 is now available! Includes updated dagger to v0.8.4 and basic docker:// support in steps and custom docker actions support. Check out the release notes at https://github.com/aweris/gale/releases/tag/v0.0.5.

[prime sail]

@orchid panther Can gale be setup for macos build simulation? I'm having some real troubles getting macos builds to work where linux are fine.

[orchid panther]

@prime sail Not sure if I understand completely. Are you trying to use macos runners in Linux?

[prime sail]

https://github.com/hofstadter-io/hof/actions/runs/5992905676

I'm having failures that only occur on the mac build in GHA, I'd love to be able to debug this without pushing commits to test, wait, retry

the dream that dagger + gale gives us

but, I can imagine mac simulation locally might be out of reacch currently

[orchid panther]

Yes, exactly. It’s out of scope at the moment. Currently, we’re focusing on Linux runners

But i’ll take note and look at it if it’s a low hanging fruit

Can you create an issue in Gale for this to make sure we don’t forget

[prime sail]

sure thing

https://github.com/aweris/gale/issues/103

[prime sail]

How do I run the docs/docs workflow/job here?

┃ Workflow: dagger (path: .github/workflows/dagger.yml)  
┃ Jobs:       
┃  - inception                                                                                         ┃  - hof                                                                                               ┃                                                                                                      ┃ Workflow: docs (path: .github/workflows/docs.yml)                                                    ┃ Jobs:                                                                                                ┃  - docs

I've tried gale run docs docs and gale run .github/workflows/docs.yml docs, but both error, the second with not found, the first Error: new client: failed to run container: : fork/exec /usr/bin/docker: exec format error

[orchid panther]

can you share how did you execute the command?

when I run gale run --repo hofstadter-io/hof docs docs which executed command on default branch. I received error :

137: [119.0s] ┃ Setup 137: [119.0s] ┃ ┏ 137: [119.0s] ┃ ┃ starting: prettier v0.6.8 137: [119.0s] ┃ ┃ formatter prettier is not ready 137: [119.0s] ┃ ┗

[orchid panther]

gale run docs docs is correct on if you are inside of the root of the your repository or you can execute the command I just send to clone repository with default branch and execute workflow / job on top of it

[prime sail]

I'll try again, the error you see is because we are using docker from inside our workflows. Have you crossed that bridge yet?

[orchid panther]

I checked the workflow and didn't see a reason for not working. This week I'll look closer at workflows from hofstadter-io/hof and try to run some of them. Probably, we need to do some fine-tuning in the gale.

An issue would also be great for this one, with some examples of what we should see in the logs.

[prime sail]

We have some dagger-in-dagger and dockerd-in-dagger examples we run for sanity checking in our workflows

check the dagger/inception wf/job

some may fail because of missing credentials for pulling private repos (used only for testing) (anything in the hof mod testing

[orchid panther]

137: [57.3s] ┃ ┗ 137: [57.3s] ┃ Complete job 137: [57.3s] ┃ ┏ 137: [57.3s] ┃ ┃ Complete job="inception" conclusion=success 137: [57.3s] ┃ ┗ 137: [57.3s] ┗ 137: exec /usr/local/bin/ghx run 3 DONE

it was successful and didn't see any particular error in the execution.

gale run --repo hofstadter-io/hof dagger inception this is the cmd I used

[prime sail]

This could be related to limitation on my local machine around nested virtualization & kernel hardening

I don't think I can run everything locally anyway

[orchid panther]

hmm, it seems you're having an issue connecting the dagger engine from inside of the container. I really didn't understand why it's giving fork/exec /usr/bin/docker: exec format error it's an interesting error to see

[orchid panther]

Hey everyone!

The new release v0.0.6 is now available! This release introduces:

• The jobs.<job_id>.steps[*].shell parameter now supports bash, python, and sh.
• Using the --runner flag to replace the default runner image.
• A new experimental command called validate allows for preflight checks to identify potential issues. This command is subject to change or removal at a later stage and is currently hidden and experimental. (e.g. gale validate --repo hofstadter-io/hof dagger inception)

Checkout release notes at https://github.com/aweris/gale/releases/tag/v0.0.6

@prime sail , can you try to run new validate command in your machine. I would like to see if gale misses something in your local machine

[shut kindle]

@orchid panther on main, dagger run ./gale validate --workflows-dir ci/workflows ci/workflows/lint.yaml golangci-lint:

█◀──╯ [0.64s] ERROR exec which node
┃     panic: fork/exec /usr/bin/which: no such file or directory                                                                        
┃                                                                                                                                       
┃     goroutine 1 [running]:                                                                                                            
┃     main.shim()                                                                                                                       
┃         /app/cmd/shim/main.go:336 +0x11c8                                                                                             
┃     main.main()                                                                                                                       
┃         /app/cmd/shim/main.go:62 +0x5d             

(run doesn't work either, ERROR reslove image config for ghcr.io/aweris/gale/services/artifactcache:v0.0.0-dev 😛 )

(trying things on the gale repository itself 😅 )

(If that helps, I am running NixOS, and my DOCKER_HOST is a remote one — ssh://…)

[orchid panther]

Did you install gale using go command?

[shut kindle]

well, go build ./cmd/gale and then dagger run ./gale … 🙃

[orchid panther]

Ah, it’s normal to too see v0.0.0-dev then. I’ll document how to use development version.

[orchid panther]

We use the internal version 'gale' to select versions for other services and tools. For easier development, we utilize a local register and custom dagger engine. It's normal to receive an error without changing https://github.com/aweris/gale/blob/fd44102b7d7239d8c793e75e622b9601191f7fe3/internal/version/version.go#L11.

I am going to create an issue and document the steps on how to run gale from the repository. In the meantime you can:

• download v0.0.6
• changing https://github.com/aweris/gale/blob/fd44102b7d7239d8c793e75e622b9601191f7fe3/internal/version/version.go#L11 manually to version v0.0.6
• run ./hack/mage dev:engine:start && eval $(./hack/mage dev:engine:env) && ./hack/mage dev:tools:publish dev:services:publish to start engine, update env and publish dev versions of the components. Then you can use local cli

[orchid panther]

Yes, this could also create an issue. I'll test it to validate but gale respects dagger's runner host and session variable. If you pass these as env var to gale, it should work fine

[shut kindle]

ok good to know, I'll try the ./hack/mage way

[orchid panther]

🙏 , let me know the results. I'll make some improvements about it next week.

[shut kindle]

ah it won't work with a remote docker engine though 🙂 (mounting local folder 😛 )

[orchid panther]

if you can run dagger, you should be able to use gale

[shut kindle]

yes but ./hack/mage dev:engine:start won't work

"docker run -d --name dagger-engine-v0.8.4 -v /home/vincent/src/dagger/gale/internal/mage/engine.toml:/etc/dagger/engine.toml --privileged registry.dagger.io/engine:v0.8.4" <- the -v cannot work, the folder is local, the daemon is remote

(I can work my way around this ;))

[orchid panther]

ahh yeps

[shut kindle]

(I have this setup "kiind-of" on purpose 😛 )

[orchid panther]

can you run go run ./demo/ --run --auto --auto-timeout 1s

it'll download and execute same command in demo format automatically.

https://github.com/aweris/gale/tree/main/demo here more info about demo

[shut kindle]

Yeah that works it seems 😉

ah wait nope

┃ 1: starting engine                                                                                                                    
┃ failed to list containers: fork/exec /usr/bin/docker: no such file or directory                                                       
┃ 1: starting engine [0.45s]                                                                                                            
┃ 1: connect ERROR: new client: failed to run container: : fork/exec /usr/bin/docker: no such file or directory                         
┃ Error: new client: failed to run container: : fork/exec /usr/bin/docker: no such file or directory  

but that's normal as well 😛

😁

[orchid panther]

Can you set _EXPERIMENTAL_DAGGER_RUNNER_HOST ? before running the example

it can still fail but I want to see if we workaround this issue

[shut kindle]

oh yeah I forgot to setup this one probably

(#etoomany-terminal-opened)

alright, I broke something, I'll restart from scratch, but can't right now

thanks for the help @orchid panther 👼 🤗

[orchid panther]

Your welcome. Please ping me if you need any additional help. I'm curious about the results. I think 'gale' needs a bit more optimization in this regard.

[orchid panther]

Hey everyone, new release v0.0.7 is up. Checkout change log: https://github.com/aweris/gale/blob/main/CHANGELOG.md#v007---2023-09-04

[strange knot]

@orchid panther with your permission I'm going merge this into "Dagger and... #1122941618056614048". This will help 1) make your work more discoverable in the context of Dagger and 2) fight channel sprawl 😛

[orchid panther]

Sure, it makes sense 🙏

[strange knot]

Done, this channel is now #github

[orchid panther]

Great, then as the first message! new release v0.0.8 is up. Check out the change log: https://github.com/aweris/gale/blob/main/CHANGELOG.md#v008---2023-09-18

Highlight of this release, gale run can execute workflows instead of single jobs, and job became an optional parameter.

[long vector]

Fixed the indentation in the GHA example:

https://github.com/dagger/dagger/pull/5822

Also, I think adding if: always() is necessary, to properly run docker stop even after a failure.

[primal wolf]

Hello, i'm trying out dagger on my small app.
Big Problem :
Once I try to push the image from the runner, I get the following 403 error :

panic: input:1: host.directory.dockerBuild.publish failed to export: failed to push my-region-docker.pkg.dev/my-project/my-repo/my-reg failed to authorize: failed to fetch anonymous token: unexpected status from GET request to https://xxx/v2/token?scope=repository%3Axxx%2Fxxx%2Fxxxx%3Apull%2Cpush&service=xxxx-docker.pkg.dev: 403 Forbidden

I'm working on Google Cloud, locally it pushes the image and it works fine, but in the runner it doesn't work, even with Owner role json_creds (lol)

Has anyone faced this yet ? I'm using the JSON method to authenticate : https://github.com/google-github-actions/setup-gcloud#workload-identity-federation-preferred

[orchid panther]

GitHub - google-github-actions/setup-gcl...

[pure glacier]

Wondering how fello Daggernauts handle accessing different private repos when running in Github Actions vs locally?

We have a set of local pipelines that use SSH for authorisation when running locally, but I'm now struggling to think how best to do things in Github Actions. It feels like the options are:

• Create deploy keys for each repo I need to access, and use the ssh-agent action?
• Create personal access tokens, but then I'm tieing the actions to my personal account...
• Create a Github App to handle generating the tokens dynamically in workflows

Do I have that right?

[strange knot]

@pure glacier could you store the private ssh key as a Github secret?

[pure glacier]

I think yes, if I want to keep using SSH auth. My headache then becomes ensuring those keys are setup, rotated etc for each repo. Especially as Github requires a deploy key to be unique per user and per repository! Hence @orchid panther 's suggestion of creating a 'bot' user for the org instead, so we can use organization-level permissions instead of deploy keys

[strange knot]

(entering the realm of peronal opinion and not Dagger best practices...)

Personally I would avoid if possible, locking myself into Github-specific constructs like "deploy keys". I'm assuming there's a way to set a regular secret org-wide; then rotate that once per org as you see fit; and if that secret happens to be the contents of a private ssh key... well that's none of Github's business

Organizing your whole security infrastructure and practices around individual git repos... That only makes sense if you're a company that sells git repo hosting (again - my personal opinion)

[pure glacier]

Opinions much appreciated 🙂

[echo loom]

We've moved a lot of our secrets into 1password, making use of the op command (and potentially direct API calls down the line). Seems to work well - we started off using AZDO Variable Groups - but once I realised they weren't version controlled, or tracked - I steered our teams away from that - Azure Keyvaults get used for some things azure specific, but 1Password is looking better.

[uncut cove]

Another fan of 1Password 🙌 My intention is to convert this to a Zenith module, and make it non-Fly.io specific, perhaps just in time for Christmas 🎁 https://github.com/thechangelog/nightly/blob/f5456abb73b948c84643e1816f2caa1be949a0b7/pipeline.go#L160-L208

[ionic sable]

Does this support docker? I tried it real quick but my workflow assumes that docker is available and it couldn't find the docker socket

[orchid panther]

which version and command did you try?

[ionic sable]

dagger call --focus=false run --repo acorn-io/acorn --branch main --workflow test --job integration --token $(gh auth token) sync

[orchid panther]

ahh you're using docker itself not docker action?

[ionic sable]

Correct, running debianmaster/actions-k3s@v1.0.5 which does /usr/bin/docker run -d --privileged --name=k3s-v1.27.2-k3s1 --expose=6443 -e K3S_KUBECONFIG_OUTPUT=/tmp/output/kubeconfig-v1.27.2-k3s1.yaml -e K3S_KUBECONFIG_MODE=666 -v /tmp/output:/tmp/output -p ┃ 6443:6443 -p 80:80 -p 443:443 -p 8080:8080 rancher/k3s:v1.27.2-k3s1 server

this is a pretty advanced use case. running k3s in docker in dagger (in docker?)

[orchid panther]

I removed the docker config during zenith migration and replaced it using ContainerWithExecOpts{ExperimentalPrivilegedNesting: true}

however, I make a similar thing for the KinD module. I think I can re-introduce the supprt

let me have a quick look.

[orchid panther]

do you have any runner images you used with necessary dependencies?

added docker-in-dagger support and it seems it failed because of the missing dependency in default runner image

[ionic sable]

Not sure I follow. In GitHub we just use the default Ubuntu environment.

[pure glacier]

As a follow up, thanks for everyone's input. The approach I settled on is:

1. Use a Github App to authorise access to the private repos we need (terraform modules, etc...)
2. Setup GHA to checkout the repo via that github app
3. Modify the dagger pipelines to pull the repository in using client.git(...) if the folder isn't already present, which allows local users to just pull in dependencies with their existing git+ssh setup without even thiniking about it 🙂

[storm pulsar]

Hi @light tree I'm a newbie of Dagger. I watched your great session KubeCon last month. https://youtu.be/L4hvYWknBfw?si=ynVg3U9QsDvNwirU Thank you for sharing this great insight at first😀 then I got motivated to do a hands-on of Dagger by myself. During this hands-on, I got an error when running ci on Github Actions by using greetings-api repository without any changes. Do you think is there anything I should do for running ci pipeline?

[light tree]

Hey 👋 Glad you enjoyed 🙂 I think what you probably need to update are these two lines in your repo: https://github.com/kpenfound/greetings-api/blob/main/ci/main.go#L10-L11 . It looks like it's trying to checkout kpenfound/greetings-api but using the GITHUB_SHA from your run

[storm pulsar]

Thank you for the reply. I could resolve the issue. After fixing the code based on your advice, I also noticed that I had missed to update a unit test after changing the greeting message to my own one. 😫

[nova wadi]

i am meeting one of dagger.io's developer advocate in about 12 hrs time. but in any case i like to ask about this use case.

I have an existing python package up on pypi and i use github to host and github actions to run CI stuff like run tests, generate coverage files, and then coverage reports.

i was wondering if it's a good use case to convert my existing (and working) github actions yml file into something that works with dagger.

and if so, whether anyone would be interested to learn how i made it work.

If people are interested, then i can work with one of the developer-advocates to post it as a guide somewhere in dagger.io's guides

[drowsy pelican]

Hello I need to do a POC of dagger at work. Anyway to temporarily use github actions cache instead of dagger cloud?

[delicate comet]

Not really, but if you already have a Dagger Cloud account, we can talk about extending the trial period until your POC is done if that's something that could work for you 🙏. cc @uneven widget

[finite ermine]

Does anyone know the best way to leverage the GitHub CLI with Dagger? Typically this is done fairly easily in GitHub actions and I would like to mock up some of our existing uses of GitHub CLI into Dagger

[strange knot]

My guess is that you could easily run the Github CLI in a container, orchestrated with the Dagger SDK of your choice

In fact that would make for a great module 🙂

[finite ermine]

Thanks @strange knot is there a resource you would recommend on getting up and and running with modules?

I found this actually, which I think is what I was looking for 🙂 https://docs.dagger.io/zenith/developer/python/419481/quickstart

[strange knot]

Modules are still in development under the nickname "Project Zenith" (stable release planned soon, in the next few months) but already actively in use by power users 🙂 You can see the dev docs at https://docs.dagger.io/zenith/ and discuss with fellow module devs on #daggernauts

[strange knot]

Looks like it, yes!

[long vector]

@finite ermine https://daggerverse.dev/mod/github.com/sagikazarmark/daggerverse/gh@ef6b5bf932277ffc4b7e1ea9fd9a1c26a2780da3

[long vector]

Is there an example for pushing to Git from Dagger (or even opening pull requests)?

[uncut cove]

I remember @dusky raven & @fast lintel having something for this. Not sure if it's public. It should be!

[dusky raven]

We have this module we are using internally that uses go-git inside Dagger to push to a repository: https://daggerverse.dev/mod/github.com/matipan/daggerverse/image-updater@dc37011c6578e44b83e61976ca92336350d96b02

[long vector]

That's the exact same use case I have!

[quick hornet]

Hi All,
I have really noob questions.. As a Service Platform, we would like to have a centralized repository to develop our functions.
In Github Actions, how each engineering teams can access to functions that we created via dagger? Should they checkout the centralized repository in their repo or is there any there suggested way?

I also would be really happy to get some suggestion how the folder structure looks like.

[strange knot]

Hello! The dagger CLI takes care of checking out the module's repository.

• To directly call a function from a module they will dagger call -m MODULE_URL FUNCTION_NAME
• To install the module into their own module: dagger install MODULE_URL

Repo layout: completely up to you, one common layout is the "daggerverse repo" layout: each module is a directory in the repository root. Perhaps some modules have sub-modules. So, very simple.

but you can also embed your dagger modules anywhere in an existing repo

[quick hornet]

thanks @strange knot , I ll try to make some PoC and probably come with some silly solutions

[quick hornet]

I have another noob question.. For instance, i would like to develop one think for Docker build, push, sign and etc, but i want to keep it in our internal private repository.
What should I do? Should i develop everyhing from scratch or what?

[strange knot]

you can have one module depend on another module. The dependency can be a relative path in the same repo, eg. dagger install ../path/to/utils/build

[quick hornet]

@strange knot for instance I would like to develop a module for docker build, scan, push and sign.. Can I use any module has already build,scan,push and added only sign functionality?

[strange knot]

sure!

[quick hornet]

@strange knot is there any disadvantages to create /src or modules folder and add all of the modules under this folder?

[obsidian mortar]

Hey guys, new to dagger , would love to hear how to connect to a private repo within dagger ssh key

[obsidian mortar]

Mainly from a public repo to an org private repo

[delicate comet]

hey! welcome! you'll probably find this thread useful https://discord.com/channels/707636530424053791/1224350244620468345.
Let us know if that's what you're aiming to do 🙂.

happy to hear more about your use-case!

[tight path]

Hello, we’ve a monorepo with several Terraform workspaces (but it could also be various Go apps or other things) and we’d like to run Terraform on PR for every workspace impacted by this PR, if possible in parallel. How would you do this? Would you create a shim GitHub workflow for each workspace, filtering on its path, and calling a shared workflow which would call Dagger? Would you handle the path filtering in Dagger?

[blissful owl]

That sounds pretty reasonable to me. I'd probably specify a shared Dagger function, but provide the path as an argument to the function.

[woeful skiff]

Using tsx in GitHub Actions

[woven socket]

Hi all, I was wondering if somebody could give me a hint before I dive deep in documentation and google/openai results on this. I am just beginning to look at dagger, testing it with python/pytest and github actions. My goal is really to be able to run the same dagger code on my laptop and on github. I now have a simple
test = runner.with_exec(["pytest","-v","tests"]) in my dagger code
But I don't see the output of pytest in the github action logs.
It's called as such in the action workflow

        run: dagger run python ci/test_base.py```
Do I have to do some explicit redirection of sdtout/stderr?

[light tree]

Hi all, I was wondering if somebody

[woeful skiff]

how do you specify the location of the dagger.json file in a GHA?

[strange knot]

You point to the module directory (which contains the dagger.json) with -m PATH

If -m is not specified (I don't know how to specify it in the latest GH action), then the current workdir is used, same as for a git repo: walk up the directory tree until finding a dagger.json

cc @light tree who has more practical experience with the "Dagger for Github" action

[light tree]

with the module: parameter. So it could be module: github.com/foo/bar or module: ./ci

[woeful skiff]

so module can be a subdirectory of the repo

[light tree]

yup! it could even be a subdirectory of another repo 😄

[woeful skiff]

Is it possible to cache dagger's python or other requirements inside GHA? Last year's discussions indicate not but I wanted to verify if the situation has changed.

[woven socket]

I have a working dagger code running in github action. When using the same dagger run python ci/test_base.py on my macbook M1, it doesn't work.
I get this

venv/lib/python3.12/site-packages/dagger/
dagger.py", line 71                                                                                      
    if type(item) <> ldict_node:                                                                         
                  ^^                                                                                     
SyntaxError: invalid syntax                                                                              
exit status 1

[woven socket]

This dagger call also has a weird bug that removes the cursor of my zsh/iterm console. I haven't figured out what's happening there yet...

[strange knot]

maybe different python versions on your local dev & ci?

[woven socket]

I use 3.12 everywhere I believe. I'll double check it's indeed the case.

[thorn blaze]

I have the same issue. See here: https://github.com/dagger/dagger/issues/7160

[delicate comet]

👋 just checking if you were able to resolve this

[woven socket]

added a few more details from my side

[woven socket]

thanks for checking. Didn't get a chance yet.

[woven socket]

so I double checked that python is 3.12.3 everywhere. In the venv and in the image used by github action.

so I double checked that python is 3.12.

[hard cave]

TOTALLY new to Dagger and working on wrapping my head around it. I have an issue I can't understand - re: github Actions and an error. Can anyone assist? Here's the github worfkflow yaml. on: workflow_dispatch

jobs:
hello_world_job:
runs-on: [ubuntu-latest]
steps:
- name: dagger run
uses: dagger/dagger-for-github@v5
with:
workdir: db-service
verb: run
args: node build.js
cloud-token: ${{ secrets.DAGGER_CLOUD_TOKEN }}
version: "0.9.3"
- name: dagger call
uses: dagger/dagger-for-github@v5
with:
module: github.com/shykes/daggerverse/hello
verb: call
args: with-greeting --greeting Hola with-name --name Jeremy message
cloud-token: ${{ secrets.DAGGER_CLOUD_TOKEN }}

[orchid panther]

TOTALLY new to Dagger and working on

[long vector]

If anyone needs module build ideas:

• upload artifact to Actions
• upload SARIF to GitHub Security

Most of these are already implemented in JS libs, so you can probably reuse some of those.

[strange knot]

we should make a channel "module ideas" or something like that 🙂

[orchid panther]

I already have upload artifact to Actions part in gale I should extract it to my daggerverse. 😄

[long vector]

That would be cool!

Right now I'm kinda forced to use the builtin action and the piece of crap it is, you can't upload multiple artifacts separately with it.

[robust smelt]

hey folks, has anyone managed to implement a cache strategy on github actions? Im testing out some ci tasks I implemented with go-sdk but of course runners are ephemeral no cache out of the box

not using the dagger action, btw

[restive smelt]

I'm trying to think about how to pass outputs between steps in our GHA pipeline while I convert the existing workflows to use dagger (trying to maintain compatibility for now). It looks like I probably can't use the dagger-for-github action since it doesn't provide any outputs and I'd need to handle parsing the output myself anyways.

Also, if I have something like

type MyContainer struct {
  RepoAddr string
  Tag      string
}

func (m *MyMod) BuildContainer() MyContainer {
  // do some stuff
  return MyContainer{RepoAddr: "fooRepo", Tag: "myTag"}
}

I know I can do dagger call build-container repo-addr to get the repo address, but is there a way for me to get the full struct as json so I can pipe it to jq? I

[strange knot]

I know I can do dagger call build-container repo-addr to get the repo address, but is there a way for me to get the full struct as json so I can pipe it to jq? I

Yes there is a PR by @carmine crystal for doing exactly that https://github.com/dagger/dagger/pull/7479

@restive smelt I 100% understand wanting to stay compatible, and retrofit Dagger into your existing workflow with minimal disruption. That is definitely "the Dagger way".

But something to watch out for: it's possible to go too far in that direction. If you find yourself jumping through hoops to pass a few json values around in GHA, it might be a sign that you should daggerize just a little more, and pass those values between Dagger functions instead.

It's a delicate balance, and in the end you know best the constraints of your project. Just wanted to put this on your radar, because it has bitten me in the past too 🙂

[restive smelt]

Thanks! Yeah agree I'm trying to tread that line carefully. Definitely something to keep in mind 😄

[strange knot]

What's cool about the incremental pattern, is that after a while, you've daggerized a few islands in your DAG, and then one day you realize that you can connect then, and simplify more than you initially thought. For example if you daggerized a build+push, then later the downstream pull+test+deploy, you may realize - wow I don't need that intermediary registry anymore, I can just build+test+deploy on the fly. Like collapsing 10 lines at once in tetris, it's a great feeling

https://tenor.com/view/tetris-t-spin-gif-24148025

[restive smelt]

Looking forward to it 😄 though my main concern with collapsing things that much is visibility for end users so they know what's going on. Though I know that's something dagger cloud can help with 😉

[restive smelt]

Is there a dagger equivalent of this https://github.com/docker/metadata-action somewhere? I don't see one with a similar name. Just wanted to check before I write my own 🙂

[rare iron]

great idea!

[strange knot]

Sorry for the slow answer. I don't think it's been done. But I agree it seems like a great idea 👍

[strange knot]

FYI https://x.com/solomonstre/status/1823803851696398475

Solomon Hykes (@solomonstre) on X

As an experiment, I made a Dagger pipeline to generate the Github Actions pipelines that run my Dagger pipelines. And wow! I was not ready for how much nicer it is. I'm never writing a GHA yaml file by hand ever again.

Twitter · Aug 14, 2024, 7:28 PM

[hushed eagle]

Can we see? 😅

[strange knot]

https://daggerverse.dev/mod/github.com/shykes/gha

[elfin pecan]

I want this one but for GitLab CI 😭

Just see from your module, Daggerverse is now support the example from code?

[strange knot]

let's do it 🙂

[strange knot]

yes 🙂

[green void]

Is something like this the most appropriate way to capture dagger output in github actions? Is it possible to capture directly from a dagger-for-github step? I've played around with a few things without success, but I'm also relatively new to both dagger and GHA.

    output=$(dagger do build)
    echo "dagger_output=$output" >> $GITHUB_OUTPUT

Specifically I'm pushing containers and having the function return the []string of addresses, per the cookbook.

[elfin pecan]

From https://github.com/dagger/dagger-for-github/blob/f2c001361568c9081f4a69e51930df182fe8c4fc/action.yml#L1. The action doesn’t return any result to caller. But it is possible to make that returns. 😃

[strange knot]

@green void if you take a look at my generator module above, it will expose 2 job outputs: stdout and stderr

I'm curious what use case you have in mind? In my experience, when you find yourself wanting to plug the output of a dagger pipeline into a non-Dagger pipeline, it's often easier to just collapse both sides into a single dagger pipeline, and do the composition in code instead of proprietary DSL

[strange knot]

YES! First successful generation post-push @light tree 🙂 https://github.com/shykes/gha/commit/a0e994ab38c80618ab02f9f1495480fc20b4e85f

It's still not awesome, because the generated workflows will still show up in all branches as soon as I pull back from them

so it doesn't abstract away the generation, the way continuation can do it in circleci for example

so maybe not worth the trouble... I don't know

[green void]

@strange knot I have a dagger pipeline that builds and pushes some containers to AWS ECR, returning the URI and SHA256. I was going to use them in a subsequent step updating image references in an ECR task definition. I've got viable workarounds at the moment so no big deal but you're probably right in that it would be best handled entirely in dagger.

[elfin pecan]

Do you have a plan to add custom registry mirror support?

I found this use-case during evaluate Dagger in my team that need to start the engine with custom registry mirror configuration. I think it would be great if the generator could accept the registry mirror and then start the engine.

[strange knot]

Sure, you mean a custom docker run command with extra flags right? Do you mind opening an issue with details? Seems very doable

[elfin pecan]

Yes, exactly. Will open an issue in around 2 hours. ✌️

[elfin pecan]

https://github.com/shykes/gha/issues/10

[elfin pecan]

Just make on schedule support! https://github.com/shykes/gha/pull/11 😄

[strange knot]

Awesome. Thank you, merged!

[elfin pecan]

@strange knot What do you think about adding fetch depth and submodule option to the with_pipeline? Fetch depth can be 0 by default. The submodule configuration can be a boolean to not clone (false) or clone it recursively (true).

[strange knot]

Makes sense, seems similar to the sparseCheckout option I already have

[elfin pecan]

Will submit a PR within today. 👍

[elfin pecan]

The fetch-depth PR opened https://github.com/shykes/gha/pull/13

[elfin pecan]

I experimented with daggerize the GitHub custom action (specific to the js custom action at the moment). The idea is convert all inputs to function arguments. And return all ::set-output values back to the caller.

From the screenshot, it is https://github.com/actions/typescript-action.

It would be fun if it can daggerize https://github.com/actions/stale. 😂

[ionic pawn]

Is there any benefit of using actions/checkout@v4 to check if the code is there if a dagger pipeline will fail without a directory argument anyway? (assuming one did not use optional arguments)
jobs: test: name: test runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v4

[glacial charm]

is there an example of a custom dagger engine running on a custom github runner, with company certs/customised toml with private registry?

[trail lintel]

and for context, ive already created a custom engine with certs, but it was local. there's not much guidance on how to glue that kind of thing with runners in a remote environment.

[sullen nexus]

Hi, I'm trying to use this dagger module https://github.com/shykes/gha but I don't find how to setup a strategy block (https://github.com/shykes/gha/blob/main/workflow.go#L114), someone knows how to do that ?

[strange knot]

It's not directly exposed at the moment. You can use withPipeline(pullRequestConcurrency) which uses it in a specific way. See https://daggerverse.dev/mod/github.com/shykes/gha@bf36a0b37a75882e4a985179f090d86d2ad1dfb4

May I ask about your use case? If it's not supported we could add it.

[sullen nexus]

gha :: Daggerverse

[woeful skiff]

GA error: Unable to process file command 'output' successfully.

[verbal belfry]

Quick one: how do I pass a secret to a call --param env:<what goes here>?

I haven't used Github actions more than about three times, and many years ago

I'm guessing call --param env:${{ secrets.SECRETNAME }} doesn't work since that'll resolve the secret rather than the actual name

Or do I just skip the env: bit and pass a string

[strange knot]

@verbal belfry allow me to answer with a dagger command 🙂

dagger -m github.com/shykes/gha shell  -c 'with-pipeline demo "foo --param SECRETNAME" --secrets SECRETNAME  | config | file .github/workflows/demo.gen.yml | contents'

[verbal belfry]

I'm going to blindly paste that in my terminal and see what happens

[strange knot]

It will dump a generated github actions config 🙂 And if you look towards the end, you will see COMMAND:

So in your case you would do:

run: dagger call --param SECRETNAME
env:
  SECRETNAME: ${{ secrets.SECRETNAME }}

[verbal belfry]

Yep that did it

Bit unfortunate it has to be that way, would get a bit long with a lot of secrets but it's good for now! Thanks 🤝

[verbal belfry]

Didn't work, neither did --param env:SECRETNAME

Too late to continue debugging, this is a weekend problem 🙂

[strange knot]

That's why we made a dagger module to generate those yaml configurations from a much smaller definition, all in code 🙂

[keen roost]

hey, hopefully a quick question… is there a way to use the github actions cache for dagger? E.g. in a workflow like this. https://github.com/bytes-zone/beeps/blob/ad5ba0e804283c9f9db66d16dc7871c5ed1af5e0/.github/workflows/ci.yml

I don't see anything in dagger/dagger-for-github that indicates how to do this. I'm hoping I've just overlooked it somehow!

[elfin pecan]

Cache volume is not supported but it can use with_directory or with_mounted_directory to mount github action cache to the container, I guess.

[uneven widget]

Oh very interesting, has anyone tried this before?

[keen roost]

Oof. Ok. Well thanks anyway!

[trail lintel]

that deffo looks very interesting

[trail lintel]

.WithGithubCache as a wrapped function to better describe it?

[light tree]

You could try it but there's a few reasons why you might not have a good time

• you'll have to explicitly pass the directories as arguments pointing at wherever they've been mounted in the runner
• you'll have to export the directories back to the host at the end of each pipeline because directories aren't bind mounted
• if you have working layer caching, you probably wont anymore because the cache will be slightly nondeterministic

[trail lintel]

that makes it sound like it's actually not worth the effort

point 2 deffo makes it a bit of a "urgh"

[strange knot]

note we're looking into making directory exports more convenient, for use cases like codegen. This 👆 would benefit since from the engine's point of view it's just another export

my thinking is that export() should be smart enough to not require an explicit path argument. it would just write back to its original source

[strange knot]

also smart enough to sync deletions without wiping the entire target directory

[umbral quarry]

Any issue to track on GitHub?
Please share

[proper igloo]

i guess this one: https://github.com/dagger/dagger/issues/7132

[strange knot]

That is one of them 👍 There is another that I have yet to write...

I'll do it now

[strange knot]

https://github.com/dagger/dagger/issues/9079

[green void]

I'm feeling the lack of caching in github actions and it's definitely an 'urgh' feeling 😂

[green void]

On a very separate note, having a bit of a weird one with file permission setting since updating to 0.15.1 in github actions.

I have a container I'm copying a file into.

WithFile("entrypoint.sh", src.File("entrypoint.sh"), dagger.ContainerWithFileOpts{Permissions: 0500})

Locally in 0.14.x and 0.15.x, works fine, 0500 set regardless of host file permission. Same in github action 0.14.x.

With github action 0.15.1, the file ends up in the container with 0644, which is the host permissions. I commited the file again with new permissions, sure enough, they passed through.

I'm seeing some messages with dagger engine startup that I wasn't seeing before either, e.g.

3   : │ │ starting engine DONE [8.5s]
9   : │ │ connecting to engine
10  : │ │ │ moby.buildkit.v1.Control/Info
10  : │ │ │ moby.buildkit.v1.Control/Info ERROR [0.1s]
10  : │ │ │ ! connection error: desc = "error reading server preface: command [docker exec -i dagger-engine-v0.15.1 buildctl dial-stdio] has exited with exit status 1, make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=Error: dial unix /run/buildkit/buildkitd.sock: connect: no such file or directory\nUsage:\n  dial-stdio [flags]\n\nFlags:\n      --addr string   The address serving the grpc api (default \"unix:///run/buildkit/buildkitd.sock\")\n  -h, --help          help for dial-stdio\n      --timeout int   The timeout in seconds for connecting to the grpc api (default 5)\n\n"

I took a punt that it might be related to ubuntu-latest on gha moving from 22.04 to 24.04, but doesn't seem so.

[strange knot]

would you mind filing an issue? 🙏 looks like a potential regression

[green void]

Sure thing, would you prefer on the dagger repo or the dagger gha repo?

[strange knot]

Dagger please... You'll get more eyeballs for sure

[green void]

Done. 🚀

[elfin pecan]

I tried github.com/dagger/dagger/modules/gha with the public repository but got no luck because it try to start dev engine from source. 😦

Is it possible to make it not to start the dev engine?

[elfin pecan]

Here is the error from runner log https://github.com/wingyplus/dagster-pipes-elixir/actions/runs/12518710875/job/34921619022?pr=13

[uneven widget]

I don't think this module is designed to be used as a library, its specifically made for testing dagger itself.

@reef crane maybe you have a different opinion!

@elfin pecan can you share more details about what you are trying to do?

[strange knot]

it should be fine, the module is designed to be usable by anyone; when it got refactored maybe we accidentally added some ties to the dagger repo?

In the original version you could configure the version of the engiy. @elfin pecan can you share the source for how you call gha?

[elfin pecan]

I have 2 reasons:

The first reason is to uses gha module to generate github workflow in my repo.

The second reason, I working on the gha generator to create .github more quickly. (The source is https://github.com/wingyplus/dagster-pipes-elixir/blob/9f3e858bea08b658a61644f31bc86437cd812a3a/ghagen/ghagen/lib/ghagen.ex).

[elfin pecan]

It is https://github.com/wingyplus/dagster-pipes-elixir/blob/9f3e858bea08b658a61644f31bc86437cd812a3a/.github/dot_github/lib/dot_github.ex

It comes from the PR https://github.com/wingyplus/dagster-pipes-elixir/pull/13/files

[ionic pawn]

Thank you. You helped pass a secret as a parameter to my function. I was doing dagger call build --token=env: ${{ secrets.<mySecret> }} and was not understanding why it was failing.

[reef crane]

Yeah it should work

The dev engine should be started if the version is not a semver version I think? If it's a path to a directory it'll try and build the dev engine to that

Kinda afk 🌴, so a bit hard to check the exact behavior

But that is weird you don't seem to set the version anywhere, so I would expect it to just get the latest dagger version

^ that's the intended behavior - if it's not it's a bug, issue/pr welcome

[reef crane]

@wide moon @uneven widget @uncut cove it feels like having separate versions of dagger and the github action for dagger is confusing (recent example here https://github.com/dagger/dagger-for-github/issues/168)
given that we match version numbers for cli+engine+sdks+helm+etc, i wonder if we should bring our github action into that same scheme?

unfortunately, we can't downgrade and make v0.X.Y releases in dagger-for-github anymore - but, we could create a new action, e.g. dagger/dagger-action or similar. WDYT?

[uncut cove]

I fully support this idea. We talked about this in the past, and I also think that we should definitely do it.

I find this more descriptive: dagger/github-action

WDYT @wide moon @strange knot ?

[strange knot]

I agree that tightly coupled versioning would be ideal (I argued for this in the past but can't find the record for it).

As for the name change and whether it's worth the disruption: I defer to @wide moon and @final heron .

[trail beacon]

Hi, I have a strange behavior when run dagger from github action with dagger/dagger-for-github@v7. I can look the job on dagger cloud only when is niot started from release. When the job start because I create a release, is not appear on dagger cloud ?

[trail lintel]

same action and is the cloud token provided in all your configuration? as from my understanding it shouldn't matter where it was triggered. if it runs, it should be visible in cloud if the token is passed and is correctly setup

[trail beacon]

I can access on dagger cloud when use direct URL printed from github action. But I can't found it from "CI traces" or "locale traces" menu

[elfin pecan]

Gha engine behaviour

[rancid cipher]

How do I run a Github Action for a local module in the folder samples/keys

Here is the CI job

jobs:

  test:
    name: sample-ci
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Test
        uses: dagger/dagger-for-github@v7
        with:
          version: "v0.15.2"
          verb: call
          workdir: samples/keys
          args: make-txt --home-dir-key=. --password-string=helloworld

Results in:

Error: unknown command "make-txt" for "dagger call"

I expected this to be straightforward from a local env, instead I am starting "Push&Pray" right away :/

[rancid cipher]

locally I am running exactly dagger call maketx ... from the given folder

[elfin pecan]

What's your function signature looks like?

[rancid cipher]

func (m *Gkey) MakeTx(ctx context.Context, homeDirKey *dagger.Directory, passwordString string) (string, error) { ... }

[elfin pecan]

The args should make-tx, not make-txt.

[rancid cipher]

Just noticed while writing.... a typo... so dumb
Thanks @elfin pecan 🙏 , also for acting as rubber duck 😭

[woeful skiff]

Has anybody run into Error response from daemon: No such image: registry.dagger.io/engine:v0.15.2 in response to exec docker inspect --type=image registry.dagger.io/engine:v0.15.2 in GA lately?

[strange knot]

yeah happened to me once. It's a "normal" error (checking for the need to pull the image) that should be hidden but maybe gets passed to the error by mistake?

are you seeing it frequently? would you mind filing an issue? 🙏

[woeful skiff]

a few times; I retried unsuccessfully. I'll PM you my trace. Is it safe to share here; visible only to your admins? Maybe the error is a red herring. But the test passes locally.

[strange knot]

yes it's safe.

[woeful skiff]

It went away after another retry. I'll see if it happens again.

[strange knot]

Well that "correct" error only appears when the engine at the CLI's version is not currently installed. So in practice it happens on the first CLI call after either 1) a fresh dagger CLI install, or 2) a fresh dagger CLI update, or 3) after manually wiping the engine container

The issue is that somehow the error is bubbled up as a top-level error, which should never happen

(its normal for it to be visible in the TUI as an inner execution detail)

[woeful skiff]

Hopefully my trace will help.

[restive pumice]

I have an issue running my pipeline in Github actions that relates to the way a tool (Terramate) queries the git commit history.

I need to checkout the repo with fetch-depth: 0 to get the full history so that Terramate can compare changes.

Running "natively" works fine but when I mount the checked out repository into my Dagger function, it fails! Any ideas why this would happen?

name: Dagger

on:
  pull_request:
    branches:
      - main

jobs:
  dagger:
    name: Run dagger functions
    runs-on: ubuntu-latest

    permissions:
      id-token: write
      contents: read
      pull-requests: write
      checks: read

    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          ref: ${{ github.head_ref }}
          fetch-depth: 0

      - name: Call Dagger Function to run pre-commit checks
        uses: dagger/dagger-for-github@v7
        with:
          version: "latest"
          call: pre-commit
          cloud-token: ${{ secrets.DAGGER_CLOUD_TOKEN }}

      - name: Configure AWS credentials via OIDC
        id: creds
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-region: ap-southeast-2
          role-to-assume: ${{ vars.AWS_PLAN_ROLE_ARN }}
          output-credentials: true

      - name: Call Dagger Function to run Tofu plan
        uses: dagger/dagger-for-github@v7
        with:
          version: "latest"
          call: plan --aws-access-key-id=env:AWS_ACCESS_KEY_ID --aws-secret-access-key=env:AWS_SECRET_ACCESS_KEY --aws-session-token=env:AWS_SESSION_TOKEN
          cloud-token: ${{ secrets.DAGGER_CLOUD_TOKEN }}

[restive pumice]

This also works fine locally btw

git repo not being mounted with full git history?

[trail lintel]

Does the 'featured' need updating to the dagger one? - I remeber this was merged into dagger but i was actually still using solomons one

[light tree]

Good question, I'm not sure but will check on it

[opal marten]

Is there a good way to use Dagger with GCP Workload Identity Federation from GitHub Actions?
I want to run some gcloud commands from Dagger.

Never mind, it was easy to get a tmp credentials file with GHA

[prime gulch]

does anyone here know how you would make something simliar to these two github actions with Dagger.io

https://github.com/kcl-lang/examples/blob/main/ci-integration/github-actions/app.yaml

https://github.com/kcl-lang/examples/blob/main/ci-integration/github-actions/manifests.yaml

trying to generate k8s manifests using KCL and commit them into main

[faint storm]

what is a good way to structure a GHA workflow that executes a repos dagger FNs optimally?

Naively i created a yml:

name: dagger

jobs:
  format:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Run Format Checks
        uses: dagger/dagger-for-github@8.0.0
        with:
          verb: call
          args: "format --source=."
          cloud-token: ${{ secrets.DAGGER_CLOUD_TOKEN }}
          version: 0.16.3
  lint:
# same as format
       args: "lint --source=."

# etc

Which seems not great

I was going to do a separate job that runs build-env function first, but then it seems to be eroding what dagger does where it dynamically discovers the nodes to run. What is the correct approach here?

[light tree]

Depends on the complexity and requirements of your pipelines, but in general I'd say if you can have one dagger function that does each of the required tasks, like

func check(source: Directory):
  await format(source)
  await lint(source)
  await test(source)
  ...

If there's a need to call things separately, like if one of the functions needs to return something specific from the CLI, I'd have them as consecutive steps in the same job. That way it'll benefit from the caching in the previous steps. In that scenario you can also just use the dagger-for-github action like a "setup" action without verb/args, and then in each step just dagger call format --source .. The yaml will be a bit shorter that way

[faint storm]

good points. GH workflow should just be a single job which is run dagger CI, and let dagger do all the concurrency.

Speaking of which, how do i split my dagger execution across multiple runners?

[light tree]

Today that would be at the orchestrator level (github actions), so you'd have multiple jobs. Eventually we'll be able to scale out in the dagger engine itself. The main consideration before you do that in github actions is: are the jobs faster running independently on separate machines, or are they faster on the same machine with a shared local cache

[prime gulch]

is this still a thing ?

https://www.youtube.com/watch?v=kqotAzSkhjw

[trail lintel]

I think the gha module is in the main dagger repo now under modules

[prime gulch]

i am getting this error:

✘ .generatedContextDirectory: Directory! 2.3s
! failed to run codegen: failed to load dependencies as modules: failed to load module dependencies: select: failed to load dependencies as modules: failed to load module dependencies: select: failed to load dependencies as modules: failed to load module dependencies: select: failed to get module runtime: failed to call sdk module moduleRuntime: select: process "go build -o /bass ./entrypoint" did not complete successfully: exit code: 1
Error: failed to update dependencies: input: moduleSource.withDependencies.withEngineVersion.generatedContextDirectory failed to run codegen: failed to load dependencies as modules: failed to load module dependencies: select: failed to load dependencies as modules: failed to load module dependencies: select: failed to load dependencies as modules: failed to load module dependencies: select: failed to get module runtime: failed to call sdk module moduleRuntime: select: process "go build -o /bass ./entrypoint" did not complete successfully: exit code: 1

when trying to install the module

command: dagger install github.com/shykes/gha@v0.9.2

Nevermind, it worked using

dagger install github.com/dagger/dagger/modules/gha@v0.15.2

[heavy river]

Where's my "module: github.com/shykes/daggerverse/hello" where the dagger.json is or where the code lives?

dagger.json -> github.com/go-orb/plugins
code -> github.com/go-orb/plugins/.github/

I try the first

It works 🙂

No more copy&paste of code, just use "module", n1!

[uneven widget]

A module is any directory with a dagger.json in it

So you can have a single repo with endless nested modules 🙂

[heavy river]

Can I prepare that unzip install once and run multiple containers based on that one after? https://github.com/go-orb/plugins/blob/main/.github/main.go#L199

[outer charm]

Anyone have any idea why my dagger-for-github job isn't printing the dag logs?

It does if I have the args set to args: go run ./pkg/build/e2e but not args: go run ./pkg/build/e2e -package=grafana.tar.gz -suite='various-suite'

Is there something weird with the quoting that I can't figure out? D:

I removed the quotes and nah. This happens locally too; I have to do dagger run -- go run ./pkg/build/e2e -package=... i'm really not sure why...

I see now. It's because of the flags. It works when -package is changed to --package . I'm assuming -s(uite) was interpreted as --silent in the dagger run command.

[long vector]

Looks like v8 of of the action was tagged without a v prefix and now Dependabot isn't picking up the latest version: https://github.com/dagger/dagger-for-github/releases/tag/8.0.0

cc @wide moon

[long vector]

Scratch that, looks like it just doesn't like it in certain repos?

Though it adds the v prefix: https://github.com/sagikazarmark/template-tutorial-dagger-gha/pull/1

weird

[wide moon]

hey 👋 !

[wide moon]

we switched to 8.0.0 style as that was preferred in GitHub's new immutable actions scheme

[woeful skiff]

Is dagger install supposed to work with commit references in branches other than main/master? Is the dependency cloned locally, so I can inspect it?

[strange knot]

Yes it should work with any valid git ref - branch, tag or commit

The dependency is not cloned locally, but you can use dagger itself to get the same ref, eg. dagger -c 'git REPO_ADDR | commit COMMIT | tree | terminal or dagger -c 'git REPO_ADDR | commit COMMIT | tree | export ./checkout

[woeful skiff]

So how is the dependency used; incorporated on installation, or cached remotely? Or do you hit GitHub with every call?

[strange knot]

it's loaded at runtime and cached like all other dagger operations

[woeful skiff]

So I should be able to see it in the tracer.

[strange knot]

yes absolutely

[umbral quarry]

is there any way to use github action cache with dagger cache?

[delicate comet]

not yet and no current ETA on when that will be possible. We know eventually we'll support it in some way but some things need to happen before getting there

[strange knot]

also their cache service performance is pretty unreliable

[frosty idol]

https://youtu.be/Ahg-3_Ok-rY

[frosty idol]

https://x.com/sidpalas/status/1945475520851857717 😂

Sid Palas (@sidpalas)

Here is my extended version including @dagger_io 😅

X · Jul 16, 2025, 1:28 PM

[tulip vessel]

Hello everyone. We are working on adopting dagger and running on github actions. Is there a recommended way to authenticate the actions jobs to interact with other repos in the org? Ideally it'd be with a github app but I am thinking that might put hurdles on local testing for things like updating container image tags in a deployment repo. Anyone have any suggestions? Maybe multiple apps with different permissions?

[worthy vale]

Hey, we have been using dagger for our pipelines for a while and its been great, but now we migrated from bitbucket to github and encountered some issues with authentication. We want to use the github app installation token provided by the github branch source plugin inside of jenkins, while locally the Git credentials manager should be used instead. Locally I get to the part where authentication works fine for dagger dependencies and also in dagger shell when running the git command, but as soon as I add a dagger function and try calling dag.git to clone a private repo, the authentication fails. Any ideas what I might be doing wrong?

[rapid sable]

Hey all — I’m hitting a weird Dagger LLM issue in GitHub Actions that I can’t reproduce locally.

Locally, the dagger module runs fine. But in a GitHub Actions calling/reusable workflow it blows up right when it tries to use the LLM:
LLM.withPrompt DONE LLM.lastReply ERROR failed to prompt user for LLM API access from [e-mail]/daggerverse.git/agents/pr-reviewer: Unimplemented: unknown service dagger.prompt.Prompt

Feels like it’s trying to do an interactive prompt for LLM access in CI. I'm running dagger version 0.19.3. Any help is welcomed.

[strange knot]

Mmm I could be wrong, but this looks possibly like a version mismatch between dagger CLI and engine? Possibly the dagger CLI is older than the engine?

[rapid sable]

It doesn't look to be the case. 'client' and 'server' versions appear to be the same.

[long vector]

Any reason why this shouldn't work?

      - name: Check
        uses: dagger/dagger-for-github@d913e70051faf3b907d4dd96ef1161083c88c644 # v8.2.0
        with:
          verb: check
          version: ${{ env.DAGGER_VERSION }}
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

[long vector]

https://github.com/dagger/dagger-for-github/issues/199

[light tree]

I'm surprised this doesn't work 🤔 I'll have a look. Thanks for the issue!

[long vector]

The issue explains why it doesn't work

[light tree]

Yup, makes perfect sense! I have this PR up for the fix. I'll release v8.3.0 when its merged