dagger runs fine locally but not in a GH Action pipeline | Dagger | Page 1

slim dawn Jun 19, 2024, 1:36 PM

#

Here's a snippet of the logs. I'm lost - I worked on this yesterday for a long time and still can't get it to work. I did drop the 'with: module: demo' part from the GHA - either way same issue.

name: Run Dagger
env:
HOOK: "https://hooks.slack.com/services/xxxxxxxx"
uses: dagger/dagger-for-github@v5
with:
workdir: ${{github.workspace}}/dagger_io/demo
version: 0.11.6
verb: functions
args: send-slack-message --message frompipeline --webhook %HOOK%

13:32:06 INF Connected to engine name=5bc15fdd28fe version=v0.11.6
6 : connecting to engine DONE [2.1s]
7 : starting session
7 : starting session DONE [0.2s]
1 : connect DONE [8.5s]
8 : initialize
9 : ModuleSource.asModule: Module!
9 : ModuleSource.asModule: Module! ERROR [5.9s]
9 : ! failed to create module: select: failed to update codegen and runtime: failed to generate code: failed to get modified source directory for go module sdk codegen: select: process "/usr/local/bin/codegen --output /src --module-context-path /src/dagger_io/demo/dagger --module-name demo --introspection-json-path /schema.json" did not complete successfully: exit code: 1
8 : initialize ERROR [5.9s]
8 : ! input: moduleSource.withContextDirectory.asModule resolve: failed to create module: select: failed to update codegen and runtime: failed to generate code: failed to get modified source directory for go module sdk codegen: select: process "/usr/local/bin/codegen --output /src --module-context-path /src/dagger_io/demo/dagger --module-name demo --introspection-json-path /schema.json" did not complete successfully: exit code: 1

Slack API

Unlock your productivity potential with Slack Platform

Connect, simplify, and automate work.

proper wave Jun 19, 2024, 1:59 PM

#

cc @loud hemlock any clues?

slim dawn Jun 19, 2024, 2:03 PM

#

I puit the module: demo back and pinned everything to 0.11.6 - and same. current log: : [5.6s] | Status: Downloaded newer image for registry.dagger.io/engine:v0.11.6
4 : [5.6s] | registry.dagger.io/engine:v0.11.6
4 : exec docker pull registry.dagger.io/engine:v0.11.6 DONE [5.6s]
5 : exec docker run --name dagger-engine-32e0269fab8e9d98 -d --restart always -v /var/lib/dagger --privileged registry.dagger.io/engine:v0.11.6 --debug
5 : [0.0s] | c09c3dc2534fbca736fa2d5dc9d0e1153268b7cf0a5e0ac26f09215320d37563
5 : exec docker run --name dagger-engine-32e0269fab8e9d98 -d --restart always -v /var/lib/dagger --privileged registry.dagger.io/engine:v0.11.6 --debug DONE [0.3s]
2 : starting engine DONE [6.3s]
6 : connecting to engine
14:00:53 INF Connected to engine name=c09c3dc2534f version=v0.11.6
6 : connecting to engine DONE [2.1s]
7 : starting session
7 : starting session DONE [0.2s]
1 : connect DONE [8.6s]
8 : initialize
9 : ModuleSource.asModule: Module!
9 : ModuleSource.asModule: Module! ERROR [5.9s]
9 : ! failed to create module: select: failed to update codegen and runtime: failed to generate code: failed to get modified source directory for go module sdk codegen: select: process "/usr/local/bin/codegen --output /src --module-context-path /src/dagger_io/demo/dagger --module-name demo --introspection-json-path /schema.json" did not complete successfully: exit code: 1
8 : initialize ERROR [6.0s]
8 : ! input: moduleSource.withContextDirectory.asModule resolve: failed to create module: select: failed to update codegen and runtime: failed to generate code: failed to get modified source directory for go module sdk codegen: select: process "/usr/local/bin/codegen --output /src --module-context-path /src/dagger_io/demo/dagger --module-name demo --introspection-json-path /schema.json" did not complete successfully: exit code: 1
Stdout:

prime snow Jun 19, 2024, 2:07 PM

#

@slim dawn are you able to share a Trace via Dagger Cloud, so we can inspect it? You need a free Dagger Cloud account (dagger login), then add your dagger cloud API token to CI environment

slim dawn Jun 19, 2024, 2:11 PM

#

I'll do my best to setup that up. I think I have an account and api - any special commands to issue in the pipeline or does the action handle it for me (like the dagger login?)

prime snow Jun 19, 2024, 2:12 PM

#

all you have to do is authenticate the engine and it will start sending telemetry. No change to your code required.

slim dawn Jun 19, 2024, 2:12 PM

#

sweet - on it - will repspond when done - thanks a million for the offer of help

prime snow Jun 19, 2024, 2:12 PM

#

If you dagger login on your local machine it will start sending traces as well (no env variable needed). That way you can compare a working trace to a not-working trace

winter elk Jun 19, 2024, 2:36 PM

#

Why is the verb set to functions instead of call?

Intentional?

#

You can omit verb and it will default to dagger call

loud hemlock Jun 19, 2024, 2:44 PM

#

Yeah, the verb isn't right.

#

I wouldn't expect it to fail during codegen though.

#

What's in dagger_io/demo, is it from us?

slim dawn Jun 19, 2024, 2:54 PM

#

sorry - got on a meeting...

#

ok - I'll set it to call - I was just trying to get it to output the functions. @winter elk

#

brb - let me try that

#

#

same with verb of call

#

here is the main.go from dagger_io/demo dir in my repo - it is not from you. I developed all of this - no laughing...I'm new to golang. @loud hemlock

#

// A generated module for Demo functions
//
// This module has been generated via dagger init and serves as a reference to
// basic module structure as you get started with Dagger.
//
// Two functions have been pre-created. You can modify, delete, or add to them,
// as needed. They demonstrate usage of arguments and return types using simple
// echo and grep commands. The functions can be called from the dagger CLI or
// from one of the SDKs.
//
// The first line in this comment block is a short description line and the
// rest is a long description with more detail on the module's purpose or usage,
// if appropriate. All modules should have a short description.

// err = slack.PostWebhook("https://hooks.slack.com/services/T1C1DRE91/xxxx", &msg)

package main

import (
"encoding/json"
"fmt"
"strconv"
"time"

"github.com/slack-go/slack"

)

type Demo struct{}

func (m *Demo) SendSlackMessage(message string, webhook string) {
if message == "" {
message = "<!channel> Hi From Dagger.io and GoLang"
}
attachment := slack.Attachment{
Color: "good",
Fallback: "You successfully posted by Incoming Webhook URL!",
AuthorName: "Bob Hoss",
AuthorSubname: "github.com",
Text: message,
Ts: json.Number(strconv.FormatInt(time.Now().Unix(), 10)),
}

msg := slack.WebhookMessage{
    Attachments: []slack.Attachment{attachment},
}

//err := slack.PostWebhook("https://hooks.slack.com/services/T1C1DRE91/xxxx", &msg)
err := slack.PostWebhook(webhook, &msg)
if err != nil {
    fmt.Println(err)
}

}

Slack API

Unlock your productivity potential with Slack Platform

Connect, simplify, and automate work.

proper wave Jun 19, 2024, 3:08 PM

#

@slim dawn is the dagger.json file in the demo folder? Or is it somewhere else in the project path?

slim dawn Jun 19, 2024, 3:09 PM

#

#

a screenshot from vscode @proper wave

#

contents of dagger.json {
"name": "demo",
"sdk": "go",
"source": "dagger",
"engineVersion": "v0.11.6"
}

proper wave Jun 19, 2024, 3:11 PM

#

slim dawn

Thx mind sharing this screenshot again? For some reason it's unreadable for me

slim dawn Jun 19, 2024, 3:13 PM

#

no worries...

#

#

if that doesn/t work... here's a dir

proper wave Jun 19, 2024, 3:14 PM

#

Oh sorry, I meant the actions output terminal

slim dawn Jun 19, 2024, 3:14 PM

#

roger

#

no worries!

#

#

any better?

proper wave Jun 19, 2024, 3:16 PM

#

Yes, thx. Do you have more output after the stdout line at the bottom?

#

That might show where the error is coming from

slim dawn Jun 19, 2024, 3:16 PM

#

Stdout:
generating go module: demo
writing dagger_io/demo/dagger/dagger.gen.go
writing dagger_io/demo/dagger/go.mod
writing dagger_io/demo/dagger/go.sum
creating directory dagger_io/demo/dagger/internal
creating directory dagger_io/demo/dagger/internal/dagger
writing dagger_io/demo/dagger/internal/dagger/dagger.gen.go
creating directory dagger_io/demo/dagger/internal/querybuilder
writing dagger_io/demo/dagger/internal/querybuilder/marshal.go
writing dagger_io/demo/dagger/internal/querybuilder/querybuilder.go
creating directory dagger_io/demo/dagger/internal/telemetry
writing dagger_io/demo/dagger/internal/telemetry/attrs.go
writing dagger_io/demo/dagger/internal/telemetry/batch_processor.go
writing dagger_io/demo/dagger/internal/telemetry/init.go
writing dagger_io/demo/dagger/internal/telemetry/processor.go
writing dagger_io/demo/dagger/internal/telemetry/proxy.go
writing dagger_io/demo/dagger/internal/telemetry/span.go
running post-command: go mod tidy
post-command failed: exit status 1
Stderr:
go: github.com/99designs/gqlgen@v0.17.44: Get "https://proxy.golang.org/github.com/99designs/gqlgen/@v/v0.17.44.mod": EOF
Error: exit status 1
Error: input: moduleSource.withContextDirectory.asModule resolve: failed to create module: select: failed to update codegen and runtime: failed to generate code: failed to get modified source directory for go module sdk codegen: select: process "/usr/local/bin/codegen --output /src --module-context-path /src/dagger_io/demo/dagger --module-name demo --introspection-json-path /schema.json" did not complete successfully: exit code: 1

Stdout:
generating go module: demo
writing dagger_io/demo/dagger/dagger.gen.go
writing dagger_io/demo/dagger/go.mod

#

writing dagger_io/demo/dagger/go.sum
creating directory dagger_io/demo/dagger/internal
creating directory dagger_io/demo/dagger/internal/dagger
writing dagger_io/demo/dagger/internal/dagger/dagger.gen.go
creating directory dagger_io/demo/dagger/internal/querybuilder
writing dagger_io/demo/dagger/internal/querybuilder/marshal.go
writing dagger_io/demo/dagger/internal/querybuilder/querybuilder.go
creating directory dagger_io/demo/dagger/internal/telemetry
writing dagger_io/demo/dagger/internal/telemetry/attrs.go
writing dagger_io/demo/dagger/internal/telemetry/batch_processor.go
writing dagger_io/demo/dagger/internal/telemetry/init.go
writing dagger_io/demo/dagger/internal/telemetry/processor.go
writing dagger_io/demo/dagger/internal/telemetry/proxy.go
writing dagger_io/demo/dagger/internal/telemetry/span.go
running post-command: go mod tidy
post-command failed: exit status 1
Stderr:
go: github.com/99designs/gqlgen@v0.17.44: Get "https://proxy.golang.org/github.com/99designs/gqlgen/@v/v0.17.44.mod": EOF
Error: exit status 1

proper wave Jun 19, 2024, 3:17 PM

#

Oh, interesting... Looks like a go proxy error fetching that dependency?

slim dawn Jun 19, 2024, 3:18 PM

#

we do have a firewall on our github runners - I don't know about a proxy per se. I could try github hosted runners vs our self hosted.

proper wave Jun 19, 2024, 3:18 PM

#

slim dawn we do have a firewall on our github runners - I don't know about a proxy per se....

Oh, this is very likely to be the issue then

slim dawn Jun 19, 2024, 3:18 PM

#

I'm pretty sure github.com would not be blocked but need to confirm

proper wave Jun 19, 2024, 3:19 PM

#

I didn't realize that you were using a self hosted runners

proper wave Jun 19, 2024, 3:19 PM

#

slim dawn I'm pretty sure github.com would not be blocked but need to confirm

In this case it's proxy.golang.org

slim dawn Jun 19, 2024, 3:19 PM

#

yes - wasn't aware that would be different so long as it had access (aka whitelisted) - ok....golang may not be allowed.

#

good catch...

#

I see it now

proper wave Jun 19, 2024, 3:19 PM

#

Maybe your company has an internal proxy or something.. that's quite common

#

Using GitHub hosted runners should work

slim dawn Jun 19, 2024, 3:20 PM

#

no proxy - only allowed whitelisted fqdns.

#

I'll try the github hosted ones...will report back in

#

so for dagger.io and api.dagger.cloud - I had to have those whitelisted to work on our runners...makes sense.

#

ok - confirmed this is a self hosted runner vs github runner situation our firewall is blocking the golang proxy mentioned in the thread. UGH. This is like 2x's this has gotten me. Thanks for the help everyone!!

#

MANY MANY Thanks to all who helped!

prime snow Jun 19, 2024, 3:50 PM

#

@slim dawn glad you could track it down! Are you stuck in your adoption of Dagger because of this?

#

how can we help

slim dawn Jun 19, 2024, 3:52 PM

#

I wouldn't say stuck - but it was rough to figure out. I REALLY want to adopt Dagger.io here at the office. In fact I'm giving a demo of it next Friday to my tech org as I'm so excited about it.

#

I would like to know - per this article: https://stackoverflow.com/questions/71023129/module-lookup-disabled-by-goproxy-off-but-go-env-shows-goproxy-is-set you can turn off the GOPROXY - but I'm not sure how to get that into the GH Action yaml without modifying it which kind of defeats the idea of using the action. Because I 'think' the export GOPROXY=direct would have to be handled by the dagger/dagger-for-github@v5 action. @prime snow

Stack Overflow

module lookup disabled by GOPROXY=off, but go env shows GOPROXY is set

When trying to lookup some modules, I am having an issue from VS Code where the error pictured below indicates that my GOPROXY is set to off, but when I run go env, I see that GOPROXY is actually set:

prime snow Jun 19, 2024, 3:57 PM

#

slim dawn I would like to know - per this article: https://stackoverflow.com/questions/710...

yeah I think that would even need a change to the Go SDK. cc @wet monolith who has been tackling the overall "dagger behind corporate proxy" topic

slim dawn Jun 19, 2024, 3:58 PM

#

understood. It would be nice if it was a flag - something like GOPROXY: disable for the dagger/dagger-for-github@v5 action or something similar.

#

also: https://webtips.fly.dev/how-to-disable-go-proxy/

Software Engineer Tips And Tricks

How to disable Go Proxy

To disable Go module proxy, you can use the following steps:

Set the GOPROXY environment variable to "direct":

$ export GOPROXY=direct

This will instruct Go to use the modules directly from their source repositories, without going through any proxy.

Optionally, you can unset any other related environment variables

prime snow Jun 19, 2024, 4:00 PM

#

just to be sure, disabling go proxy still has the go tooling downloading dependencies from upstream git repos, that won't be blocked by your proxy?

#

if the goal is to "airgap" the loading of that module unfortunately disabling the go proxy won't be enough

slim dawn Jun 19, 2024, 4:01 PM

#

we have github open - so I don't think that will be an issue. I'd have to test it obviously.

proper wave Jun 19, 2024, 4:02 PM

#

I think there's a way to make this work with the default action. Currently on mobile,.will reply in a sec

prime snow Jun 19, 2024, 4:02 PM

#

slim dawn we have github open - so I don't think that will be an issue. I'd have to test ...

ok. technically those dependencies could be on any git server but given github's market share it's possible that in practice github is all you need for our go deps

slim dawn Jun 19, 2024, 4:03 PM

#

could be @prime snow , good thought. Thanks again for all the help and input!!!!!

proper wave Jun 19, 2024, 4:08 PM

#

@slim dawn can you try adding

uses: dagger/dagger-for-github@v5
env:
  GO_PROXY: direct
  _DAGGER_ENGINE_SYSTEMENV_GOPROXY: direct

and see if that works? 🙏

slim dawn Jun 19, 2024, 4:09 PM

#

on it!

proper wave Jun 19, 2024, 4:14 PM

#

hmmm.. I think that won't work.. we're not automatically passing that variable in the engine startup from the CLI 😬

slim dawn Jun 19, 2024, 4:15 PM

#

currently hangning here:

#

yup - something not right. Just sitting at the conecting to engine

proper wave Jun 19, 2024, 4:22 PM

#

you can do the following as a temporary workaround:

- name: Start custom engine
  run:  docker run --name dagger-engine -e _DAGGER_ENGINE_SYSTEMENV_GOPROXY=direct -d --restart always -v /var/lib/dagger --privileged registry.dagger.io/engine:v0.11.6 --debug
- name: Build
  uses: dagger/dagger-for-github@v5
  env:
    _EXPERIMENTAL_DAGGER_RUNNER_HOST: docker-container://dagger-engine
  with:
    version: "latest"
    verb: call
    args: ......

#

the action currently doesn't support starting the engine with custom env variables and in this particular case we need to set the _DAGGER_ENGINE_SYSTEMENV.... so it picks the right goproxy

#

I'm already taking note of that to open an issue to address this in a better way

slim dawn Jun 19, 2024, 4:23 PM

#

testing away - brb - and thanks for looking into this, very appreciated!

proper wave Jun 19, 2024, 4:23 PM

#

the snippet above runs the engine manually with the correct env and then sets the _EXPERIMENTAL_DAGGER_RUNNER_HOST to that engine is used afterwards 🙏

slim dawn Jun 19, 2024, 4:24 PM

#

makes sense - brb with results

proper wave Jun 19, 2024, 4:25 PM

#

brb

slim dawn Jun 19, 2024, 4:29 PM

#

hmm...similar error as before. logs:

📎 message.txt

proper wave Jun 19, 2024, 4:30 PM

#

That's what Solomon was mentioning

#

Looks like the goproxy worked

#

But now it's trying to resolve go.opentelemetry.io

#

And your firewall it's blocking it

slim dawn Jun 19, 2024, 4:31 PM

#

yeah - exactly.

#

it would be nice to be able to turn off things like this for companies like us.

#

like why do I need opentelemetry.io - I wonder if that can be done....have to look

proper wave Jun 19, 2024, 4:36 PM

#

slim dawn it would be nice to be able to turn off things like this for companies like us.

Yes, a totally valid point. We're lazily initializing the Go SDK to keep engine image size under control which requires pulling some deps on demand. Having said that we could definitely improve this to avoid running into situations like these 🙏

#

If there's a way you could whitelist proxy.golang.org that'd be the best approach I think

slim dawn Jun 19, 2024, 4:37 PM

#

totally cool - and makes 120% sense - it's not really dagger so much as it's golang so far as I can tell.

#

I can most likely get it whitelisted - good to know about all this though. I'm used to powershell - so go is all new to me as to how it needs things and so on.

proper wave Jun 19, 2024, 4:37 PM

#

slim dawn totally cool - and makes 120% sense - it's not really dagger so much as it's gol...

Yeah.. Golang deps model is tricky to tackle correctly in constrained environments

slim dawn Jun 19, 2024, 4:38 PM

#

finding that out quickly, would python be a better choice of language for me in this case?

prime snow Jun 19, 2024, 4:38 PM

#

slim dawn it would be nice to be able to turn off things like this for companies like us.

I worry that would lead to endless wack-a-mole... I sterted a thread to explore a possible long-term solution. Would love your thoughts. #maintainers message

proper wave Jun 19, 2024, 4:40 PM

#

slim dawn finding that out quickly, would python be a better choice of language for me in ...

Changing the SDK won't help in this case since the engine will still try to download some Go deps the first time it runs for an internal code generation process

#

Come to think about it, "maybe" the python SDK doesn't need this given that it has a specialized code generation process. cc @loud hemlock will know for sure.

slim dawn Jun 19, 2024, 4:48 PM

#

sounds good - I'm open to python - if he knows, I'll be curious.

loud hemlock Jun 19, 2024, 4:53 PM

#

Yeah, Python's codegen is all python. Only depends on a python lib for graphql introspection.

#

But, all non-Go SDKs use a Go module for their runtime modules.

#

So atm you always run at least one Go module, even if you don't use any directly or as a dependency.

slim dawn Jun 19, 2024, 6:35 PM

#

makes sense - so @loud hemlock , it sounds like I should try using Python as my sdk for now with as much firewall blocking as we have.

wet monolith Jun 19, 2024, 6:50 PM

#

There’s an issue in our github repo, called Modules Break in Corporate Environments (im on a bad cell connection atm and can’t get gh to load).

In there the next steps are described, which is to support setting ca/proxy settings (including GOPROXY) set by the client, whereas right now they have to be set on the engine container. I think that’s part of what you want.

The problem of “arbitrary proxy settings” also is described somewhere in the issue. The idea is to give a mechanism for configuring arbitrary env vars on module containers. We started with just GOPROXY because it was by far the most common request but there’s paths to generalizing it.

proper wave Jun 19, 2024, 6:57 PM

#

slim dawn makes sense - so <@768585883120173076> , it sounds like I should try using Pytho...

Python will also not work per @loud hemlock's last comment:

So atm you always run at least one Go module, even if you don't use any directly or as a dependency.

#

this imples that all Dagger SDK's will currently try to install some go dependencies as part of the bootstrapping phase

prime snow Jun 19, 2024, 7:00 PM

#

@wet monolith sorry I didn't realize you were on vacation. enjoy!

slim dawn Jun 19, 2024, 7:03 PM

#

Thanks all for the explanations!! This helps me understand. Thanks for taking the time to help and inform!

restive wing Jul 30, 2024, 8:18 PM

#

can someone explain me how can i add GOPROXY on engine ? (docker)

proper wave Jul 30, 2024, 8:54 PM

#

restive wing can someone explain me how can i add GOPROXY on engine ? (docker)

@restive wing would this be for GHA?

restive wing Jul 30, 2024, 8:54 PM

#

no

proper wave Jul 30, 2024, 8:55 PM

#

restive wing no

ok, so in order for the engine to pick a GOPROXY, you need to set _DAGGER_ENGINE_SYSTEMENV_GOPROXY env variable when starting the engine

#

you'll have to provision a custom engine https://docs.dagger.io/manuals/administrator/custom-runner for this

Use a Custom Runner | Dagger

A runner is the "backend" of the Dagger Engine where containers are actually executed.

#dagger runs fine locally but not in a GH Action pipeline