I am trying to containerize a golang app | Dagger | Page 1

deep girder Apr 2, 2023, 9:57 PM

#

@dire kraken Let us use a thread so i don't spam general channel, Yikes

dire kraken Apr 2, 2023, 10:01 PM

#

A good idea, but I'm still not sure what you're trying to do. It sounds like you're trying to build and publish a go app in a container, but I'm not sure what from there

deep girder Apr 2, 2023, 10:06 PM

#

This is a golang app that uses dagger; the image(nerdeveloper/ubuntu) I am pulling has a docker client to package applications and push them to a registry(that is the goal).

After packaging this app with alpine and deploying the docker daemon host. When i call the API that runs the CI function, I get an error

 failed to list containers: exec: "docker": executable file not found in $PATH
noxe  | Error: failed to run container: : exec: "docker": executable file not found in $PATH
noxe  | 2023/04/02 21:24:03 http: panic serving 172.21.0.1:60870: EOF: failed to list containers: exec: "docker": executable file not found in $PATH
noxe  | Error: failed to run container: : exec: "docker": executable file not found in $PATH
noxe  |
noxe  | Please visit https://dagger.io/help#go for troubleshooting

dire kraken Apr 2, 2023, 10:08 PM

#

If all you seek to do is build and publish do you need the inner container? https://pkg.go.dev/dagger.io/dagger#Directory.DockerBuild

deep girder Apr 2, 2023, 10:11 PM

#

The inner container uses Cloudnative build packs which build the image https://buildpacks.io/

I do not control it under the hood, but it needs Docker client and dockerd

Cloud Native Buildpacks

Cloud Native Buildpacks transform your application source code into images that can run on any cloud.

#

It converts the source files it pulls from git clone to an image

dire kraken Apr 2, 2023, 10:14 PM

#

Ah I see. I have never seen/used nerdeveloper/ubuntu - does it come with docker preinstalled and starts up on container start?

deep girder Apr 2, 2023, 10:17 PM

#

Haha, nerdeveloper/ubuntu image I made that has a docker installed. Docker Client is installed in it, i use the Host system to mount the docker daemon into the container when it starts,

slate radish Apr 2, 2023, 10:17 PM

#

@deep girder can you share your code in a Git repo or Gist? Or maybe paste it in?

#

Makes it easier to reproduce what you're seeing 🙏

deep girder Apr 2, 2023, 10:18 PM

#

okay okay

stoic mountain Apr 2, 2023, 10:28 PM

#

@deep girder you're trying to run that dagger app in ECS?

#

that's not possible since ECS doesn't expose a docker socket

deep girder Apr 2, 2023, 10:41 PM

#

https://github.com/nerdeveloper/noxe @slate radish

GitHub

GitHub - nerdeveloper/noxe

Contribute to nerdeveloper/noxe development by creating an account on GitHub.

#

@stoic mountain I think it does, Our TeamCity agents use it

stoic mountain Apr 2, 2023, 10:44 PM

#

well.. that depends on how your ECS cluster is configured

#

this need to be ECS classic, since Fargate definitely doesn't support it

slate radish Apr 2, 2023, 10:45 PM

#

Docker-in-docker supported with Team City agents, it looks like.

https://hub.docker.com/r/jetbrains/teamcity-agent#:~:text=Running Builds Which Require Docker

stoic mountain Apr 2, 2023, 10:45 PM

#

with ECS classic, if you have an AMI that has docker installed or you use docker-in-docker you can make it work

slate radish Apr 2, 2023, 10:48 PM

#

deep girder https://github.com/nerdeveloper/noxe <@933501536624054272>

Got your error. Thanks for nice repo for repro!

#

Will take a look

#

@deep girder what is calling docker and causing the docker not found?

noxe ➤ grep -r docker *                                                          git:main
README.md:docker-compose up
docker-compose.yml:      dockerfile: dev.Dockerfile
docker-compose.yml:      - /var/run/docker.sock:/var/run/docker.sock
pkg/ci/ci.go:    //dockerUsername := "nerdeveloper"
pkg/ci/ci.go:    //dockerPassword := os.Getenv("DOCKER_PASSWORD")
pkg/ci/ci.go:    dockerSocket := client.Host().UnixSocket("/var/run/docker.sock")
pkg/ci/ci.go:        WithUnixSocket("/var/run/docker.sock", dockerSocket).
pkg/ci/ci.go:        // WithExec([]string{"docker", "login", "--username", dockerUsername, "--password", dockerPassword}).
runners/ubuntu/jammy.Dockerfile:    curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg && \
runners/ubuntu/jammy.Dockerfile:    echo "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null && \
runners/ubuntu/jammy.Dockerfile:    apt-get install -y docker-ce docker-ce-cli containerd.io git pack-cli  && \

stoic mountain Apr 2, 2023, 10:50 PM

#

@deep girder Dagger requires the docker CLI ( as well as the daemon) to be present in your runner environment where your Dagger pipeline runs. For what I see in your error, seems like the ECS task that effectively calls Dagger, doesn't seem to have the docker CLI installed.

slate radish Apr 2, 2023, 10:50 PM

#

Or is it Dagger itself

#

seems to be, right?

deep girder Apr 2, 2023, 10:52 PM

#

this is what called docker cli https://github.com/nerdeveloper/noxe/blob/3153b70bab04c980fce7e73f770165b62cc70b28/pkg/ci/ci.go#L36

GitHub

noxe/ci.go at 3153b70bab04c980fce7e73f770165b62cc70b28 · nerdevelop...

Contribute to nerdeveloper/noxe development by creating an account on GitHub.

#

if you run this project without containerizing it, it does not fail

#

i think it is failing cos it looking for docker inside alpine rather than inside dagger image that was pulled

#

also, @stoic mountain i am not sure how you made pack to build the image. when I build mine hangs and remains there till i kill on dagger

#

On my local, it works, but when I run it via dagger.

#3 9.108 Setting default process type 'web'
#3 9.108 Saving obinna...
#3 23.76 *** Images (3fac46745e7a):
#3 23.76       obinna
#3 23.76 Reusing cache layer 'paketo-buildpacks/node-engine:node'
#3 24.30 Reusing cache layer 'paketo-buildpacks/npm-install:npm-cache'
#3 24.33 Reusing cache layer 'buildpacksio/lifecycle:cache.sbom'
^Ccleaning...
deleting /Users/odirionye/Documents/go/src/github.com/clouddley/noxe/tmp
see you again~

it hangs at #3 24.33 Reusing cache layer 'buildpacksio/lifecycle:cache.sbom'

till i kill the process

slate radish Apr 2, 2023, 11:29 PM

#

Seems to work for me when I run go run main.go from the root of your repo and then send the curl

#

oh, I see what you mean. Mine hanging at same step

#3 20.46 Adding label 'io.buildpacks.project.metadata'
#3 20.46 Setting default process type 'web'
#3 20.46 Saving obinna...
#3 22.26 *** Images (47e899c0de65):
#3 22.26       obinna
#3 22.26 Reusing cache layer 'paketo-buildpacks/node-engine:node'
#3 22.51 Reusing cache layer 'paketo-buildpacks/npm-install:npm-cache'
#3 22.52 Reusing cache layer 'buildpacksio/lifecycle:cache.sbom'

deep girder Apr 3, 2023, 1:46 AM

#

aha, that was the second bug I noticed. you were able to reproduce it

slate radish Apr 3, 2023, 1:32 PM

#

@leaden gust @devout condor see anything obvious here? Networking etc?

stoic mountain Apr 3, 2023, 3:13 PM

#

👋 the "hang" issue comes from here: https://github.com/dagger/dagger/issues/4673. What I did for my demo was to patch the pack CLI so it doesn't hang

GitHub

Issues · dagger/dagger

A programmable CI/CD engine that runs your pipelines in containers - Issues · dagger/dagger

stoic mountain Apr 3, 2023, 3:34 PM

#

stoic mountain 👋 the "hang" issue comes from here: https://github.com/dagger/dagger/issues/467...

let me send you a branch with my fork so you can test with that 🙏

stoic mountain Apr 3, 2023, 5:52 PM

#

https://github.com/marcosnils/pack/tree/hack/omit_handler

GitHub

GitHub - marcosnils/pack at hack/omit_handler

CLI for building apps using Cloud Native Buildpacks - GitHub - marcosnils/pack at hack/omit_handler

stoic mountain Apr 3, 2023, 5:52 PM

#

stoic mountain https://github.com/marcosnils/pack/tree/hack/omit_handler

if you build the pack CLI from this branch it should work 🙏

deep girder Apr 3, 2023, 6:22 PM

#

Okay, I will definitely try it. Hopefully, it will get merged into packs in the future. How about the docker issue, any ideas to run on a docker daemon host

stoic mountain Apr 3, 2023, 6:30 PM

#

deep girder Okay, I will definitely try it. Hopefully, it will get merged into packs in the ...

it's not a packs issue. It's a Dagger issue we're looking at

deep girder Apr 3, 2023, 6:31 PM

#

Noted.

stoic mountain Apr 3, 2023, 6:32 PM

#

How about the docker issue, any ideas to run on a docker daemon host

not sure I follow Obinna. WDYM about running the daemon host

deep girder Apr 3, 2023, 7:08 PM

#

I shared a repo in the thread, about running an application using dagger golang SDK, and it fails

deep girder Apr 3, 2023, 7:10 PM

#

slate radish <@624053529044320318> what is calling `docker` and causing the `docker not found...

this to be precise

stoic mountain Apr 3, 2023, 8:01 PM

#

deep girder this to be precise

I see.. do you get this docker error when you run it locally? or only in ECS?

deep girder Apr 3, 2023, 8:04 PM

#

both, any docker daemon host at all,

stoic mountain Apr 3, 2023, 8:17 PM

#

deep girder both, any docker daemon host at all,

k, let me try this really quick. If you're available to do a quick #911305510882513037 that might help to speed up the troubleshooting 🙏

deep girder Apr 3, 2023, 8:20 PM

#

Sign me up

stoic mountain Apr 3, 2023, 8:21 PM

#

deep girder Sign me up

you're muted

deep girder Apr 3, 2023, 8:21 PM

#

one moment, having mic issues

stoic mountain Apr 3, 2023, 8:21 PM

#

sure, np

#

https://meet.google.com/uyz-nmsn-bjs

Meet

Real-time meetings by Google. Using your browser, share your video, desktop, and presentations with teammates and customers.

#I am trying to containerize a golang app