docker - Official Image | Docker Hub

[halcyon cobalt]

hey Clay!
re 1:

did this ever work? or it happened the first time that you're trying to run your pipelines? If this was working before and suddently stopped working, one thing that generally works for me is removing the docker container engine via docker rm $engine_container_id and the next time you run your pipeline it should work

re 2:

how is your test calling docker-compose currently? Are you using testcontainers or a similar framework? What I'd recommend is leveraging Dagger services (https://docs.dagger.io/757394/use-services/) to spawn your test dependencies instead of using docker-compose. Services provide a better API and dependency lifecycle which works better with your application instead of using compose. Let us know if you need any guidance on how to set this up. Happy to help! 🙏

[tranquil hound]

The npm ci had never worked on the container. I tried removing and restarting the engine. It always fails with a timeout.

The test calls a shell script which calls docker compose directly. Nothing like test containers is involved. We could refactor to using dagger services, but this is the test that we currently have. Is there are reason that it wouldn't run this way?

[halcyon cobalt]

Ok, seems like your dagger pipelines can't access the internet for some reason. Have you tried running a simple "ping" or "curl" pipeline to validate this? Or it only happens with npm ci? Do you generally use a proxy or something?

[tranquil hound]

I didn't do anything to setup a proxy of any kind. My dagger pipelines are very simple just running a few basic commands on the node:20-alpine container. It seems to be able to pull images down just fine. I have pulled down scripts to install things on the containers. The only problems I have seen was with npm ci and docker compose.

[halcyon cobalt]

Perfect, thx. Do you get a consistent timeout error in the same package each time?

[tranquil hound]

Yes, same package every time.

[halcyon cobalt]

@tranquil hound by any chance does this work for you?

    ctx := context.Background()
    defer dag.Close()
    dag.Container().From("node:20-alpine").
        WithEntrypoint([]string{"npm"}).
        WithWorkdir("/tmp").
        WithExec([]string{"install", "support-colors"}).Sync(ctx)

just a simple pipeline installing only that package

[tranquil hound]

Yes, that did work.

█ [2.09s] connect
┣ [0.13s] starting engine
┣ [0.23s] starting session
┃ OK!
█ [2.72s] go run multibuild/main.go
┣─╮
│ ▽ from node:20-alpine
│ █ [0.55s] resolve image config for docker.io/library/node:20-alpine
│ █ [0.01s] pull docker.io/library/node:20-alpine
│ ┣ [0.01s] resolve docker.io/library/node:20-alpine@sha256:2f46fd49c767554c089a5eb219115313b72748d8f62f5eccb58ef52bc36db4ad
│ ┣─╮ pull docker.io/library/node:20-alpine
│ ┻ │
█◀──╯ [1.24s] exec npm install support-colors
┃
┃     added 1 package in 1s
┃     npm notice
┃     npm notice New minor version of npm available! 10.2.4 -> 10.4.0
┃     npm notice Changelog: <https://github.com/npm/cli/releases/tag/v10.4.0>
┃     npm notice Run `npm install -g npm@10.4.0` to update!
┃     npm notice
┻

[halcyon cobalt]

can you run []string{"npm", "config", "list", "-l"} instead of npm ci and share the output?

wanna check no weird npm config arguments is generating noise here

[tranquil hound]

    client.Container().From("node:20-alpine").
        WithWorkdir("/tmp").
        WithExec([]string{"npm", "config", "list", "-l"}).Sync(ctx)

This ran fine with the above input.

[halcyon cobalt]

ok nice. Would it be possible for you to share your package.json and package.json.lock dependencies? It's ok to remove all the dependencies that are private to your project. At this stage I'm trying to get a repro since everything seems ok on your side

[halcyon cobalt]

re this: you can make compose work with Dagger but it's more moving pieces since you need to mount your docker socket into your pipeline and then setup some container-to-host port forwards so your tests can actually connect to the services started by compose

[halcyon cobalt]

how do your tests connect to your compose services? Via localhost?

[tranquil hound]

The tests just run a shell script that uses cli docker compose commands. So I guess it is defaulting to local host.

Above it the package.json.lock for my project. It works fine on my local machine and for the team in GitHub actions. But fails consistently in the dagger container.

Is there any documentation on making compose work with Dagger?

[halcyon cobalt]

can you share the package.json also please? Since I'm not sure I can install dependencies with only the lock file

[halcyon cobalt]

@tranquil hound mind changing your npm ci to: WithExec([]string{"npm", "ci", "--loglevel", "verbose"}) and share the results?

just executed your package.json locally and I'm not having any issues

[tranquil hound]

when I run WithExec([]string{"npm", "ci", "--loglevel", "verbose"}). I get a lot of npm verb unfinished npm timer.

[tranquil hound]

@halcyon cobalt Any idea what is causing these npm verb unfinished npm timer errors? They only happen in the dagger container.

[halcyon cobalt]

Hey Clay! any chance we can jump into a quick #911305510882513037 session so you can share your screen? I think it might be faster to find what could be happening 🙏

[tranquil hound]

Yes, when do you have a chance to do that?

[halcyon cobalt]

hey, sorry! it's a bit late for me right now. Tomorrow I have an open day as well

happy to chat then 🙌

since you seem to have time now maybe @distant meadow could help if he has a few minutes 🙏

otherwise, talk to you tomorrow!

[distant meadow]

Hey guys! Can't right now unfortunately, left home a few minutes ago to do some groceries . But I'll join you tomorrow!

[tranquil hound]

I can be available tomorrow. That will be great.

It started working for me today. I no longer get the timeout error. But it is weird because I didn't change anything and can't explain why the error is not happening now. I'm also getting a coworker to look at it and he is seeing the timeout error.

[halcyon cobalt]

got it. Ping me tomorrow please and we'll see if we can repro or if we can spot what might be happening 🙏

[tranquil hound]

I think we have it figured out. It wasn't a container networking issue like the error suggested. We have a fairly complicated ui project. We had to make some changes to our npm build, make sure some dependencies were manually installed on the image, and make use of max-old-space-size and npm_config_maxsockets. Now things seem to work reliably. Thanks for your help.