#Unexpected behavior go SDK - multi stage build

1 messages · Page 1 of 1 (latest)

worldly wren
#

Hi I am trying to run a simple go, build and test pipeline for a multi arch image. I have 2 arch variants and I just need 2 containers. However, I am seeing that multiple invokes of the same commands within dagger. Could someone please help me identify what i'm doing wrong here?

# 
func build(ctx context.Context, client *dagger.Client) (error, []*dagger.Container) {
    gocache := client.CacheVolume(time.Now().Weekday().String())

    platformVariants := make([]*dagger.Container, 0, len(platforms))
    for _, platform := range platforms {

        p := platform
        builder := client.Container().
            From(buildImage).
            WithMountedCache("/cache", gocache).
            WithMountedCache("/root/.cache/go-build", client.CacheVolume("go-build")).
            WithEnvVariable("GOMODCACHE", "/cache").
            WithEnvVariable("GOOS", platformOf(p)).
            WithEnvVariable("GOARCH", architectureOf(p)).
            WithMountedDirectory("/src", client.Host().Workdir()).
            WithWorkdir("/src")

        builder = builder.
            WithExec([]string{"go", "mod", "download"}).
            WithExec([]string{"make", "build"})

        tester:= client.Container(dagger.ContainerOpts{Platform: p}).
            From(testImage).
            WithMountedDirectory("/src", builder.Directory("/src")).
            WithWorkdir("/src").
            WithMountedCache("/cache", gocache).
            WithMountedCache("/root/.cache/go-build", client.CacheVolume("go-build")).
            WithEnvVariable("GOMODCACHE", "/cache").
            WithExec([]string{"apk", "add", "make"}).
            WithExec([]string{"make", "test"})

        path := "dist"
        output := tester.Directory(path)

        binaryCtr := client.Container(dagger.ContainerOpts{Platform: p})

        binaryCtr = binaryCtr.WithDirectory("/dist", output).
            WithFile("/bin/keel", output.File("keel")).
            WithEntrypoint([]string{"./bin/keel"})
        platformVariants = append(platformVariants, binaryCtr)
    }
    return nil, platformVariants
}

func publish(ctx context.Context, client *dagger.Client, variants []*dagger.Container) error {
    ....client.Container.publish()
}
#

Unexpected behavior go SDK - multi stage build

#

The tester container and subsequently the go test command seems to run 4 times. I am baffled at why that is.

rapid yew
#

@worldly wren could you supply a complete runnable example in a gist, for example 🙏 It would make it quicker to reproduce.

worldly wren
#

let me try and put together something tomorrow and share.

worldly wren
#

So this is a tricky one to generalize because it seems the issue happens when I publish the image to the registry. We are not allowed to push to public registries. I am posting the gist of the entire code here sanitized to remove company references. This may give a better idea of what I am trying to do. Basically, when I try to publish with containerPublishOpts it repeats my tests 4 times. Otherwise it doesn't. What's causing that? https://gist.github.com/nipuna-perera/2153f79062d945352ee0eb19f3da91f6

Gist
Code to reproduce the test duplication

Code to reproduce the test duplication. GitHub Gist: instantly share code, notes, and snippets.

rapid yew
#

@mild jasper can you take a look

#

Just change "jeremyatdockerhub" to your dockerhub username

#

Once we get this working, it's a very nice multi-arch container build example.

#

For example I get

...
#2 Build
#2 1.596 ?       main    [no test files]
#2 DONE 13.2s
...
#2 Build
#2 13.83 ?       main    [no test files]
#2 DONE 27.1s
...
#2 Build
#2 13.83 ?       main    [no test files]
#2 DONE 27.1s
...
#2 Build
#2 -16.121 ?       main    [no test files]
#2 DONE 27.8s

#2 Build
#2 DONE 28.6s
...
#6 pushing layers
#6 pushing layers 6.6s done
#6 pushing manifest for docker.io/jeremyatdockerhub/keelclimulti:latest@sha256:2e91de8075a516faf825892bf7418e659d05ff0488357cef4850991966f1752e
#6 pushing manifest for docker.io/jeremyatdockerhub/keelclimulti:latest@sha256:2e91de8075a516faf825892bf7418e659d05ff0488357cef4850991966f1752e 1.1s done
#6 DONE 14.6s
docker.io/jeremyatdockerhub/keelclimulti:latest@sha256:2e91de8075a516faf825892bf7418e659d05ff0488357cef4850991966f1752e
#

between runs, I blow away the dagger engine and volumes.

C=$(docker ps | grep dagger | tail -n1 | cut -d' ' -f1)
docker kill $C; docker rm $C; docker system prune --force --volumes
mild jasper
#

I can confirm it's only actually running once per arch, just double logging. Not sure why on that yet

#

I changed your go test ./... exec to WithExec([]string{"sh", "-c", "echo now: $(date +%s%N) $(uname -m) $(pidof pidof)"})
and I see:

#2 Build
#2 0.064 now: 1676391142 aarch64 15 <---------------
#2 DONE 1.1s

#4 from index.docker.io/golang:alpine
#4 resolve docker.io/library/golang:alpine 0.1s done
#4 CACHED

#2 Build
#2 0.111 now: 1676391143 x86_64 21 <---------------
#2 0.111 now: 1676391143 x86_64 21 <---------------
#2 DONE 1.2s

#3 host.directory /Users/kylepenfound/scratch/repro-214
#3 transferring /Users/kylepenfound/scratch/repro-214: 3.91kB done
#3 DONE 0.0s

#4 from index.docker.io/golang:alpine
#4 resolve docker.io/library/golang:alpine 0.1s done
#4 CACHED

#2 Build
#2 -1.032 now: 1676391142 aarch64 15 <---------------
#2 DONE 1.2s

#2 Build
#2 DONE 1.2s
mild jasper
#

I've narrowed it down further. Reduced repro incoming

#

https://gist.github.com/kpenfound/0de9abd133344fb4e69f31255d5e96fa
run go run main.go vs DO_REPRO=1 go run main.go.
some notes:

  • i can only repro when the image is passed to publish. If I get an exit code instead, there's no double output
  • the second condition (which the DO_REPRO env var toggles) is that publish has to publish the secondary container, not the build container
Gist
reduced repro

reduced repro. GitHub Gist: instantly share code, notes, and snippets.

#

another random interesting note: the x86 output always shows the same execution time for both, but the arm64 output (my host arch) always shows a negative number for the second output

worldly wren
#

Interesting findings! Thank you for simplifying this. Yes I can confirm I saw the same

i can only repro when the image is passed to publish. If I get an exit code instead, there's no double output

#

One more interesting detail. I did a dagger-graph of the run and it looked really convoluted and it also showed the run happening 4 times.

mild jasper
#

oooh nice let me add that to my repro notes