Bug caching distinct services when they should be separate

[bright bronze]

I'm instantiating multiple of the same service and binding them to different containers. I was noticing strange issues, until I finally figured out that the different services I thought I had ended up just being the same service due to caching.

I've reduced the issue to the following code sample you can replicate. I load 2 HTTP server's to serve data from different cache bind mounts, and expect each one to serve different data. But then both of them appear to show the same response (because it ends up being the same service).

func (m *Treasury) TestEnv(
    ctx context.Context,
) (*dagger.Container, error) {

    devContainer := dag.Container().From("alpine:latest")

    var wg sync.WaitGroup

    for i := 0; i < 2; i++ {
        i := i
        wg.Add(1)
        go func() {
            cache := dag.CacheVolume(fmt.Sprintf("cache-%d", i))
            _, err := devContainer.
                WithMountedCache("/shared", cache).
                WithExec([]string{"sh", "-c", fmt.Sprintf("echo 'SERVER %d' > /shared/index.html", i)}).
                Stdout(ctx)
            if err != nil {
                panic(err)
            }

            service := dag.Container().
                From("python").
                WithMountedCache("/shared", cache).
                WithWorkdir("/srv").
                WithExec([]string{"sh", "-c", "cp /shared/index.html /srv/"}).
                WithExec([]string{"python", "-m", "http.server", "8080"}).
                WithExposedPort(8080).
                AsService()

            serviceName := fmt.Sprintf("www-%d", i)
            clientContainer := devContainer.
                WithEnvVariable("HOST", serviceName).
                WithServiceBinding(serviceName, service)

            output, err := clientContainer.
                WithExec([]string{"wget", "-O-", fmt.Sprintf("http://%s:8080", serviceName)}).
                Stdout(ctx)

            if err != nil {
                panic(err)
            }
            fmt.Println(output)

            wg.Done()
        }()

    }
    wg.Wait()

    return devContainer, nil
}

https://dagger.cloud/cordialsys/traces/21bf3cd32bc667bdacf6446f10fb4013

I expect the output to have different lines, like:

SERVER 0
SERVER 1

But instead it's like:

SERVER 0
SERVER 0

[inland thistle]

Thanks for the great reproduction, its half the battle sometimes!

I think since the inputs don't change in your service variable the computed result is re-used. This is normally good because it speeds things up, but in this case is undesired behavior.

I think you can either use the "cachebuster" pattern on your service https://docs.dagger.io/cookbook/#invalidate-cache or introduce some different inputs.

One idea I had was to make sure you're using a different cache volume name depending on the service number. This makes sure that your services don't interfere with each other but also solves this service being cached problem.

That approach worked for me:

service := dag.Container().
    From("python").
    WithMountedCache(fmt.Sprintf("/shared-%d", i), cache).
    WithWorkdir("/srv").
    WithExec([]string{"sh", "-c", fmt.Sprintf("cp /shared-%d/index.html /srv/", i)}).
    WithExec([]string{"python", "-m", "http.server", "8080"}).
    WithExposedPort(8080).
    AsService()

[bright bronze]

got it, thanks!

I think it should cache all the steps for speed, but why should it be the same service? I would expect two copies of the service.

It may be different services, but maybe both are using the first cache.. not sure.

[inland thistle]

Overall caching is strictly based on inputs, so if the inputs do not change then the step will be cached. This is why the "cache buster" works because it introduces a volatile input.

I am surprised though that cache := dag.CacheVolume(fmt.Sprintf("cache-%d", i)) in your original code was not enough to change the "input" and invalidate the cache :/

In addition, I also do agree that services are special and different, in fact I thought they were already treated differently.

@bronze mantle do you have any thoughts here? I may be missing something.

[bright bronze]

not causing problems for me anymore, but thought i'd bring it up, feels fishy 🙂