#Cache Volume Expecations

1 messages · Page 1 of 1 (latest)

latent barn
#

I was wondering what the expectations around cache volumes are.
Should I expect the cached volumed to persist between invocations of my program? As in, does each volume entry it persist within the buildkitd instance by the named key?

e.g.

// initialize some golang container in "golang" variable.
golang.
  WithMountedCache("/root/.cache/go-build", client.CacheVolume("go-build")).
  WithMountedCache("/go/pkg/mod", client.CacheVolume("go-mod")).
  WithExec([]string{"go", "build", "./..."})

I tweaked this example for brevity. I've also explored using go.sum files dagger ID as a cache key.

For these static keys, I started out assuming that each time I run my program, the same contents of this volume will persist between runs.

Or is caching just a way to persist state between execs and containers in a single client session?

I ask as I wanted to explore caching Go's module and build cache between invocations of my build program.
However, when I list the contents of the cache (WithExec([]string{"ls", goCache})) it always starts out empty and I get a full build from scratch.
Wondering if my expectations are wrong, or I am doing something wrong, or something isn't working as expected.
Thanks!

#

Hmmm no. That really is just an initial source I see.

latent barn
#

Using GODEBUG=gocachetest=1 doesn't produce any output. Which I think means that it is actually using cache.

At-least when I run with that locally before and after go clean -cache it only produce output after a clean.

slim creek
#

Your expectation sounds right, caches should persist between invocations of your build program, so I'm not immediately sure why that's not the behavior you're getting. If you do something like this:

golang.
  WithMountedCache("/root/.cache/go-build", client.CacheVolume("go-build")).
  WithMountedCache("/go/pkg/mod", client.CacheVolume("go-mod")).
  WithExec([]string{"ls", "/root/.cache/go-build"}).
  WithExec([]string{"go", "build", "./..."}).
  WithExec([]string{"ls", "/root/.cache/go-build"})

Can you see that the go build cache dir actually has contents after the call to go build?

#

Just to be 100% sure it's actually getting populated (and then emptied out in a subsequent run for unknown reasons)

latent barn
#

Thanks for the help 🙌

So I get a whole lot of nothing:

#24 ls /root/.cache/go-build

#24 DONE 0.1s

#25 sh -c go build -trimpath -tags assets -x -o ./bin/linux/arm64 ./... 2>&1 | grep "compile" | wc -l
#25 47.71 486
#25 DONE 47.8s

#26 ls /root/.cache/go-build

go run cmd/build/main.go  0.28s user 0.18s system 0% cpu 55.299 total

For more context I have these options:

WithMountedCache(goBuildCachePath, cacheGoBuild).
        WithMountedCache(goModCachePath, cacheGoMod).
        WithMountedDirectory(".", project).
        WithMountedFile("./ui/embed.go", embed.File("./ui/embed.go")).
        WithMountedDirectory("./ui/dist", ui.Directory("./dist")).
        WithExec([]string{"mkdir", "-p", target}).
        WithExec([]string{"ls", goBuildCachePath}).
        WithExec([]string{"sh", "-c", cmd}).
        WithExec([]string{"ls", goBuildCachePath})

And when I run it again after changing some of my go main.go source it does the exact same thing.
I see go build -x outputting a whole lot of compile all the things.

#

goBuildCachePath is "/root/.cache/go-build" I just acquire it further up by doing an exec and running go env GOCACHE in the container to get it.

#

You know what I think I may have spotted it.

goBuildCachePath has a newline on the end of it. As in:

"/root/.cache/go-build
"

I have been mounting it with that newline.

#

Yep that did it

#

Is this something that could be improved in terms of sanitizing or validating paths passed to WithMountedCache ?
Is there a use-case where \n in a mount path would be valid?

slim creek
#

That being said, putting \n in your path is incredibly obscure

latent barn
#

That makes total sense. I wondered as much.
Definitely dont want to put in arbitrary rules because a usecase isn't immediately obvious.

Maybe this post can just exist as a cautionary tale 😂
This is really all because I retrieved the go mod cache and build cache paths dynamically (and didn't think to sanitize the output). Shot myself in the foot there.

icy phoenix
latent barn
#

@icy phoenix if you invoke “go env GOCACHE” in an exec in the container it prints the path (with a trailing newline - beware).

icy phoenix
#

@latent barn thanks for the heads up.

I was wondering if you used dagger to do that programmatically or if you mean that you used the docker CLI to exec in and get the value.

Trying to automate a delivery process with dagger and I'm hoping to have the ability to execute arbitrary commands in a container using the dagger go SDK without producing an artifact. Not sure if thats possible. Seems like you have to call either Export or Publish to get the dag to run.

#

Nvm, I think I just figured out the answer facepalm

#

Stdout

red magnet
# icy phoenix `Stdout`

Yep, or ExitCode if you don't care about getting the log output and just want to know if it failed or not.

latent barn
#

Ah yes, apologies I wasn't explicit .WithExec([]string{"go", "env", "GOCACHE"}).Stdout() to get the cache location.
I used strings.TrimSpace() on the result to remove the trailing newline which causes silent havoc with the mount capabilities.

red magnet
#

Oh, good to know.

#

Also, instead of getting the go env GOCACHE env var to mount your cache volume there, you could set the GOCACHE env var yourself to the directory where you're gonna mount your cache.