Magic cache in 0.11.4?

[graceful igloo]

hey guys, me again. Probably missing something dumb here.... could use a pointer.

I've go ta simple module setup, and I'm trying to test out cache volume caching with dagger magicache. I've tinkered quite a bit, still not seeing my cache vol in the "configuration" page in cloud. I do see the traces from my pipeline run.

Steps I've taken.

1. Completely nuke dagger engine/volume.
2. export dagger cloud token
3. dagger call touch-file
4. check logs / cloud console

ubuntu@ip-10-64-32-26:~/tmp$ dagger version
dagger v0.11.4 (registry.dagger.io/engine) linux/amd64

My module.

package main

import (
        "context"
)

type Test struct{}

func (m *Test) TouchFile(ctx context.Context) (string, error) {
        mod := dag.CacheVolume("tobydumbtest")
        return dag.Container().
                From("alpine:latest").
                WithMountedCache("/vol", mod).
                WithExec([]string{"touch", "/vol/test"}).
                Stdout(ctx)
}

Run in question if you have access there -> https://dagger.cloud/ourskyai/traces/9d8131b6c9e78a202a77c0c1bfe42a10#9abcdd8fc4ed1ae0

Interesting logs showing volume import, and some type of export, but my volume still missing.

(at engine boot)
time="2024-05-13T00:55:06Z" level=debug msg="parsing cache config"
time="2024-05-13T00:55:06Z" level=debug msg="finished parsing cache config in 19.174757ms"
time="2024-05-13T00:55:06Z" level=debug msg="finished importing cache in 1.989309449s"
time="2024-05-13T00:55:06Z" level=debug msg="engine name: 965b75bcfcc5"
time="2024-05-13T00:55:06Z" level=debug msg="starting optional cache mount synchronization"
time="2024-05-13T00:55:07Z" level=debug msg="syncing cache mount locally modgobuildcache"
time="2024-05-13T00:55:07Z" level=debug msg="syncing cache mount locally modgomodcache"
time="2024-05-13T00:55:07Z" level=debug msg="created new ref for cache dir \"avVT/m+4OLXQgQB+MOWhs3+oJGHEuzbaeAUv6wqhAqo=\": qbpbokrxlkggdit8zimdaspo0"
time="2024-05-13T00:55:07Z" level=debug msg="created new ref for cache dir \"Abmf3C+9LSJja/F87o7zZ3wsgKx6I3bs/0Q+AEiy5S4=\": xb0njuewqkaclwqr2ynoh03fg"
time="2024-05-13T00:55:08Z" level=debug msg="synced cache mount locally modgobuildcache"
time="2024-05-13T00:55:08Z" level=debug msg="synced cache mount locally modgomodcache"

some time after run...

time="2024-05-13T01:00:24Z" level=debug msg="finished pushing layers for cache ref x3i42ymp241wjopmo515e6b74 in 401.973862ms"
time="2024-05-13T01:00:24Z" level=debug msg="finished exporting cache ref x3i42ymp241wjopmo515e6b74 in 406.213584ms"
time="2024-05-13T01:00:24Z" level=debug msg="exporting cache ref rx8au33r6rytfdis0b0txur51"
time="2024-05-13T01:00:24Z" level=debug msg="getting remotes for cache ref rx8au33r6rytfdis0b0txur51"

[lone moat]

@graceful igloo cache volumes get synced when the engine stops. If you docker stop -300 $engine_container you should see your cache volumes in cloud after the engine finished stopping

[graceful igloo]

bingo! thank you!!!

[graceful igloo]

hey @lone moat - following up on an old thread here, I'm trying to troubleshoot this behavior with v0.11.8 and not seeing any logs related to exporting cache ref .

has any of this behavior changed?

Just seeing

--
time="2024-06-20T00:30:33Z" level=debug msg="stopping cache manager"
time="2024-06-20T00:30:33Z" level=info msg="stopping server"
time="2024-06-20T00:30:33Z" level=debug msg="cleaning up cni pool"
dnsmasq exited: signal: killed
time="2024-06-20T00:30:34Z" level=debug msg="shutting down otel pub/sub"
time="2024-06-20T00:30:34Z" level=debug msg="shutting down otel pub/sub"

[lone moat]

hey Steven! no it hasn't. I'd double check if the engine has the correct DAGGER_CLOUD_TOKEN set. That's required for caching to work.

I just tested it in v0.11.8 and still works.

ime="2024-06-20T01:24:05Z" level=debug msg="stopping cache manager"
time="2024-06-20T01:24:05Z" level=debug msg="starting cache export"
time="2024-06-20T01:24:05Z" level=debug msg="starting cache export key store walk"
time="2024-06-20T01:24:05Z" level=debug msg="syncing cache mount remotely modgobuildcache"
time="2024-06-20T01:24:05Z" level=debug msg="syncing cache mount remotely modgomodcache"
time="2024-06-20T01:24:05Z" level=debug msg="skipping cache result va9a4lz8svoactwzizzyq5as6::pcgxow83o6hwqg4s5w0983heg for mdgja0xkgvnwrfvixnfwt8mda: failed to load ref: missing descriptor handlers for lazy blobs [sha256:d25f557d7f31bf7acfac935859b5153da41d13c41f2b468d16f729a5b883634f]"
time="2024-06-20T01:24:05Z" level=debug msg="skipping cache result va9a4lz8svoactwzizzyq5as6::qp22xe2w2ebgcdnc7q8xcsvbm for sha256:fa79334d0ca364c87711129e632b8c023a9a2e2e72a24433d71024082d2ccd64: failed to load ref: missing descriptor handlers for lazy blobs [sha256:d25f557d7f31bf7acfac935859b5153da41d13c41f2b468d16f729a5b883634f]"
time="2024-06-20T01:24:05Z" level=debug msg="skipping cache result va9a4lz8svoactwzizzyq5as6::qp22xe2w2ebgcdnc7q8xcsvbm for sha256:fbc352dbf973f7f55a3449230e72185b169b4ec59160d4712b3c3737bf25387c: failed to load ref: missing descriptor handlers for lazy blobs [sha256:d25f557d7f31bf7acfac935859b5153da41d13c41f2b468d16f729a5b883634f]"
time="2024-06-20T01:24:05Z" level=debug msg="finished cache export key store walk in 24.233834ms"
time="2024-06-20T01:24:05Z" level=debug msg="calling update cache records"
time="2024-06-20T01:24:06Z" level=debug msg="finished update cache records call in 1.025212289s"

[graceful igloo]

Hm.... Might be having other issues I guess? Confirmed it was set, logs told me to regenrate a token, cleared old engine, regened token, and now I get
time="2024-06-20T03:12:14Z" level=error msg="failed to emit telemetry" error="map[error:failed to send logs to https://api.dagger.cloud/v1/logs: 401 Unauthorized kind:*errors.errorString stack:<nil>]"

and same behavior on shutdown

yeah ok so I guess I initted my local container w/o the cloud token, then ran with token set later, so the local container did not have it set