failed to load git dir: ...: not found | Dagger | Page 1

lavish pendant May 13, 2026, 5:11 PM

#

when it happens, sometimes I just run again and it's fine shrug

#

there's a repeated pattern in fact:

run dagger check => runs everything without cache hits
run again => error above
run dagger check => runs everything without cache hits (?!)
run again => error above
[...]

#

ok, that happens on main too

umbral lodge May 13, 2026, 5:21 PM

#

Weird, what's the repro?

#

like the exact steps?

lavish pendant May 13, 2026, 5:23 PM

#

git clone https://github.com/vito/midterm
cd midterm
dagger check
dagger check

(dagger above being dagger/dagger @ b5c5e427c359b6bd0471385d0093cb8d5b547f99)

#

Happens in dagger/dagger too but the above runs faster when it works, so makes for an easier repro

umbral lodge May 13, 2026, 5:27 PM

#

sipsma@dagger_dev:~/repo/github.com/sipsma$ git clone https://github.com/vito/midterm
Cloning into 'midterm'...
remote: Enumerating objects: 4987, done.
remote: Counting objects: 100% (55/55), done.
remote: Compressing objects: 100% (42/42), done.
remote: Total 4987 (delta 21), reused 37 (delta 12), pack-reused 4932 (from 1)
Receiving objects: 100% (4987/4987), 1.53 MiB | 2.35 MiB/s, done.
Resolving deltas: 100% (4543/4543), done.
sipsma@dagger_dev:~/repo/github.com/sipsma$ cd midterm/
sipsma@dagger_dev:~/repo/github.com/sipsma/midterm$ ls
benchmark_test.go  canvas_test.go  format.go  go.mod  handler.go  LICENSE  marshal.go       package.go  render.go  screen.go  search_test.go  terminal_test.go
canvas.go          dagger.json     ghostest   go.sum  html.go     log.go   marshal_test.go  README.md   replay     search.go  terminal.go     testdata
sipsma@dagger_dev:~/repo/github.com/sipsma/midterm$ dagger check
▶ ✔ go:lint-all 34.1s ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣇⡇ OK

▶ ✔ go:test-all 31.4s 10×⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡇⡀⡀⡄ OK

✔ go:generate-all 0.2s ⣿⣿⣿⣿ OK
sipsma@dagger_dev:~/repo/github.com/sipsma/midterm$ dagger check
▶ ✔ go:lint-all 2.4s ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡇⡄⡀⡀ OK

▶ ✔ go:test-all 2.4s ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣧⣿⣿⣿⡀⡀ OK

✔ go:generate-all 0.2s ⣿⣿⣷⣇ OK
sipsma@dagger_dev:~/repo/github.com/sipsma/midterm$ dagger version
dagger v0.21.0-20260512233048-dev-b5c5e427c359 (image://registry.dagger.io/engine:b5c5e427c359b6bd0471385d0093cb8d5b547f99) linux/arm64/v8
sipsma@dagger_dev:~/repo/github.com/sipsma/midterm$ dagger check
▶ ✔ go:lint-all 2.6s ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣷⣿⣿⣿⡀⡄⡄⡀ OK

▶ ✔ go:test-all 2.5s ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣷⣿⣿⣿⡆⡀ OK

✔ go:generate-all 0.2s ⣿⣿⣿⡄⣧ OK

Isn't repro'ing for me yet

#

What's your disk space looking like (df -h or whatever)?

And can you send the logs of your engine? At least the most recent logs where this was happening if the full logs are too much

#

I'm wondering if it's pruning related somehow

lavish pendant May 13, 2026, 5:34 PM

#

❯ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p2  914G  743G  126G  86% /
devtmpfs         63G     0   63G   0% /dev
tmpfs            63G  558M   62G   1% /dev/shm
efivarfs        148K   93K   51K  65% /sys/firmware/efi/efivars
tmpfs            26G  2.9M   26G   1% /run
none            1.0M     0  1.0M   0% /run/credentials/systemd-journald.service
tmpfs            63G  1.1G   62G   2% /tmp
/dev/nvme0n1p1  2.0G  104M  1.9G   6% /efi
tmpfs            13G  136K   13G   1% /run/user/1000
none            1.0M     0  1.0M   0% /run/credentials/systemd-resolved.service

#

will grab engine logs in a sec (gonna re-provision and repro so they're clean)

#

Captured engine logs after running each of these commands:

midterm th3godfather-fix/resize-grow-height-bloat
❯ ~/src/dagger/main/bin/dagger check
▶ ✔ go:lint-all 27.0s ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡆⡄⡀⣧ OK

▶ ✔ go:test-all 27.0s 10×⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡀⡄ OK

✔ go:generate-all 0.2s ⣿⣿⣿⡇ OK

midterm th3godfather-fix/resize-grow-height-bloat 38s
❯ ~/src/dagger/main/bin/dagger check
▶ ✔ load module: go 0.4s
✔ load module: midterm 0.4s
Error: loading modules: loading module "github.com/dagger/go@main": resolving module source "github.com/dagger/go@main": failed to load git dir: xwovk1illdjdy3i3fjcbr7xpn: not found

midterm th3godfather-fix/resize-grow-height-bloat
❯ ~/src/dagger/main/bin/dagger check
▶ ✔ go:lint-all 25.3s ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡆⡀⡆ OK

▶ ✔ go:test-all 23.1s 10×⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣇⡀⡀⡄ OK

✔ go:generate-all 0.2s ⣿⣿⣿⣿ OK

midterm th3godfather-fix/resize-grow-height-bloat 30s
❯ ~/src/dagger/main/bin/dagger check
▶ ✔ go:lint-all 1.2s ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡆⣿⣿⣿⡄⡄⡄ OK

▶ ✔ go:test-all 1.1s ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡆⣿⣿⣿⡄⡄⡄ OK

✔ go:generate-all 0.1s ⣿⣿⣷⣇ OK

midterm th3godfather-fix/resize-grow-height-bloat
❯ ~/src/dagger/main/bin/dagger check
▶ ✔ load module: midterm 0.2s
✔ load module: go 0.2s
Error: loading modules: loading module "github.com/dagger/go@main": resolving module source "github.com/dagger/go@main": failed to load git dir: 4duh1ispwr46abmbpnaxglse6: not found

midterm th3godfather-fix/resize-grow-height-bloat
❯ ~/src/dagger/main/bin/dagger check
▶ ✔ go:lint-all 25.7s ⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣷⡄⡀⡇ OK

▶ ✔ go:test-all 23.5s 10×⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡀⡀⡄ OK

✔ go:generate-all 0.2s ⣿⣿⣿⣷ OK

midterm th3godfather-fix/resize-grow-height-bloat 32s
❯

Interestingly the pattern broke for the fourth run, where it actually hit the caches like I would have expected in the happy path, but the run after then hits the git error and the pattern continues. It does kind of feel like caches are getting pruned

📎 engine-check-1.log 📎 engine-check-2.log 📎 engine-check-3.log 📎 engine-check-4.log 📎 engine-check-5.log

umbral lodge May 13, 2026, 5:48 PM

#

Yep, in the second one there's

time=2026-05-13T17:40:30.035Z level=INFO msg="dagql pruned result" resultID=4142 call=Query._remoteGitMirror digest=xxh3:66e593814d06eb2a args="[remoteURL=\"https://github.com/dagger/go\"]" description=_remoteGitMirror measuredSizeBytes=172032 reclaimedBytes=0

So probably there's some reference to the git checkout ref that's left dangling after prune, must be missing a dependency declaration between things somewhere which would prevent a prune while a reference still exists

#

I'll track it down, thanks! And sorry for the headache! If you are super blocked freeing up disk space should help if possible, the cache won't be trying to prune as aggressively then

lavish pendant May 13, 2026, 5:51 PM

#

np! happy to help push these moles out so they can get wacked 🔨

tardy flower May 13, 2026, 6:19 PM

#

Anything I need to fix in github.com/dagger/go?

#

Or was it just coincidental that it happened in that module?

umbral lodge May 13, 2026, 6:19 PM

#

seems coincidental

tardy flower May 13, 2026, 6:19 PM

#

https://giphy.com/gifs/uefa-uel-uefa-europa-league-2021-CAukyJ5esw36CyRllk

Giphy

UEFA

potent lily May 13, 2026, 6:32 PM

#

Did we identify the commit that introduced it? Just so i can revert locally

umbral lodge May 13, 2026, 6:35 PM

#

potent lily Did we identify the commit that introduced it? Just so i can revert locally

it's probably been there since the beginning of the new cache/pruning system, so not easily revertible. I am working on a fix

potent lily May 13, 2026, 6:36 PM

#

Weird, because i only had this this morning

umbral lodge May 13, 2026, 6:36 PM

#

If you can clean up the disk that should help. I do this every few days and get most of my disk back: docker rm -fv dagger-engine.dev dagger-engine-v0.20.8; docker system prune -a -f; docker image prune -a -f; docker volume prune -a -f

potent lily May 13, 2026, 6:37 PM

#

i'm only having this with dagger-engine.dev right now so why would I need to do the rest ?

umbral lodge May 13, 2026, 6:37 PM

#

potent lily i'm only having this with dagger-engine.dev right now so why would I need to do ...

to save disk space

#

the prunes trigger after a disk space usage threshold

potent lily May 13, 2026, 6:37 PM

#

ahhhh

lavish pendant May 14, 2026, 3:46 PM

#

found the root cause for me

umbral lodge May 14, 2026, 5:32 PM

#

Fix here https://github.com/dagger/dagger/pull/13141

#failed to load git dir: ...: not found