#Dagger and Disk space issues on Github actions

1 messages · Page 1 of 1 (latest)

frail bane
#

Apologies if this is the wrong place to ask but I have been trying to track down an issue on Github Actions where the runner is running out of disk space. My build and my source repository have not massively changed and I can’t figure out why this is happening. At present I am just at the theory stage and trying to rule out a few things. Is it possible that I am pulling a huge amount of cached layers from dagger cloud? I am just testing the build without, but my theory on this is that the first build seems to work fine, but when I merge into main it fails with out of space. Could there be something with pulling large amounts of cache?

I am not going to lie, I am clutching at straws here, my next step is to employ the services of Hercule Poirot.

hushed timber
#

Hey Nic,

It's possible that caching has something to do with it. I am investigating on our end now. Ill get back to you here soon.

frail bane
#

Thanks Lev, I am almost done for the day, and apologies again for causing you any work. It is a curious one, I can run some tests in the morning. The build with Dagger Cloud disabled has just passed, will re-instate tomorrow now that my release has gone out

#

These are the CI runs for info

hushed timber
frail bane
#

ty Lev, have a great evening

frail bane
#

@hushed timber is there any way I can determine the size of the cache? I have just run the following workflow and got an out of space error. Looking at this now it seems that I am only using go cache for mods.

karmic echo
#

Maybe run a df -h inside the engine?

docker exec -it $(docker ps --filter "name=dagger-engine" -q) df -h
#

Mine may be due to my Docker Desktop virtual disk limit

image.png
#

under setttings (gear icon) > Resources > Advanced

# 
docker exec -it $(docker ps --filter "name=dagger-engine" -q) df -h
Filesystem                Size      Used Available Use% Mounted on
overlay                  58.4G     18.2G     37.2G  33% /
tmpfs                    64.0M         0     64.0M   0% /dev
shm                      64.0M         0     64.0M   0% /dev/shm
/dev/vda1                58.4G     18.2G     37.2G  33% /etc/resolv.conf
/dev/vda1                58.4G     18.2G     37.2G  33% /etc/hostname
/dev/vda1                58.4G     18.2G     37.2G  33% /etc/hosts
/dev/vda1                58.4G     18.2G     37.2G  33% /var/lib/dagger
/dev/vda1                58.4G     18.2G     37.2G  33% /etc/dnsmasq-resolv.conf
overlay                  58.4G     18.2G     37.2G  33% /etc/resolv.conf
#

I just upped it to 512 GB

image.png
# 
docker exec -it $(docker ps --filter "name=dagger-engine" -q) df -h
Filesystem                Size      Used Available Use% Mounted on
overlay                 469.0G     18.2G    427.0G   4% /
tmpfs                    64.0M         0     64.0M   0% /dev
shm                      64.0M         0     64.0M   0% /dev/shm
/dev/vda1               469.0G     18.2G    427.0G   4% /etc/resolv.conf
/dev/vda1               469.0G     18.2G    427.0G   4% /etc/hostname
/dev/vda1               469.0G     18.2G    427.0G   4% /etc/hosts
/dev/vda1               469.0G     18.2G    427.0G   4% /var/lib/dagger
/dev/vda1               469.0G     18.2G    427.0G   4% /etc/dnsmasq-resolv.conf
overlay                 469.0G     18.2G    427.0G   4% /etc/resolv.conf