#Hi all, I'm giving Dagger + Elixir a try
full plover
The uploads dir mentioned in the error has ~400MB of development files in it.
I don't expect it to be included in the container though, as I don't include it. I'm including files/dirs from the source dir like this:
|> Container.with_directory("/app", source_dir,
include: [
".credo.exs",
".dialyzer_ignore.exs",
".formatter.exs",
"mix.exs",
"mix.lock",
"assets",
"checks",
"config",
"lib",
"packages",
"priv",
"rel",
"test"
]
)
Note the lack of uploads in the list.
I've just emptied the uploads dir to see if / how much it might help.
full plover
Update: Removing that large folder seemed to delay the outcome, but ultimately, after more runs of dagger call, I've hit the error again.
A peek at disk usage from inside the dagger-engine container
Filesystem Size Used Available Use% Mounted on
overlay 58.4G 58.4G 0 100% /
tmpfs 64.0M 0 64.0M 0% /dev
shm 64.0M 0 64.0M 0% /dev/shm
/dev/vda1 58.4G 58.4G 0 100% /etc/resolv.conf
/dev/vda1 58.4G 58.4G 0 100% /etc/hostname
/dev/vda1 58.4G 58.4G 0 100% /etc/hosts
/dev/vda1 58.4G 58.4G 0 100% /var/lib/dagger
/dev/vda1 58.4G 58.4G 0 100% /etc/dnsmasq-resolv.conf
overlay 58.4G 58.4G 0 100% /etc/resolv.conf
Docker Desktop reports volume usage of 20.6GB for the /var/lib/dagger target. Looks like most of it is stored under worker/snapshots.
wispy salmon
Not sure if it is the cache that consume your disk space. Try dagger core dagger-engine prune might help.
full plover
Thanks for the suggestion.
That does help. It frees space and allows me to run dagger commands again without deleting the container.
I'm assuming this isn't a normal situation with Dagger?
Any thoughts on how I might discover a solution?
wispy salmon
The disk usage size is look normal to me. So many questions that can help you investigate here:
- How much files that defined in include that put to Dagger Engine?
- What do you do in the pipeline (test, dialyzer, etc.)?
- How much of disk size that you allocate in Docker Desktop?
bold olive
You might need to set up ignores like I've been discussing in this thread - https://discord.com/channels/707636530424053791/1288908856662949928 - i bet you're uploading your deps, _build, .elixir-ls dirs
full plover
Thank you @wispy salmon and @bold olive for the responses!
During my first attempts I was not ignoring files and ran out of space very quickly.
I then changed to include only some files. It helped slow the problem, but I still run out of space.
Here's what I include from my application source:
|> Container.with_directory("/app", source_dir,
include: [
".credo.exs",
".dialyzer_ignore.exs",
".formatter.exs",
"mix.exs",
"mix.lock",
"assets",
"checks",
"config",
"lib",
"packages",
"priv",
"rel",
"test"
]
)
How much files that defined in include that put to Dagger Engine?
The disk space used by those included files is 55 MB.
> du -hsc .credo.exs .dialyzer_ignore.exs .formatter.exs mix.exs mix.lock assets checks config lib packages priv rel test
8.0K .credo.exs
4.0K .dialyzer_ignore.exs
4.0K .formatter.exs
8.0K mix.exs
56K mix.lock
96K assets
12K checks
44K config
4.3M lib
15M packages
32M priv
40K rel
3.2M test
55M total
What do you do in the pipeline (test, dialyzer, etc.)?
The dagger function uses 1 container to build a React app, 1 container to build an Elixir release, takes directories from each, and finally outputs a final release container.
Both the React and Elixir containers use
Container.with_directory("/app", source_dir,
include: [
# ...
])
This disk space used by the React app includes is 8.8 MB.
On the React side, node_modules and yarn cache are mounted like this:
|> Container.with_mounted_cache("/node_modules", node_modules_cache)
|> Container.with_mounted_cache("/yarn", yarn_cache)
On the Elixir side, deps and _build are mounted like this:
|> Container.with_env_variable("MIX_BUILD_ROOT", "/_build")
|> Container.with_env_variable("MIX_DEPS_PATH", "/deps")
|> Container.with_mounted_cache("/deps", deps_cache)
|> Container.with_mounted_cache("/_build", build_cache)