#Should `apk update` run inside registry.

1 messages · Page 1 of 1 (latest)

pastel marsh
#

I did absolutely barbaric fix, but this works

alpine = dag.container().from_("alpine:3.20.6")
base = (
    dag.container()
    .from_("registry.dagger.io/engine:v0.17.1")
    .with_env_variable("DAGGER_VERSION", "0.17.1")
    .with_env_variable("_EXPERIMENTAL_DAGGER_RUNNER_HOST", "unix:///var/run/buildkit/buildkitd.sock")
    .with_directory("/lib/apk/db", alpine.directory("/lib/apk/db"))
    .with_directory("/etc/apk", alpine.directory("/etc/apk"))
)
sharp vigil
#

Hmm, this doesn't look right the right solution. Are you trying to run dagger-in-dagger?

pastel marsh
#

Exactly, my application needs to run dagger transient containers (agents stuff) and I want it containerized to use simple autoscaling stuff from AWS

sharp vigil
#

It's not well documented, but @rancid matrix may be able to help out

pastel marsh
#

Hm is this not for cases when you have baremetal access to machine?

If my python code runs in a container and needs to call dagger I need dagger engine inside the container, that was my logic so far

sharp vigil
#

Can Dagger run your python code?

pastel marsh
#

App call chain looks like this
[Python Rest Server] -> [Spaw dagger sandbox and run commands] -> [Return response to user]

and the trick is getting this containerized and placed under a common gateway

sharp vigil
#

Instead of having a python server to front it, if you install the dagger CLI you won't need python at all. You can run the dagger sandbox directly and run commands. But I'm assuming you are doing some parsing/filtering of the output from dagger engine? If Go is an option, you can directly use the dagger SDK in go to achieve that too.

#

And the Go piece can be a standalone binary (no other local deps)

pastel marsh
#

My application is a service which runs agents. Agents need dagger to execute commands in sandboxed environments. I need to be able to communicate with it using HTTP requests, seems like dagger in docker is unavoidable or I'm missing something

rancid matrix
#

Another option which we use frequently in the dagger test suite that requires dagger-in-dagger:

  • Get a container that's the appropriate runtime for your python code
  • install the dagger binary: here you can either use the github.com/dagger/dagger repo as a module and get the CLI, or call the install script
  • On the with_exec that executes your python script that needs to shell to dagger, set the option experimental_privileged_nesting=True. This will let the inner container use the outer engine
pastel marsh
#

Yeah, that's exactly what I did, but I need the inner dagger engine to execute and cache some images

#

It works so far without cahcing, slower to start but works. Ideally yeah, I would like to us the dagger engine inside the container to cache the images

#

I tried weird stuff like

with_exec(["sh", "-c", "/usr/local/bin/dagger-entrypoint.sh & uv run init_images.py"], insecure_root....)

to no avail though, it starts but doesn't pull images

#

In short - my pattern is have dagger available inside my container like I have my grep and sed 😆

Then push my app to aws ECS of k8s

rancid matrix
#

So you can run the actual engine with dagger-in-dagger, it's just more complicated than the nesting option. But it sounds like you need to!
Here's how we do that part for the dagger test suite. Basically build the engine container and bind it as a dagger service with the proper options. You could actually use this function directly from this module in your code
https://github.com/dagger/dagger/blob/main/.dagger/engine.go#L152-L169

GitHub
dagger/.dagger/engine.go at main · dagger/dagger

An open-source runtime for composable workflows. Great for AI agents and CI/CD. - dagger/dagger

pastel marsh
#

As a service this is gonna be the neighboring container, right?

rancid matrix
#

Yes, so you'll set _EXPERIMENTAL_DAGGER_RUNNER_HOST to point to that binding in the app container

pastel marsh
#

Then in deployment I spawn a number of dagger engines and my application servers and the figure out the networking to connect them?

rancid matrix
#

They could use the same service since they'll have their own session and it's all on the same compute anyway. I think spawning multiple engines would use a lot of extra compute. Unless I'm missing something

rancid matrix
pastel marsh
#

I would love advice on that. My app runs agent (with BFS / tree searches) and in parallel executes multiple dagger sandboxes. Depending on tree width could be 20-50. I fear with multiple users a single dagger engine could get overloaded

#

Hence I think having each client it's own dagger engine and scale them as a unit is better design

#

Offtopic: coming to London hack night? (:

rancid matrix
#

Yep that makes sense! In what we've talked about so far, whether the inner engine is run inside the process(es) or as a service binding, all of that is still running on a single pod. It sounds like you probably want to scale out your app across the cluster

rancid matrix
pastel marsh
#

Nice, see you there!

Yeah, scaling across the cluster is the idea. On a first glance baking dagger inside the container is simpler, since AWS ECS will do that for me and spawn these instances on separate machines as needed under a scaling group

#

The only challenge I couldn't solve is using dagger to run inner dagger and populate the engine cache 😆

rancid matrix
#

Nice yeah I don't know all of the contstraints but another option could be to run the engine as a sidecar and install the cli in your app's runtime, and then it's "basically" there

pastel marsh
#

That seems like the cleaner option, but I'm no devops magician to write placements policies. I'm a simple creature - make container -> scale container

rancid matrix
#

Maybe I can help at the hack night!