How to use Dagger SDK with a Service or Another Container | Dagger | Page 1

potent moss Dec 22, 2022, 10:15 PM

#

Please can anyone guild me on how to implement dagger with a service container. for example running unit test in one container that require another container (e.g database, etc)

attached is my dagger code using python sdk. not sure how to go about it

📎 gojira.py

chrome fiber Dec 22, 2022, 10:45 PM

#

Hi @potent moss , Dagger does not yet support running services in containers, but it is in development.

#

In the meantime there are workarounds. One workaround is to run Docker alongside Dagger, and mount the Docker socket into a container run by Dagger, so that you can run services on Docker, from Dagger

potent moss Dec 22, 2022, 11:02 PM

#

okay, will try that now

potent moss Dec 23, 2022, 12:02 AM

#

chrome fiber Hi <@650524498319376421> , Dagger does not yet support running services in conta...

I came up with this, from the program configuration i pointed the redis server url to redis instead of localhost. also mounted the socket to both container thinking it will put then on same network.

async def test(client: Client, src: Directory, artifacts: Directory) -> None:
    docker_host: Socket = client.host().unix_socket("/var/run/docker.sock")
    redis: Container = (
            client.container()
                .from_("rancher/dind")
                .with_unix_socket("/var/run/docker.sock", docker_host)
                .with_exec(["docker", "run", "-d", "-p", "6379:6379", "--name", "redis", "redis:6"])
            )

    test: Container = (
            client.container()
                .from_("rust")
                .with_workdir("/gojira")
                .with_unix_socket("/var/run/docker.sock", docker_host)
                .with_mounted_directory(".", src)
                #.with_mounted_directory("/gojira/target", artifacts.directory("docker/build/target"))
                #.with_mounted_directory("/usr/local/cargo", artifacts.directory("docker/build/cargo"))
                .with_exec(["cargo", "test", "run", "--verbose"])
            )

    # execute
    await redis.stdout()
    await test.stdout()
    return None

got stuck at

#

i checked using docker ps --all , container not showing up

potent moss Dec 23, 2022, 4:05 AM

#

was doing some digger, i found this this link "https://docs.dagger.io/1211/go-docker-swarm/" although it was cue example i might be able to port it to the python

arctic hinge Dec 23, 2022, 5:57 PM

#

Any progress on this? I'm trying to do something similar, my approach was to use a dind container and grep for the dagger network to spin the dind container up on. Haven't tried to see if that approach would work yet tho

potent moss Dec 23, 2022, 7:18 PM

#

@arctic hinge yes, i made progress, just having a minor issue, but it works as i wanted, will paste the code in few minute

#

@arctic hinge was thinking i could remote in from where i am to copy the code, but give me 2hrs will be home and paste it here

arctic hinge Dec 23, 2022, 7:29 PM

#

If it works with my setup I will give it a shot, essentially wanting to spin up several services with docker-compose setup

potent moss Dec 24, 2022, 12:47 AM

#

@arctic hinge here is the function, Note this is using the python sdk

async def test(client: Client, src: Directory, artifacts: Directory) -> None:
    docker_host: Socket = client.host().unix_socket("/var/run/docker.sock")
    redis: Container = (
            client.container()
            .from_("docker:dind")
                .with_unix_socket("/var/run/docker.sock", docker_host)
                .with_env_variable("CACHEBUSTER", datetime.now().strftime("%m/%d/%Y, %H:%M:%S"))        # prevent caching from this point
                .with_exec(["docker", "ps", "--all"])
                .with_exec(["docker", "stop", "dagger-redis"])
                .with_exec(["docker", "run", "-d", "--rm", "-p", "6379:6379", "--name", "dagger-redis", "redis:6"])
            )

    test: Container = (
            client.container()
                .from_("rust")
                .with_exec(["cargo", "install", "cargo-nextest"])
                .with_workdir("/app")
                .with_unix_socket("/var/run/docker.sock", docker_host)
                .with_mounted_directory(".", src)
                #.with_mounted_directory("/app/target", artifacts.directory("docker/build/target"))
                #.with_mounted_directory("/usr/local/cargo", artifacts.directory("docker/build/cargo"))
                .with_exec(["cargo", "nextest", "run", "--verbose"])
            )

    # execute
    await redis.stdout()
    await test.stdout()
    return None

#

@arctic hinge one thing to note is that for my test to connect to the redis, i used the docker interface ip address, in my case it was 172.18.0.1

potent moss Dec 24, 2022, 1:15 AM

#

@arctic hinge i wrote a complete example, to fully explain it. thanks @chrome fiber for helping figure it out

📎 message.txt

arctic hinge Dec 24, 2022, 2:53 PM

#

This is great, I will try this out and report back!

arctic hinge Dec 28, 2022, 9:08 PM

#

Alright, I was able to get this to work with docker compose. At least for MacOS, I haven't tried yet to set up Linux.

Basically I wanted a fresh, local supabase(https://supabase.com/) instance to run nextjs app integration tests against. Supabase is a series of 8 or so containers as you can see in the docker-compose.yml file referenced in the code.

The idea is basically:

Spin up containers using a docker-compose file that all reference each other by service name and expose ports on the host
Spin up a second container in a different network containing the next-js app that hits the exposed container endpoints via the host network. The way that the next-js app "sees" the services is via host.docker.internal. There is a test .env file in the mounted next-js app that has host.docker.internal in its environment variables

I ran into some weird behavior around volumes. Dagger wanted to look inside the container for volume paths that existed on the host, so I had to export the paths out to a shared volume on the host. Otherwise everything else was pretty straightforward.

Someone could try this themselves pretty easily by taking this and having their own nextjs app that consumes supabase services and update it to use host.docker.internal in the environment variables, otherwise using the code as-is

Some todo's

Tear down docker compose if a container errors, not just if it succeeds
Make it work with linux somehow (I haven't tried it on a linux machine yet, but my understanding is that host.docker.internal doesn't 'just work' on linux). One option is to do like what @potent moss did and reference the IP directly (this can be derived programmatically with some exec commands). Another is to figure out how to start with --add-host=host.docker.internal:host-gateway

Lastly, if anyone else is thinking about doing this, don't try to use supabase CLI. It's not possible currently due to a lot of hardcoded assumptions in the source that don't work with containers

📎 example-docker-compose.txt

Supabase

The Open Source Firebase Alternative | Supabase

The Open Source Alternative to Firebase.

arctic hinge Dec 28, 2022, 9:24 PM

#

For @chrome fiber and @grizzled dagger I think the thing that would have helped me a lot and given me a lot more control was somehow be able to specify what network the container would run in, not being able to do that really made me feel like I was hampered a lot. Like, one thing that would have been nice is to just say that the node app should run in docker_default network alongside the supabase containers, that way I could have just said "http://supabase-kong:8000" in my nextjs environment variables (which is the service endpoint defined in docker-compose.yml) instead of http://host.docker.internal:8000. Perhaps with that approach we could avoid using host networking entirely

chrome fiber Dec 28, 2022, 9:41 PM

#

arctic hinge For <@488409085998530571> and <@336241811179962368> I think the thing that would...

I believe you can specify which network to use directly in the compose file, perhaps that is what you need in this scenario?

arctic hinge Dec 28, 2022, 9:42 PM

#

chrome fiber I believe you can specify which network to use directly in the compose file, per...

I mean I want to say "dagger, run my node app in this existing network" - that way I can keep the compose file as its own thing and then just arbitrarily run containers in dagger that can "see" the services that are specified in the compose file

#

something like:


        next: Container = (
                client.container()
                .from_("node:alpine")
                     
                    .with_mounted_directory("/src",local_src) # a simple nextjs app with .env referencing services that exist in docker_default network by name e.g. http://supabase-kong:8000
                    .with_network("docker_default") # this could be some other network that I define in the docker-compose.yml. Or really just any network
                    .with_workdir("/src")
                    .with_exec(["npm", "install"])

                    .with_env_variable("CACHEBUSTER", datetime.now().strftime("%m/%d/%Y, %H:%M:%S")) 
                    
                    .with_exec(["npm", "run", "bootstrap"]) # code that seeds db/runs tests etc
                )

fossil slate Dec 29, 2022, 2:55 PM

#

arctic hinge I mean I want to say "dagger, run my node app in this existing network" - that w...

Hi @arctic hinge,
I agree with you. I have felt the same need 😇

However, I don't think this is the direction that we are currently taking to solve that networking limitation.
Even though it might indeed help solve the need short term, there are several issues / PRs discussing the design of a long-term / more scalable networking API, to hide complexity to end-users while letting them have a strong networking control inside the Dagger API (instead of relying on other tools)

All the context can be found here: https://github.com/dagger/dagger/issues/4080 👼

Nonetheless, thank you for your feedback, it is very insightful. Do not hesitate to add your opinion on those issues / PRs 🙏

arctic hinge Dec 29, 2022, 2:58 PM

#

That makes sense! thanks for sharing!

chrome fiber Dec 29, 2022, 10:38 PM

#

What Guillaume said 👆

And to add some extra context on the design tradeoffs…

I understand the desire to say “run in this docker network I already have”. However if we added that ability to the core API, we would have a core dependency on a proprietary feature of the Docker platform, at the expense of eg. kubernetes which does networking very differently (ironically we tried and failed to unify the container ecosystem around docker’s networking model, but I digress).

We could try to add some sort of networking plugin system that could support both docker, kubernetes and the various other systems out there. It would be a lot of effort with imperfect results, because the container networking space is so fragmented and APIs so incomplete. For example there is no reliable way to get a service IP from the Docker or Kubernetes API: the method you describe is highly specific to your setup and not universally portable - so we would have to support the multi-dimensional matrix of supported platforms and plugins…

OR we could do none of that, and give you a simple API to run services and access their porte in a reliable and portable way, which Docker and Kubernetes cannot do. This makes the whole experience easier and more reliable end-to-end. The tradeoff is that as an operator you have less control over the underlying networking backend (for now). But since Dagger is made to run dev services, not prod… we think it’s a good tradeoff 🙂

#

In practical terms, in your case this would mean running your entire set of docker compose services in Dagger, and use the Dagger networking API to interconnect them with minimal effort

arctic hinge Dec 31, 2022, 4:11 PM

#

Thanks for expounding. I think from a DevEx perspective one thing that would be nice is to either

provide docker-compose.yml compatibility with dagger in some way

or

make it straightforward and ergonomic for people to support porting their docker-compose.yml stuff over to a dagger pipeline (since a lot of people think about container networking in the context of docker-compose in my experience). Maybe from what you are saying option number two is better since someone could do the same for their k8's CRDs

If for my use-case all that means is that I need to iterate through the docker-compose yml to pull out and possibly massage some values that are then passed to this networking API, it's a bit of an extra step, but seems doable to me and not an undue burden on the dev to make it work. Maybe then an article like "run E2E/integration tests with docker-compose in dagger" is just demonstrating which values in the docker-compose.yml need to be instantiated in the container and networking api. That way the person who is running docker-compose (or k8's, or whatever) in prod can keep doing so for their setup but also have an automatable, easy-to-understand translation of their production networking model into dagger

chrome fiber Dec 31, 2022, 7:21 PM

#

I think option 1. will become trivial on top of the services API 🙂

#

I agree that docker-compose compat out of the box will be a huge improvement for many projects

#

We will make sure to get there

grizzled dagger Jan 1, 2023, 7:21 PM

#

arctic hinge For <@488409085998530571> and <@336241811179962368> I think the thing that would...

definitely part of the services / networking features as Solomon and Guillaume already commented. Nothing to add here.

Another is to figure out how to start with --add-host=host.docker.internal:host-gateway
this is possible to do for linux engines, but seems like a non-very useful hack until we come up with the proper networking feature. I'd keep using host.docker.internal for DD and manually changing the gateway IP for linux until we ship services.

arctic hinge Jan 8, 2023, 9:05 PM

#

@grizzled dagger how do I go about "manually changing the gateway IP for linux" here?

#How to use Dagger SDK with a Service or Another Container