Exposing multiples services to host in one command

[long kindle]

Hi, I'm looking at Dagger as a replacement for Docker compose for local development. We use Docker Compose to start up service dependencies and expose them to the host. I'd like to have one dagger call command that starts up multiple services and runs them. This example code is what I'm thinking. Is there a way using the dagger call CLI to call multiple services up command at once, or a way to use the SDK to set up a functions like devDeps that starts them all?

  @func()
  postgresService(): Service {
    return dag.container()
      .from("postgres:16")
      .withEnvVariable("POSTGRES_PASSWORD", "postgres")
      .withExposedPort(5432)
      .asService()
  }

  @func()
  minioService(): Service {
    return dag.container()
      .from("minio/minio:latest")
      .withEnvVariable("MINIO_ACCESS_KEY", "minioadmin")
      .withEnvVariable("MINIO_SECRET_KEY", "minioadmin")
      .withExposedPort(9000)
      .withExec(["server", "/data", "--address", ":9000"])
      .asService()
  }

  @func()
  devDeps(): Service[] {
    return [this.minioService(), this.postgresService()]
  }

[elder rain]

Hello!

It's possible to get all the services up and running with a single call using WithServiceBinding and returning a container.

Here's a concrete example: https://github.com/levlaz/snippetbox/blob/main/ci/main.go#L126

but the way that mounting volumes currnetly works in Dagger is not ideal for local development more here:

https://github.com/dagger/dagger/issues/6990

Here's another approach using this proxy module: https://daggerverse.dev/mod/github.com/kpenfound/dagger-modules/proxy@33bc3eef07555d9fd4c22b02ff5c008a71a649e7

[long kindle]

Thanks for that link. I still plan on running code directly on my laptop, I'd just like to expose all our service dependencies for our code with ports bound to the host. That proxy module looks a little like what I want, I was just wondering if there was an up-all function or something I could write to expose multiple services to the host in one line. Don't need hot reloading or to run my application code in the containers

Really appreciate the quick resoonse, will try out the proxy module now

[elder rain]

Got it - so this is just for all the other services, makes sense!

I think the proxy module is the best bet - @rugged tinsel and I were just dicussing today that it would be interesting to think about having the functionality t hat the proxy module implements be a built in part of the core API

[rugged tinsel]

Yep! Definitely enjoy it 🙂
https://github.com/jpadams/legend-daggerize-minimal-maven/blob/main/dagger/src/index.ts#L21-L31

[long kindle]

Awesome to hear, I'm giving it a try now, thanks for the tip!

[elder rain]

I actually really like this hybrid approach for local dev - I would love to see dagger be able to support the full capability of docker-compose for local development, but using to to handle the services complexity still feels quite nice.

Thanks for the inspiration @long kindle !

[long kindle]

Happy to be any help, love the service and really excited to get started with Dagger. The pace of development is inspiring to see

[round bridge]

I wonder what happens when we call up on an array of Service?

eg. from your example: dagger call dev-deps up

Assuming that doesn't bring all the services up today, I'm making an educated guess that we could tweak the implementation so that it does. It seems like the most natural behavior. wdyt @round harbor ?

[round harbor]

that might just work as-is - I think we already support sub-selecting fields of arrays of objects, though there are UI quirks here and there, and up is just another field, so...

[round bridge]

If we select serially, rather than concurrently, it might block on the first service. (https://github.com/dagger/dagger/issues/7813)

[rugged tinsel]

Semi-related, I find this module super useful often: https://daggerverse.dev/mod/github.com/kpenfound/dagger-modules/proxy@fd1a6e75721454d8ee3bcb0bd1a94d6bb4be1737#Proxy.proxy

There has been talk of making something like this core...currently uses nginx https://github.com/kpenfound/dagger-modules/blob/fd1a6e75721454d8ee3bcb0bd1a94d6bb4be1737/proxy/src/main.py#L15

[long kindle]

I tried the array of services approach, can confirm it just starts the first service

The proxy approach worked for me, although I had to make some changes to configure TCP streaming with nginx, since one of the services I was trying to start was a postgres database

[rugged tinsel]

Nice! Is there an improvement for that module that we should propose as a PR?

[long kindle]

Here is my code, I added a flag to say a service needed TCP proxying instead of HTTP, and converted the code to Typescript since that's what my existing Dagger code is. The TCP flag just ensures we write to stream.d/conf since you can't put stream protocol config in HTTP server blocks with nginx