#dagger up

1 messages · Page 1 of 1 (latest)

median hound
#

🧵

#

Location: core/up.go:64

Comment:

This looks like it can hang on partial startup failure. RunUp blocks on ctx.Done() once a service has started
successfully, but parallel.New().Run(ctx) waits for all jobs and does not cancel siblings when one returns an
error. So if service A starts and service B later fails during startup, B returns an error, A keeps waiting
forever, and dagger up never surfaces the startup failure unless the user hits Ctrl+C. I think this needs to
separate "start everything" from the long-lived wait, or use a cancel-on-error context/pool here.

somber stone
median hound
#

Nice

#

My other question was more of a general UX question: did you get a chance to dogfood, and it did it feel right?

#

I'm not worried about the UI itself: I can already picture it, it helps that we already have dagger check and dagger generate for consistency.

I'm more thinking about our service API, and the networking & composition model behind it.

#

For example if I have 2 services with +up, A and B... and A actually calls B as a dependency...

When call dagger up, will B be run once or twice? Will there be dedup or not... And if the answer is "it depends", what are the rules?

#

None of that comes from dagger up itself... It was already there in the model, it's just that dagger up surfaces it now.

#

Also I'm wondering how we will allow composing services across modules, without writing a new module 🙂 Also not really in scope for dagger up, but I can't help but think about it for modules v2. .. Since the goal is maximum module composition without writing custon code

#

Also for modules v2 we'll have to transpose your portMapping configuration into the new workspace config file 🙂

#

(thanks for the thorough review and manual testing @quaint smelt )

#

Maybe in the future we could allow +up functions to receive other services as arguments, and present a cool configuration model for easily passing one up service as argument to another from the config file

#

like "plugging" services into each other

#

actually, this could be done via a constructor argument in the module - doesn't have to be an argument to the +up function itself

somber stone
#

I'll read again this discussion to be sure.
I did multiple tries, with all +up functions in one service, with functions across multiple, etc.
Regarding dependencies, like service bindings, for now what I did is a +up function binding an other service from a dependency. So yes, in that case it's in the code.

#

Maybe this function configuration (to receive services) is close to the discussions regarding ship? As a way to compose, declare dependencies, etc

median hound
median hound
somber stone
#

If we have a +up function that declares a service as an arg, a dagger up will not start it by default (because the arg is missing)
In the config.toml if we can configure it, saying "this up function's arg is the up function from this module" then it means we can build a graph, start the services in the right order and pass them
That should be doable. Question is how to make it nice enough in the config side.

median hound
#

I mean 2 services in the same module:

type MyApp {

   pub frontend: Service! @up {
      container().from("blabla").withServiceBinding("backend", backend).asService()
   }
   pub backend: Service! @up {}
}

$ dagger up
--> runs backend
--> runs frontend
    ...which runs backend (a 2nd time? or not? dedup or no dedup?)
somber stone
#

ho ok, I was thinking about composing between across multiple modules
Yes, in that case the service will run two times, and might fail because of port conflict.
Because when we start to run the functions, we don't know about the binding

# 
type MyApp {

   pub frontend(srv: Service! @default("backend")): Service! @up {
      container().from("blabla").withServiceBinding("backend", srv).asService()
   }
   pub backend: Service! @up {}
}

@default might not be the right annotation, just writing something
With something like that, this become part of the signature, so we can coordinate the execution

somber stone
# somber stone ```graphql type MyApp { pub frontend(srv: Service! @default("backend")): Ser...

I'm experimenting on that, I think that could be an interesting solution. At least for dependencies inside a module. I'm not sure about using strings to point to a different function, but let's try.
And maybe that could then be extended for dependencies across modules:

type MyApp {
   pub frontend(srv: Service!): Service! @up {
      container().from("blabla").withServiceBinding("backend", srv).asService()
   }
}

type MyDep {
   pub backend(): Service! @up {
      ...
   }
}

[my-app.frontend]
srv=my-dep.backend

(not sure at all about the naming, but just to share the idea)
A default dagger up (without the config) will not start frontend as the argument is not defined/injected, only backend will be. But with the toml glue it will work

median hound
#

OK I'm going to bed! Thanks!

somber stone
#

An interesting approach suggested by Claude:

type MyApp {

   pub frontend(backend: Service!): Service! @up {
      container().from("blabla").withServiceBinding("backend", backend).asService()
   }
   pub backend: Service! @up {}
}

No annotation, just infering the argument name. This might be very fragile, and maybe too much magic 🪄 but this doesn't feel completely bad idea.

median hound
somber stone
#

Ok, forgot everything, this is already working 😅
There's already deduplication of services on the engine side.
I tried with this example:

type MyModule struct{}

// +up
func (m *MyModule) Gitea() *dagger.Service {
    return dag.Container().
        From("gitea/gitea").
        WithEnvVariable("DB_TYPE", "postgres").
        WithEnvVariable("DB_HOST", "db:5432").
        WithEnvVariable("DB_NAME", "gitea").
        WithEnvVariable("DB_USER", "gitea").
        WithEnvVariable("DB_PASSWD", "gitea").
        WithExposedPort(3000).
        WithServiceBinding("db", m.Db()).
        AsService()
}

// +up
func (m *MyModule) Db() *dagger.Service {
    return dag.Container().
        From("postgres:alpine").
        WithEnvVariable("POSTGRES_USER", "gitea").
        WithEnvVariable("POSTGRES_PASSWORD", "gitea").
        WithEnvVariable("POSTGRES_DB", "gitea").
        WithExposedPort(5432).
        AsService()
}

And it looks like to just work, no port conflict, no duplication of the db service.
I'll double check because I want to be sure this is really the case, as it looks too good 🙂
But it feels just right, so nothing to add, no crazy configuration, nothing.

I tried by running the gitea function only and it started the service as part of the call, and I tried with dagger up and it also just works. Just the two calls are returning the same service instance so it's good.

median hound
#

I know there is dedup on the engine side, my question was about the exact behavior of engine-side dedup: does it always happen, or will it sometimes not happen, etc. I never know with 100% certainty

#

anyway not a merge blocker in any case!