Dagger design limitations or any constraints for running on production

[meager dock]

Thank you for this amazing toolkit, this project in actuality creates new dimension into CI/CD and software development. I really like all the concepts and different language SDKs.

So my question is, I was trying to integrate dagger into our current system as a controller for our testing pipeline, where we need to evaluate interactions between dozens of services. First I wanted to do this kind of testing pipeline with docker-compose, because it's simpler, and docker-compose if much more mature and have been stable for some time. Basically I wanted to have a blueprint of all our services in docker compose format and write a controller that will manipulate docker compose. However I saw a dagger and immediately understood that it could potentially do the both job, blueprint of the services and also the interaction between them. But I believe this project is still new and I wanted to ask if there is any limitation or constraint I could face if wanted it to work in production. Thank you

Another thing I am curious about is that, I will need to allocate different gpu resources for different services, I will need to mount different volumes for multiple services to depend on. So this is a task what docker compose can actually accomplish easily, but I saw that gpu access is experimental, so I thought maybe it is too early to start using it

Or in other words I need to know what it can't do what docker compose can do

@quiet rose Thanks for creating this amazing project. After using docker and docker compose for several years I really understand that dagger is a missing piece in this puzzle of CI/CD cycle which we all were missing much. I believe that this project has a bright future, but how mature it is compared to docker compose for intra service communication.

[quiet rose]

Thanks for the kind words! Yes, your use case seems to be a solid fit for Dagger.

You're right that Dagger is relatively new, and that means it will have rough edges and bugs. I can't tell for sure whether they will specifically affect your project. But I can tell you that many people use Dagger for very similar use cases, and some of them definitely do it in production. We published many user stories here, perusing it can give you a sense of how Dagger is generally used: https://dagger.io/resources

I can also tell you that Dagger is more stable and mature than the average project of the same age. That is because Dagger's core runtime is buildkit, a very mature project which powers all docker builds everywhere. Dagger acts as a translator from a high-level GraphQL API to low-level buildkit operations. The rough edges are in the translation layer. Once the low-level operations are running, that's buildkit territory and therefore much more stable.

Because of this, my recommendation is to try Dagger first, and ask for help if you get stuck. If we can't help you, then docker-compose + wrapper tool is a good plan B. Perhaps a hybrid where some parts are daggerized and other parte aren't. Dagger works great with other tools, it's not all or nothing.