#Hi I am new to Dagger and I am trying to
coarse gust
Hey! Welcome to Dagger 🙂
When you say locally build an image, are you trying to build an image and then be able to run it locally instead of exporting it to a registry?
Do you have any code to share of what you have done so far?
golden island
Thanks Lev! I am coming from more of a docker-compose based workflow. Basically I want to duplicate the functionality that we use docker-compose for in CI/CD workflows. We typically build images with docker-compose build start them with docker-compose up -d and run some unittests against the running service. I understand Dagger uses low level services like containerd and may not need to persist images before running containers. Is that right ?
coarse gust
I see, thanks for the additional info.
The way things work in dagger is slightly different. Using that same workflow that you describe in dagger you would do something like this:
- Write some code to set up your contianer, normally this might be things like mounting your source code, building your app. This step returns a Container type.
- Create any services you may need to to support your tests (normally things like databases)
- Grab the Container from step 1, pass in the services you need, and run your unit tests inside.
I think this example in the cookbook (click the python tab) does a great job showing how all of this works together: https://docs.dagger.io/cookbook#use-transient-database-service-for-application-tests
--
Lastly, you are spot on. There is no need to persist images before running containers, but if needed you can export the image for later use.
I hope this helps!
golden island
Thank you , that makes sense, how does one export the image? Does it save as a tar file or is there an option to persist them as images ?
coarse gust
Thank you , that makes sense, how does one export the image? Does it save as a t...
There is an SDK method called Export that saves as a tar file. https://dagger-io.readthedocs.io/en/sdk-python-v0.9.8/client.html#dagger.Container.export
You can then run it with docker load https://docs.docker.com/engine/reference/commandline/image_load/
golden island
Awesome! Thanks for the help!
coarse gust
My pleasure! Happy to help any time 😄
golden island
Also I am confused about how with_mounted_directory works. I thought it was the equivalent of a docker volume binding but when I use it , it tries to copy the directory from the host's filesystem into the container rather than creating a binding for it.
This is what I am doing:
if mount_dir:
mount_directory = client.host().directory(mount_dir)
pytest = pytest.with_mounted_directory(mount_dir, mount_directory)
This is what I see in the output:
`
┣─╮
│ ▽ host.directory /sww
│ ░ [28.98s] upload /sww from tinamou (client id: wv78wo65v4vq321635yj7e6p9)
│ ┣ [28.95s] transferring /sww:
â”» â”»
coarse gust
Hey!
Yes, this is one of the main differences between dagger and docker.
The difference between withDirectory and withMountedDirectory is that withDirectory will include a directory in an image if you export it, and withMountedDirectory will not.
This is not the same as running docker -v because we are not creating a bind mount. This is described a bit more here: https://docs.dagger.io/421437/work-with-host-filesystem/#important-notes
this is an area where the DX needs to improve a bit, so I'd love to hear your feedback on what you expect compared to what is happening.
golden island
Thanks Lev, maybe if I explain our use case it will make more sense. Basically we have a need to test our builds against our software repository (which we call software warehouse - sww for short) which is huge. And it doesn’t make sense to copy it into the test container. So in our docker-compose workflows we bind the sww path into the container and test that way. So that we can catch any issues before we deploy the project into the sww.
coarse gust
Yes, that makes perfect sense, I am not sure if we have a great solution but let me dig around a bit and I'll get back to you soon.
golden island
Thank you, very much appreciated. Also we don’t need to publish the test container anywhere if that helps.
golden island
Hi @coarse gust Revisiting this topic. I was wondering if there is a way to do what we need in Dagger.
If not, please consider adding this as a feature request.
fallow jungle
hey @golden island! nice to meet you, I'm Marcos, a core Dagger developer here.
Dagger works similarly to docker build than to docker run. You could see Dagger as a docker build on steroids which allows you to define your build pipeline by code and attach services to them instead of having a rigid and limited Dockerfile configuration experience. Because of this, bind mounting folders into the Dagger pipeline is not currently possible since it kind of violates one of the principles of a build system which are determinism and being a hermetic system. That's the same reason why docker build doesn't allow using bind mounts.
Having said that, because everything happens in your machine, uploading your repository artifacts to the build context should be an operation that generally doesn't take that much time (assuming your project is not several GB's size). If uploading your entire local context is indeed very heavyweight, there's different ways you can improve this by making use of the include and exclude directives so you can be more selective about what you're uploading to the context in relation to what you're trying to build.
If you have the time, it'd be great to maybe jump into a quick #dev-audio or zoom session with you to see how you're using Dagger and work together on setting up the best environemnt possible.
golden island
@fallow jungle thanks for the response. I am happy to talk about the details. Out software repository is huge, it really doesn't make sense nor is feasible to copy it inside the container. We know what the dependencies are for a given project but one of our test matrix requirements is to test the project as if that project code was deployed to production. If we don't have any bind mounts, we can't effectively test if the unittests will work inside the said production environment. Some of these dependencies are pip installed and some of them are actually deployed from Perforce. We would have to deploy the dependencies and their dependencies, etc. if we want to recreate that environment within the docker container.
fallow jungle
<@336241811179962368> thanks for the response. I am happy to talk about the deta...
hey @golden island ! thx for clarifying. When you say huge, do you have a rough approximation about how "big" that could be? tens of GB's? hundreds?
in any case, it'd be awesome to jump into a quick call so we can talk a bit more about it. When could that work for you? cc @coarse gust