#Random ?: Is there a general

1 messages · Page 1 of 1 (latest)

marsh hill
#

build and publish would be two different steps - if you're using dagger then i think it makes so much sense to publish via dagger of course.

Depending on the complexity of your build you may get very far using Dagger due to its super abilities to elegantly express getting "just in time" artifcats via code.

If you have something simple there is nothing wrong with using existing Dockerfiles.

I think a good rule of thumb is if you are finding yourself "programming" a Dockerfile to do fancy things it might be a great time to try making a dagger function instead.

gilded cosmos
#

Sweet!

I think a good rule of thumb is if you are finding yourself "programming" a Dockerfile to do fancy things it might be a great time to try making a dagger function instead.

That's exactly where I was leaning. I just wanted to check that it's the right way as the docs/examples made me question myself.

#

I've started a module (go) - I found the Publish() function on the Container type, but there isn't a Build() like I'd expect. For the full pipeline Publish will totally do. But for a test pipeline where I don't want to push an image, just check that it builds, what's your recommendation there?

marsh hill
#

i'd love to learn more!

we're still trying to find the right balance with our docs and examples and its always really nice to hear a perspective from a fresh set of eyes.

gilded cosmos
#

Hmm.. I think it's probably just a combination of the early stages of modules & new docs and my ignorance in just getting started.

marsh hill
#

Great question - maybe this is not obvious but running a dagger function that returns somethign like dag.container() is essentially running docker build.

Under the hood we are using buildkit (the same thing that docker build uses), if you look closely you will see that all common dagger functions look very similar to what you would see in a Dockerfile

for example:

  @func()
  run(dir: Directory): Container {
    return dag
      .container()
      .from("node:lts-alpine")
      .withMountedDirectory("/src", dir)
      .withWorkdir("/src/docs")
      .withExposedPort(3000)
      .withExec(["npm", "install"])
      .withExec(["npm", "start", "--", "--host", "0.0.0.0"])
  }

So if your code looks like that you are always building a container along the way with each step, what you do with that container though is entirely up to you.

You can publish it, export it, run it as a service, or just grab a single directory or file, or just return the stdout from some final command.

So the question to ask yourself when you are writing a new module is "what do I want the final artifact to be" the answer to that question will help you find the right level of abstraction for your function.

One more thing, I always recommend keeping things as generic as possible. So if you can get away with it, return a Container instead of Directory or stdout. Because you can always get Directory or stdout out of a container through another function or via the CLI. But you cannot get a Container out of stdout.

If all that sounds greek, lets hop on a quick screenshare and ill show you a demo 😄

gilded cosmos
#

Awesome, that's super helpful! I should have gathered as much that it's still "building" the container. I was questioning it as it doesn't seem to commit it to the images (docker image ls). Thanks for the thorough explanation!

#

Regarding building the module, the docs state that modules will be published to the daggerverse either manually, or automatically if they're versioned and called. Is there a way to tell dagger to not publish the module in the latter case?

marsh hill
#

Can you tell me more about why you wouldn't want to publish it (again, curious for product discovery purposes)

If a module is public on GH and called by URL i.e. dagger call -m github.com/scotty/module $function it will be published and there is no way to prevent that right now (because remember daggerverse is basically an index of public github modules)

If its not a public repo, or not called by URL ie. invoked directly from the module directory like dagger call $function then it wont be published.

gilded cosmos
#

Got it! Two main reasons I'm leaning towards not wanting it to be

  1. More personal, and probably silly, is that one use-case I'm covering here is building a custom Fedora Silverblue image, which will be heavily opinionated towards my liking. But I suppose if someone discovers my module on the Daggerverse that's probably a good thing and they'd get an example for themselves worst case.
  2. From a business use-case I was thinking about the private repository use-cases, where we've got internal repos that we're using Dagger modules on and don't want them to be public.

This will cover my use cases for now I believe

If its not a public repo, or not called by URL ie. invoked directly from the module directory like dagger call $function then it wont be published.

marsh hill
#

Thanks! makes perfect sense. Just so you know in the future it will be possible to publish private modules if you want to.

For #1 - our current thinking is that someone could also find that code via GH Search so there really is not a huge difference between being listed in Daggerverse.dev and being listed on GitHub.com.

#

But, we should probably update the docs to make this more clear! Thanks again for the feedback.

gilded cosmos
#

Totally, I get it! Honestly, just having an option in the dagger.json (publish: false) for folks to opt out would be great. I could see that making it more friendly to folks feeling uneasy about the idea of "publishing" their stuff.

#

Will the Private repos published be more of a paid/team thing where there's RBAC for those folks in that GitHub org or something? Or is it really that the module is available in the Daggerverse, but you can't see the code?

#

I could see a use-case for the former in teams I've been a part of.

gilded cosmos
# marsh hill Great question - maybe this is not obvious but running a dagger function that re...

Digging into converting the Dockerfile, I'm finding that there doesn't seem to be a way to copy files from the project (i.e. templates or assets) to the container, at least that's what I gather from this thread: https://discord.com/channels/707636530424053791/1226590869033844768

Can you confirm that's the case?

For example:

Project structure:

assets/foo.txt
dagger/
Dockerfile
...

in Dockerfile:

COPY assets/foo.txt /etc/foo/foo.txt
brave fossil
#

is not a huge difference between being listed in Daggerverse.dev and being listed on GitHub.com
Just my 2 cents. I was in a similar discussion a few days ago and I'd like to voice my opinion once more. I believe Daggerverse should be a set of curated/ offered modules by the community and the Dagger team and not scraped via web based Dagger calls. Many people are just experimenting and thus the Daggerverse is already getting messy because of it. Even worse, the devs probably don't realize their maybe experimental module is being published for other devs to use, when in fact, they probably don't want this. In general, this take on scraping modules reduces the value of Daggerverse. Either there needs to be better ways to filter the "mess" or there needs to be prior gates to allow only "clean" modules to come through. If I remember correctly, there were thoughts to have an opt-in to purposely allow web based dagger calls to be published. I think that is a good start for a gate.

woeful atlas
#

@gilded cosmos regarding copying files to the container, do these links help?

https://docs.dagger.io/manuals/developer/overview/942201/execution-environment

https://docs.dagger.io/manuals/user/620164/host-fs

Execution Environment | Dagger

Containerization

Host Filesystem Access | Dagger

Dagger Functions do not have access to the filesystem of the host you invoke the Dagger Function from (i.e. the host you execute a CLI command like dagger call from). Instead, host files and directories need to be explicitly passed when executing dagger call.

gilded cosmos
woeful atlas