#daggerizing an npm application

1 messages · Page 1 of 1 (latest)

copper canyon
gaunt fox
#

i think what's tripping me up is that this still has you pass in --source=. from the cli

#

whereas I just want to define that directly in the dagger module

#

since it's static

copper canyon
#

Got it, it sounds like you might want 'context directories' described here https://docs.dagger.io/api/arguments#directories-and-files

Arguments | Dagger

Dagger Functions, just like regular functions, can accept arguments. In addition to basic types (string, boolean, integer, arrays...), Dagger also defines powerful core types which Dagger Functions can use for their arguments, such as Directory, Container, Service, Secret, and many more.

gaunt fox
#

i dont see how that helps?

@func()
  async readDir(
    @argument({ defaultPath: "/" }) source: Directory,
  ): Promise<string[]> {
    return await source.entries()
  }

still takes a Directory as an argument

copper canyon
#

Yeah so this would make source an optional argument, defaulting to the root of the repo. So in the quickstart command it would just be dagger call test

Unless I'm misunderstanding the question 🙂

gaunt fox
#

hmm maybe vscode is lying to me about the error

#

it's saying "loadSource expected 1 argument but got 0"

copper canyon
#

Ah yeah right now if you're calling the function from another function within the same module, the argument would still be required because the defaulting happens at the dagger layer between modules or in the CLI

A common pattern is to have the project source as a field on the module's class rather than an argument to each function since every function needs the source anyway. In that case you can still default the source to your repo in the class' constructor

gaunt fox
#

ah, that makes sense

#

and if i wanted to package this up in the future and turn it into a shared module, then source would just be defined at the module level and it'd be transparent to the caller?

copper canyon
#

Yeah constructor args can always be passed/overridden from callers, so it still works in that case!

gaunt fox
#

i still feel like i'm missing something .. i'm basically trying to package up some shared scripts and make them reusable across a couple of projects. Is a dagger module what I want? Do I actually really want to be producing a container which has the scripts I'm trying to share, and then consumers reference that container?

gaunt fox
copper canyon
#

Yeah that's a good example. Providing a container is usually ideal for consumers of the modules because any dependencies for the script would then be bundled with it, so the scripts are 100% portable. The overlay option from the module you linked is a cool solution too

gaunt fox
#

It seems like a major gap that this is somewhat complicated though. I feel like I shouldn’t have to figure out a clever way to bundle a script and make it reusable- having to publish/share an image vs share a dagger module feels like it’s a busted abstraction

copper canyon
#

Totally! What do you picture as the ideal solution? Ignoring current limitations

gaunt fox
#

I’m not 100% of the API but at a high level, “here’s a relative path within the dagger directory- copy them into an image at this path”
container.WithDirectory would make the most sense to me (and I don’t see a good reason for a new API surface for this). I guess the new API would be to “make this local path a Directory”, something like ,Directory.fromLocal(relativePath: str): Directory

copper canyon
#

so if I have a repo or dagger module with a directory scripts/ that has a bunch of bash scripts I want included, something like this works today

@func()
  async test(
    @argument({ defaultPath: "/scripts" }) scripts: Directory,
  ): Promise<string> {
    return await dag
      .container()
      .from("alpine:latest")
      .withMountedDirectory("/foo", scripts)
      .withExec(["sh", "-c", "/foo/myscript.sh"])
      .stdout();
  }

and I can just run dagger call test without passing --scripts and it'll get the directory at scripts/ in my repo. Is that what you mean or specifically grabbing this local path in code at an arbitrary time? I can show another example of that if its what you're going for

gaunt fox
#

@copper canyon but i guess what I'm trying to do is provide a module/have it be called as API

#

like i basically want to distribute a custom CLI took that is used by various teams and projects

gaunt fox
#

ah maybe i'm missing a key part of

Ah yeah right now if you're calling the function from another function within the same module, the argument would still be required because the defaulting happens at the dagger layer between modules or in the CLI
..
you said
from another function within the same module
does that mean if i defined a completely separate module, using code like you showed above, then the caller would not need to pass in scripts? I really don't even want it to be possible to pass in alternate scripts though

copper canyon
#

I really don't even want it to be possible to pass in alternate scripts though

This is doable too. What kind of things might someone pass to your function? I'll write up an example

gaunt fox
#

We use contentful as a CMS, so we have some scripts that pull data from contentful, and then reformat it before writing it out to disk, and then our npm build generates a static site. We pass in some API call data (environment id, token information)

copper canyon
#

for this example I run this script:

#!/bin/sh
mkdir /out
echo "HELLO WORLD" > /out/hello.txt

in this function:

@func()
  test(token: Secret): Directory {
    return dag
      .container()
      .from("alpine:latest")
      .withSecretVariable("TOKEN", token)
      .withMountedDirectory(
        "/foo",
        dag.currentModule().source().directory("scripts"),
      )
      .withExec(["sh", "-c", "/foo/myscript.sh"])
      .directory("/out");
  }

output:

dagger call test --token env:TOKEN file --path hello.txt contents
âś” connect 0.4s
âś” loading module 0.8s
âś” parsing command line arguments 0.0s
âś” setSecret(name: "2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae"): Secret! 0.0s
âś” example: Example! 0.0s
âś” Example.test(
    token: âś” setSecret(name: "2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae"): Secret! 0.0s
  ): Directory! 3.2s
âś” Directory.file(path: "hello.txt"): File! 0.0s
âś” File.contents: String! 0.0s

Full trace at https://dagger.cloud/kpenfound/traces/db655024d35db4fb01ca2485e69cd939

HELLO WORLD
gaunt fox
#

and then if I wanted to use that from another module it would be something like.

@func
build(source: Directory): Container {
  return this.buildEnv()
    .withDirectory("/content-dir", Contentful.test(secret))
}
copper canyon
#

yes exactly!

gaunt fox
#

damn, thank you!

copper canyon
#

no problem, hopefully that feels like a better solution than what we discussed earlier 🙂

gaunt fox
#

yeah! this feels like what i was looking for initially

#

dag.currentModule().source().directory("scripts"),
seems to be the key thing I couldn't find right away in the docs

#

the dagster one is nice, it's got github in its context and will point you to GH issues/discussions as well as docs

copper canyon
#

Yeah that's good feedback! We've tried some solutions like that in the past with varying success. But navigating our split knowledge base across docs/github/discord is a serious challenge. I like that dagster example, thanks for linking!

gaunt fox
#

i swear i saw a project somewhere that was trying to solve that exact problem

#

because .. yeah, it's tough

#

i really appreciate the help here though 🙏

copper canyon
#

There's a few out there for sure. The real trick is that (IMO) having an LLM that gives incorrect information is worse than having nothing

raw glacier
#

can somebody help me how to make dagger.Directory from str?