Have we decided on how to invoke the | Dagger | Page 1

unkempt charm Sep 5, 2023, 7:06 PM

#

Baseline will be the same sort of idea from the old project commands implementation.

There's still plenty of details to figure out there that we skipped previously, e.g. how to accept lists, how to accept objects, but I think there's some good options available to choose from.

I also think we'll just need to add support for more sugar over time; e.g. if a Directory is an input arg, you should be able to point to a local path, a git dir via git://... (same as we support for modules+deps, just for args too), etc.

And along those lines, maybe a Container input could support registry references, local tarball paths, etc.

The overall idea though is that we can't ever make the CLI as powerful+flexible as writing code. So while we need to have it and polish it as much as possible, we need additional layers of interfaces too. That's where the idea @radiant kiln has been talking about in terms of language specific repls comes in I think.

But like we were talking about the other day, if your input parameters become really complicated to invoke, that's just a sign you should package up your call into its own Function that accepts simple parameters (or none at all).

radiant kiln Sep 5, 2023, 7:58 PM

#

👆 what he said 😁

#

Also: an important breakthrough for me is that the UX should emphasize, not hide, that this is an API

#

So, even if you're querying the API from the CLI (as described by Erik above), it should be clear that it's one way to query the API, as a convenience, and there are others.

#

Different ways you can invoke functions from a Dagger module, by querying its API:

Raw graphql queries from curl or other http clients
One-liner with dagger CLI
One-liner from your python/js/... interpreter, with codegen pre-loaded
YAML/DSL straight from your CI config (eg. "Dagger for Github" action)
Web UI, possibly embedded in an IDE plugin

And of course:

From your own Dagger functions, when developing your own module using a Dagger SDK

neat shale Sep 5, 2023, 8:18 PM

#

But which dagger <command>?

radiant kiln Sep 5, 2023, 8:57 PM

#

Didn't really revisit that...

unkempt charm Sep 5, 2023, 8:57 PM

#

neat shale But which `dagger <command>`?

Ah yeah, that's another thing to decide on 🙂

One option we discussed previously was re-using dagger run such that if it's executed in a module directory or explicitly provided one (dagger run -m ./my/module) then it executes a function rather than it's current behavior.

Another option now that we're trimming Command would be to use dagger do.

I think dagger script has been brought up before. Probably others too.

radiant kiln Sep 5, 2023, 8:57 PM

#

I guess possible answers include:

dagger do
dagger run
dagger function or dagger fn
dagger call
dagger invoke
dagger query
...

#

I would also like us to consider a separate command for the "one-liner in the language of your choice" (option 2 above). I like dagger script for that, but we should probably think of other options

neat shale Sep 5, 2023, 8:59 PM

#

I've been thinking exactly like dagger run -m ./my/module.

radiant kiln Sep 5, 2023, 8:59 PM

#

dagger script --lang python -c 'frontend().build()'

#

My theory is that, if we had a very smooth language-native one-liner experience, most devs would use that instead of the CLI mapping

#

In other words, I think we should ship dagger do/run AND dagger script, and I think the latter will be a hit (also less limitations in what arguments you can pass etc)

#

OR we do something radical and only support dagger script initially... and add a bash codegen for it 😄

#

dagger script --lang bash -c 'frontend build --debug true'

unkempt charm Sep 5, 2023, 9:07 PM

#

radiant kiln OR we do something radical and *only* support `dagger script` initially... and a...

I do think it would make sense in terms of implementation order to start w/ dagger script actually, for the simple reason that it's probably easier (based on the past experience of implementing project commands, dynamically adding cobra flags in the middle of a running cobra command is a great way to induce nosebleeds...)

But comparing:

dagger script --lang bash -c 'frontend build --debug true'
dagger do frontend build --debug true
side-by-side makes me think we'll probably want dagger do/run, even if underneath the hood it just ends up "compiling" to a dagger script call 🙂

neat shale Sep 5, 2023, 9:15 PM

#

For dagger script we need the runtime to install the right SDK version instead of being a dependency of the language's package, but we can't reliably match from the engine version. They're the same now but they haven't been until recently and may not be later on (SDK specific patch versions).

#

We also rely on the function's signature for declaring the types of the arguments and the return type. We can wrap the one-liner in a simple function underneath (like no args and returning string) but that seems limiting.

radiant kiln Sep 5, 2023, 9:21 PM

#

The SDK version of the client doesn't need to be the SDK version of the server though, right?

neat shale Sep 5, 2023, 9:21 PM

#

Each SDK release has a specific engine version it's compatible with.

radiant kiln Sep 5, 2023, 9:21 PM

#

So regardless of the SDK version the target module was developed against, dagger script would always get codegen from whatever SDK version the CLI is configured to use

radiant kiln Sep 5, 2023, 9:22 PM

#

neat shale Each SDK release has a specific engine version it's compatible with.

Ah right, that. That problem exists with or without dagger script though, right?

neat shale Sep 5, 2023, 9:22 PM

#

Yes.

#

When you make a dagger module you need to add the dependency to the dagger sdk version, in go.mod or pyproject.toml.

radiant kiln Sep 5, 2023, 9:23 PM

#

Right. The idea is for the CLI to do that for you as much as possible

neat shale Sep 5, 2023, 9:23 PM

#

Yes, I'm just trying to find out how best to do that.

radiant kiln Sep 5, 2023, 9:23 PM

#

I guess something like dagger mod init, dagger mod update, etc?

#

Then you commit the dagger.json file if it changed?

neat shale Sep 5, 2023, 9:24 PM

#

Yes, but then it's not a dagger script one liner. It requires more steps.

radiant kiln Sep 5, 2023, 9:24 PM

#

I think dagger script is completely orthogonal

#

You can replace dagger script with dagger run and it's the same problem

#

Either way the engine will load the module, and if that module is developed on a version of the SDK that doesn't support that engine version, bad things ™️ may happen

neat shale Sep 5, 2023, 9:26 PM

#

Not exactly, the issue is the one-liner, or in other words, not requiring any more steps other than dagger script in a clean dev environment. dagger run and current state of zenith have dagger mod init et all...

radiant kiln Sep 5, 2023, 9:26 PM

#

Separately from all of that, dagger script would basically initialize an ephemeral empty module, with a single dependency which is the target module, get the codegen for that, and plug it into an interactive language interpreter

#

In other words there is no compat problem between your one-liner and the engine, just like there is no compat problem between dagger do and the engine (because in both cases, it's the CLI controlling all of it)

#

But in both case, there is a compat problem between the target module and the engine, because the CLI doesn't control which version of the SDK that module targets

neat shale Sep 5, 2023, 9:35 PM

#

If you use dagger run python ... today, you are already in a virtual env with your SDK version already installed. On first connect it'll make a query to the API to check version compatibility. Same happens without dagger run (automatic provisioning).

If you use dagger do (zenith), you go through dagger mod init which creates a new dagger module with possibly go.mod pointing to the latest SDK version. init doesn't create this for Python yet. Maybe not even Go, I haven't tried. But you just do it manually in the meantime. So when the dagger module loads, that go.mod/pyproject.toml is installed as a part of building the runtime (i.e., container that runs the functions/entrypoints). We can do the same as in dagger run and make the api call to check compatibility and error out. The point is there's manual control here when you commit this to git.

To use a dagger script where we do all this bootstrapping underneath, we'd need a --version 0.X.Y next to --sdk python unless we can reliably find the best SDK version for the current CLI version. The naive approach would be to first find an exact match but there can be discrepancies as we can't guarantee they'll always be the same. Another naive approach would be to use the latest version, but then it may not match your CLI version.

neat shale Sep 5, 2023, 9:42 PM

#

radiant kiln In other words there is no compat problem between your one-liner and the engine,...

As of now there can be, because you still need go.mod/pyproject.toml to declare a dependency on your SDK library package and it can be different from your current CLI version.

radiant kiln Sep 5, 2023, 9:43 PM

#

neat shale As of now there can be, because you still need `go.mod`/`pyproject.toml` to decl...

But that file will not (or at least should not) affect your one-liner

neat shale Sep 5, 2023, 9:43 PM

#

You're missing the point 🙂

#

I can sync for a few minutes if that's easier.

radiant kiln Sep 5, 2023, 9:44 PM

#

o use a dagger script where we do all this bootstrapping underneath, we'd need a --version 0.X.Y next to --sdk python unless we can reliably find the best SDK version for the current CLI version.

Yes this is the plan. CLI should be source of truth for version of everything else.

#

(not sure if that 👆is the point you had in mind)

neat shale Sep 5, 2023, 9:46 PM

#

neat shale For `dagger script` we need the runtime to install the right SDK version instead...

Yes it is. I want that but now we go back to the beginning where I said we need a way to reliably find which SDK version corresponds to the current CLI version. 👆

#

We either go with a naive solution at first, think of something else that's easy or smart or it'll be more to tackle if we want this sooner rather than later.

radiant kiln Sep 5, 2023, 9:56 PM

#

I admit I haven't thought of the specific details of how to do this, but generally assumed the CLI would hardcode a default version for each SDK that it instruments with dagger script (so for each possible value of --lang). Then a flag to override the default and pass a specific SDK version. But maybe that's wrong?

neat shale Sep 5, 2023, 9:59 PM

#

The point you're missing is that each SDK has an hardcoded engine version it's compatible with (during version bump in CI), but not the inverse. The CLI has no direct knowledge on which SDK versions are compatible with it. Since v0.8.0 they've matched (==) but that's not a guarantee.

radiant kiln Sep 5, 2023, 10:02 PM

#

We could make it a rule that SDKs should support CLI/engine on the same major+minor version, and shouldn't be merged if they don't? Then CLI simply uses its own major+major version as a default SDK version?

neat shale Sep 5, 2023, 10:04 PM

#

I think SDKs should be able to have patch releases that don't bump the engine version. Sometimes it's needed and it has happened before.

radiant kiln Sep 5, 2023, 10:10 PM

#

Yes I agree with that, I think that will still be possible

#

I don't mean we should version all these things separately. SDKs should continue to have their own versions

neat shale Sep 5, 2023, 10:11 PM

#

We can make sure you don't bump SDK supported versions on an engine's patch release, only on minor. Then get the latest major.minor.* for the SDK. Possibly even trying the most obvious release, compare the version and try the next SDK patch if they don't match. That'll take a hit on performance but it's more accurate, and only on non first matches which aren't common.

#

Currently the SDK supported versions are bumped on any engine/CLI release.

#Have we decided on how to invoke the