It feels little different I think,

[gloomy tapir]

I'd say that it's logically consistent in that whether you are making a dagger cli call (dagger run) or using an SDK (whether in a module or on the host directly), the APIs that involve "host interaction" are scoped to the caller's host.

e.g. export always results in the result being written to the caller's host filesystem:

1. In the CLI case, it's the CLI caller's filesystem
2. In the case of module code, it's the module container's filesystem
3. In the case of SDK code running outside a module, it's the filesystem of that caller

and for the import side, the paths provided are still scoped to the caller's filesystem. In CLI case, it's the CLI caller's filesystem, in module code it's the module container's filesystem, etc.

I do see the difference in that even with the new dagger run plans, the CLI will still retain some sugar specialized for input args of type Directory (whereas we were avoiding specialization entirely on the return side)... I guess I'm okay with that in lieu of better ideas, but if that's the discrepancy you're noticing too then yeah I agree

Not sure how I feel about it yet

[mint timber]

Mmmm maybe there’s a possible (slight) adjustment of the CLI plan where we differentiate between 1) side effects like running the container, forwarding ports, publishing to a registry… and 2) saving the return value to the filesystem

maybe 1) remains the same as planned, while 2) is handled by a simple -o flag?

[gloomy tapir]

If we did that you could just select id rather than require an -o.

But is what you're imagining something like:
dagger run my-func --input-dir "$(dagger run host directory --path ./some/input/dir id)" export --path ./some/output/dir

If so, I guess it would potentially make things more symmetrical, but not having the sugar to support --input-dir ./some/input/dir feels like a huge sacrifice

I'll checkpoint that's what you're actually suggesting before I continue since I may be misunderstanding 😄

(there's also overlap here with the idea of how we want to pass secrets to the cli as args)

(this also gives me deja vu back to when the first version of the SDKs required an id as input but then we changed the DX such that they accept the object itself instead)

[mint timber]

no no I mean supporting this:

dagger run -o ./some/output/dir my-func —input-dir ./some/input/dir

no change to input flags whatsoever

[gloomy tapir]

Oh okay, I mean I'm not fundamentally opposed to that or similar as a layer on top but I think we'd want to retain support for the full ... export --path ./some/output/dir too for the sake of consistency. But then at that point we'd just have two slightly different ways of doing the same thing

It would feel better if -o is applicable to any return type, so e.g. a string return results in the string being written to the filesystem

[mint timber]

yes 100%

in fact I think this should reliably work (assuming myfunc returns a directory): dagger run myfunc file —path foo.tar.gz contents | tar ztv

or if it returns a struct: dagger run myfunc | jq .keys

[gloomy tapir]

Yeah makes sense, the one extra thing is that -o would handle Directory as a special case still. Given we're retaining some degree of special handling of Directory/File on the input side, I guess that's not actually the end of the world.

• There's also potential for making it more generic via interface support, but that's a whole separate tangent and could be a later addition.

The alternative would be if we added an asTarball field to Directory that returns a File, which would enable this:
dagger run myfunc as-tarball contents | tar ztv

And thus allow -o to skip special handling of Directory. But like I said, given the tiny bit of sugar required to make that more seamless, I am not really opposed to -o having a small "dash" of sugar for that

(the as-tarball approach would also result in us missing out on optimizations of filesync to the host, so not actually a swap in replacement really)

I'm sold on this, I'll add an extra comment to the issue about this and some of the other recent discussion points on discord.

I also btw sent out a PR with the completely unpolished support for exposing core APIs to the CLI: https://github.com/dagger/dagger/pull/6293 The main thing I'd want feedback on from you @mint timber is what I mention in the description about exposing it via dagger call -x to start (more considerations written there)

But I'll also go ahead and start breaking down the rest of the tasks into issues since after that initial PR we'll be at the point the work can start fanning out