#Are we serious about making the CLI scriptable?

1 messages · Page 1 of 1 (latest)

wanton mesa
#

FYI @tender dagger @spring turret @merry bronze @copper mortar this came up in a docs discussion today.

We are finishing a first wave of docs improvements, but the next wave is blocked on this topic.

#

cc @teal fractal @peak grove @round oak @unreal charm @uncut canyon who were in that discussion also

#

Are we serious about making the CLI scriptable?

sand venture
#

My use cases for scripting:

  • no --source in dagger call, will be solved by contextual modules
  • easier access to environment details (eg. environment variables, but accessing the GHA context would be interesting). could be solved by contextual modules.

I'd personally wait to see where those improvements lead firstl

wanton mesa
#

How do you feel about adding the capability of stitching multiple function calls together from the CLI?

#

(ie. raising the ceiling of what you can implement without switching to full-blown code)

sand venture
#

Can you give an example?

wanton mesa
#

stitching multiple calls together

#

(sorry at lunch 😁)

copper mortar
#

like maybe call a module that creates a helm chart and another module that uploads that to a github release

sand venture
#

I can't really imagine how that would look like in the CLI.

Currently, I have separate calls to build helm charts, so I can run them individually.

And I have separate release and snapshot functions that wire a bunch of artifacts together (like 4-5 artifacts).

Similarly, I have 3-4 linters.

How would this look like in the CLI? Writing a function that holds everything together seems logical to me.

autumn drift
#

Maybe because it's late night here and my brain is mush. I don't think I understand why the CLI is not scriptable today. I am assuming by scriptable you mean putting it in a makefile for example. I think I agree with Mark on waiting till some more puzzle pieces are in place like the contextual modules and see if there's a need.

I personally think making dagger modules work like a regular SDK to plug into my existing code is also another piece that will determine the scriptable future.

tender dagger
#

I think instead of "User Manual" and "Developer Manual" (personas) we should have a "CLI Manual" instead, but more like Manuals > Use the Docker command line. Notice that docker ... --help has as little output as possible. You need to read the web docs to understand it. Notice also that their CLI Reference has so much more information than what the CLI provides, including more details and examples for each flag.

#

It would be great to have a stable URL we can link to at the end of dagger call --help, like:

❯ gh pr --help
...

LEARN MORE
  Use `gh <command> <subcommand> --help` for more information about a command.
  Read the manual at https://cli.github.com/manual
#

So I don't think it should be a priority to target feature parity between the CLI and the SDKs so it can be used as another tab in the multi-lingual documentation. I do see value in adding a few more capabilities to the CLI like id stitching, thinking about those in DevOps that know a bit of bash but not much else, but I don't think that's a priority for now.

I think the highest priority is:

But as part of that, these are seen as possible solutions, or parts of the solution, so they do end up being a priority as well (imo):

This already has a PR:

But I don’t know about CLI: don’t hide id from core types. It’s easy enough not to hide id from available functions in dagger call, but I don’t know how much effort it would take to properly support stitching. I'm talking about being me doing it soon. Might be easy for someone else, but everyone's so busy.

wanton mesa
#

Completely agree on the CLI reference doc

copper mortar
#

@wanton mesa I found a great real world use case for the scriptable CLI at the Raleigh meetup with @daring reef

tender dagger
copper mortar
#

High level summary is to use outputs from one pipeline (as IDs probably) as an argument to another. Specific use case is for https://runme.dev/ and notebooks in vscode

RUNME

Execute your runbooks, docs, and READMEs.

daring reef
#

hey 👋, if somebody could help me figure out how to grab and pass the ref IDs for the respective resources here:

#

then I should be able to make this Runme markdown notebook ☝️ work

#

lemme know please

paper rain
#

Looking at one of the design documents, bash is mentioned as a method that dagger could be a shell like CLI, what about other platforms? - Windows, Powershell7 - Portable, Bash... Would need WSL

#

(I sound like a complete microsoft maniac, im not suggesting powershell7 because its cross plat) haha. Just curious to non bash script users

daring reef
#

How many people do you know that run PS(7) on machines that aren't Windows? If they even running it on Windows 🙂

#

I have yet to meet somebody 🪟

paper rain
#

me 😢

#

I did a challenge once "linux with no bash"

#

i lasted about 30minutes

daring reef
#

creatures of habit

paper rain
#

Is Windows Containers a serious subject though

#

because im guessing any companies using Windows Containers (windows-server-core or something?), maybe they are using Powershell7 / Poweshell Core with the Windows Core Docker images and then in this case, they probably would use the baked in powershell 7 - but yeah, probably not an investment to consider for dagger

unreal charm
#

I don't think we're opposed to supporting it but they key problem is that BuildKit does not support windows containers yet.

It looks like there is some progress being made on that front though https://techcommunity.microsoft.com/t5/containers/experimental-windows-containers-support-for-buildkit-released-in/ba-p/4096116

TECHCOMMUNITY.MICROSOFT.COM
Experimental Windows Containers Support for BuildKit Released in v0...

We are excited to announce that the latest BuildKit release v0.13.0 contains experimental Windows Containers support. BuildKit is a toolkit for converting..

daring reef
#

Did a bit of "prototyping" to connected the dots after my convo with @copper mortar at the Raleigh meetup

#

if the CLI could output and consume IDs, I could do this with my existing functions to string them together into a pipline using a Runme notebook

#

I know type-safety is a big deal with Dagger functions: it'd be awesome if anytime I made a change in the cell editor, I could trigger a "dry run" and provide immediate feedback. Or, even use a Dagger LSP to provide feedback and remediation.

#

anyways, feedback and input are welcome... i'll continue to hack on this

wanton mesa
#

@daring reef yes that's exactly what we want to enable... I think there's a PR for it, but it's not yet merged

#

we're getting close

#

(and by "that" I mean allowing passing IDs from one shell command to the next inside bash)

daring reef
#

i was hoping you'd mean that

wanton mesa
#

I actually would like to take it one step further, and embed a bash-compatible interpreter in the dagger CLI, effectively making dagger itself a shell. This would allow us to make the shell scripting experience even better

#

but first - regular passing of IDs 🙂

daring reef
#

that's a cool idea - in any case, swing that door wide open please with those IDs

daring reef
wanton mesa
daring reef
#

this one ☝️

#

sorry blast from the past

wanton mesa
daring reef
#

k, i'll keep digging

wanton mesa
#

@tender dagger could easily answer all these, but he's on vacation

#

If you search for all PRs by him, you'll quickly get to the answer I believe

daring reef
#

👌 that'll help narrow it down. thank you.

verbal bane
unreal charm
wanton mesa
#

I'm not sure that's true unfortunately. The core devs seemed pretty pessimistic about Windows Container support in general. I would definitely check with them first. cc @silent @covert hearth @spring turret

covert hearth
#

Yeah windows containers absolutely will not just work - the support is still super experimental in buildkit, and we have a lot of custom stuff on top, in particular networking, an init system, linux namespace entry, etc, that don't have exact parallels on windows

#

I'm not ruling out that we'll ever do it, but given the difficulty in making progress in upstream buildkit (which has interest and participation from some more experienced windows folks), I don't think we have capacity right now to work on it 😭

verbal bane
unreal charm
#

sorry for the speculation 😇

wanton mesa
verbal bane
#

I assume that dagger has a direct lib style integration with buildkit for the docker commands. If it was a pure shell integration, then we wouldn't have this issue?

wanton mesa
#

It's not just about integration, but modification. Dagger doesn't just connect to buildkit, it wraps it and hooks it. And occasionally forks it (temporarily)