In summary two possible approaches

[toxic warren]

@royal field

Creating a thread to limit the flooding

[royal field]

Yep yep

Ok so

If we only want to run the original script verbatim, but “cloakify” it

[toxic warren]

I guess it is important how common that pattern is

[royal field]

I think the easy way out for us would be to ship a tool with the sdk, named “cloak-script” or something

This tool itself just uses the SDK (it’s in pure TS)

It would just set up a correct environment and run its args

DX wise:

1. install cloak SDK
2. pre-pend script entry with “cloak-script”, e.g. cloak-script react-script build

[toxic warren]

Perhaps my entire premise is flawed and, just because a new "deploy" lives in package.json alongside an existing "build", doesn't mean they need to have the same relationship to cloak

• new "deploy": cloak-only. Nothing to wrap / port.
• old "build". Maybe you want to containerize it, maybe not. But it's not an industrial scale affair

So it boils down to: how frequently does a project need to containerize ALL its existing npm scripts wholesale with cloak?

[royal field]

That’s what I meant earlier by “novel” — I think it’s super interesting, but “build” is fairly different than “deploy”

[royal field]

I think the (optional) cloak-script could give us that

You containarize it IF YOU WANT by prepending that tool

[toxic warren]

If the answer is "almost never" then my whole concern here is overblown. You just copy-paste the thing and get it over with. (the problem of what to name the "yarn" extension if it doesn't actually use yarn, remains)

[toxic warren]

Yeah I thought it applied to deploy too. But I guess it doesn't.

As long as we're OK with not actually running yarn inside the container

[royal field]

I think also depends on performance. Is it going to be as fast when containerized on real world scenarios (on a Mac, not on Linux)

Need to think this a bit more. I’m wondering if it’s orthogonal: should focus on complex new scripts, existing ones could or could not be containerized, shouldn’t have that big of an impact

[royal field]

I think you’re saying the same

[toxic warren]

Yes. I should probably change the first script in the demo then

Maybe to build a docker image ? But then need to push it somewhere

[royal field]

I think from deploy’s point of view: it will invoke yarn build. Whether build itself is containerized or not shouldn’t matter right?

(Checking the orthogonality)

[toxic warren]

well… depends if we support nesting 🙂

I guess we do! cloak-in-cloak out of the box?

[royal field]

Yeah?

[toxic warren]

Maybe that should be a highlighted feature

[royal field]

Seems like we should

[toxic warren]

We never got that to work safely in docker

I’m starting to think A) cloak could be explained as a standalone thing (tightly coupled to dagger of course) and B) it’s a container engine 🙂

[royal field]

We need to “KYC” API callers anyway at some point (because each caller gets their own API view based on dependencies)

So nesting shouldn’t be hard

Not harder than the hard problem we need to solve anyway with dependencies

Everyone sees their own thing, API is aware of who’s calling (need this for tracing/telemetry anyway, in addition to dependencies)

Since we own the SDK: doesn’t necessarily require “cloak-in-cloak”. Parent passes down means of API communication, child SDK uses it rather than starting its own cloak. Parent cloak knows who’s calling (because of KYC we already need for extensions). So there’s just a single cloak but it can be passed down to nested instances

This is no different than extensions actually

Extensions are cloak-in-cloak, kind of, already

[toxic warren]

That can be implemented in the cloak CLI and then all sdks benefit since they fork-exec it

[royal field]

TLDR’ing: Yes, nested cloak can be done because we have the infra already for extensions which is very similar

[toxic warren]

Kind of an ssh-agent pattern

[royal field]

Yep

[royal field]

So going back to this: it doesn’t matter to deploy whether yarn build is cloak enabled or not

[toxic warren]

btw this same pattern could allow us to simulate a no-daemon model

[toxic warren]

right 🙂

beautiful

I still need a new idea to replace build as step 1 in the demo though

maybe just a bunch of different deploys?

[royal field]

Hmmhmmm … yeah maybe an upload to s3?

Don’t think it applies to todoapp but some complex linting (like markdown linter which is a container) could have done the trick also

But yeah different deploys. We can make that as complex as we want.

For instance we could flush cloudflare cache on deploy (but it’s buildkit so CDN cache is only flushed when we actually deploy a change)

[toxic warren]

something with Our docusaurus site ?

[royal field]

Maybe that’s too niche

(Cloudflare not docusaurus)

I think it’s more a limitation of todoapp itself: it’s a VERY straightforward app, doesn’t need much

[twin lance]

I haven't read the whole thread here in detail yet so apologies if this is irrelevent, but if the problem is how to replace the existing build with out wrapped build, all I did was:

1. Have package.json call workflows/build https://github.com/dagger/cloak/blob/07ed249a8630ad9865866ae55e56184bd1f0bbb3/examples/todoapp/app/package.json#L19
2. Update workflows/build to just call yarn run react-script build https://github.com/dagger/cloak/blob/07ed249a8630ad9865866ae55e56184bd1f0bbb3/examples/todoapp/app/workflows/build/index.mjs#L25

The second part required a small change to the yarn extension to accept more than one arg, but that's it. You don't need to do anything special to be able to call react-script, just yarn run react-script build works (even from the command line on the host still)

I guess I'm confused where any magic or wrappers come into play

Also for the platform issues, that's an issue today already with dagger (you can export built products to your local machine) and in containerized development in general, right? So the solution there would be part of our more general solution to supporting multi-platform builds. Not that that is easy of course, just that it's a problem we have to solve generally anyways no matter what, nothing specific to this situation

[twin lance]

Caught up on thread a bit more, I agree that it would be nice to be able to just run the existing package.json scripts in place, that's what we've previously been calling zero-code adoption right? Basically, there would be a yarn (and npm, pnpm, etc.) runtime that accepts a filesystem containing a package.json and just does what I am doing above automatically.

We definitely should have that, and it could even be a stretch goal for the demo if there's time, but in the meantime I don't think the solution in the PR is bad at all, it's not doing anything magical or complicated

[toxic warren]

Zero-code adoption was something else (at least in my mind), but this is better 🙂

@twin lance you can ignore the part about magic wrappers. It was exploring an idea where you don't have to modify the package.json scripts. But it turned out to be a dead end (technically possible but not needed)

Also I didn't know yarn supported running arbitrary commands inline (thought it could only run what was written in package.json). I guess that makes yarn a full blown shell?

Anyway that solves the minor issue of extension naming. If the extension still calls yarn, then it makes sense to call it yarn

I wonder how we'll deal with this problem with Makefiles, but let's cross that bridge when we get to it 🙂

[twin lance]

Yeah my impression is that the value of a yarn script is just executed in a shell where the PATH has been setup to point to the package's .bin dir and other stuff. If you set a yarn script to just execute env you can see all of it.

I wonder how we'll deal with this problem with Makefiles, but let's cross that bridge when we get to it 🙂
Totally, I have ideas but yeah too much else to do now