binary module

[sharp seal]

Nothing planned atm, but on a technical level it would be possible to implement an SDK that supports this. The SDK needs to tell the dagger engine the schema and be able to execute the binary based on api calls coming from the engine, but otherwise there's no technical requirement that the source code itself be there.

Curious what the use case is, just so modules can use proprietary code? Makes sense if so

[quartz sigil]

No, it's more about cold starts. So I'm not required to compile from source each time.

The secondary things is that the distribution changes. Right now everybody knows how to distribute OCI around the world (airgap, proxied, private repo, etc). But from source is different and honesly slower because of the required git clone. Like go put a lot of effort into the module cache.

[sharp seal]

binary module

[sharp seal]

Oh I see, so the compilation of modules is cached and should only re-run if any of the source has actually changed (if you're seeing differently let us know, we've had problem previously where they accidentally got invalidated too often for various subtle reasons).

By default, this only is going to be using your local cache, so yeah there will be a pretty significant cold start from an initially empty local engine. But that's where our cloud cache service comes into play 🙂 We've discussed the possibility of more closely integrating that service w/ modules so that some set of daggerverse modules are built+cached and made accessible by default to any users of the service, but that work isn't scoped into our current efforts. Remains on the table though and should all fit together nicely.

With that you wouldn't need any git clone even, buildkit just sees that you want the module and we'll already have the built binary (at least, in compiled langs like go) and can just pull that directly, skipping the pull of the earlier "git clone" vertex cache.