Daggerverse noise

[abstract sandal]

What does a stdlib look like? Something like what I've made a start on here: https://daggerverse.dev/mod/github.com/sai-php/sai@d0380650a1a5ca0b0244ce34820a6b5cc99f6c1b (it's for PHP, but the idea is to daggerise the typical tools in a PHP build pipeline so they can just be slotted together)

[unborn barn]

I think many of us are eager to jump into conversations about stdlib.

I 100% agree with @fleet ingot that there has to be a set of accepted patterns before the actual work can be done.

Some of the questions that I've been trying to answer so far in the course of working on my own modules:

• How open/closed modules should be? (eg. ability to modify containers after a module is initiated)
• Should module APIs stay close to the underlying tooling? Should there be separate low and high level modules? (eg. a low-level Go module could provide a wrapper around Go itself, expose a low level API for the Go tooling, whereas a "high-level" Go module could include tools like gotestsum, golangci-lint. But those are much more opinionated and I'm not sure they should be part of a stdlib)

Figuring out module cross-dependencies and the ability to extend an underlying container in a module (eg. customize env vars) proved to be hard (or at least I struggle with those to this day).

In any case, I would be open to start a conversation about stdlib modules, even donate some of my own, but I understand why the team is cautious (I would be too) and I'm not sure it's time yet either.

[copper rose]

I agree on the fact that established/approved patterns are a pre-requisite for any stdlib modules.

I also want to chime in here because today, almost no Daggerverse module works within my corporate network. I am reinventing the wheel on most of it to cater to my company'isms. I'd love to be able to contribute some of those modules back to a stdlib library that works on any network.

So, my point is, the stdlib should be designed with the ability to support Enterprise use cases too.

[fleet ingot]

what is preventing those modules from working in your network? Configurable proxy support?

[copper rose]

CA certs, configurable proxy, default private container registry, private git etc.

Some of this is already in flight.

• configurable proxy - https://github.com/dagger/dagger/pull/8800
• PAT support - https://github.com/dagger/dagger/pull/8805
• CA Certs (regression?) - https://discord.com/channels/707636530424053791/1301297050624856136

Private container registry is a tricky one. Maybe I'm missing a trick setting up mirrors. I need a way to tell dagger to always proxy via mycompany.com when given a container URI lik alpine:3 should become mycompany.com/alpine:3. Or ghcr.io/dagger/latest:v0.x.x should be mycompany.com/dagger/latest:v0.x.x. Pretty much all Daggerverse modules today hard code the .from.

[fleet ingot]

But those are PRs to the engine itself. Do they also require changes to the modules to take advantage?

I assumed no, but maybe I was mistaken?

[copper rose]

I think it's both? For example for private registry support, I created an internal "docker" module which handles this. So my modules use dag.MyDocker().From() instead of dag.Container().From(). Maybe it's something that can be resolved on the engine side. If not, we'd have to modify every Module to have an optional arg (or config) specifying a private registry?

[unborn barn]

Shouldn't a container registry mirror help in this case? That kinda became the standard solution when Docker introduced the new rate limits, but it's been around for airgapped setups as well. Naturally, you need to change docker configuration (I don't think you need any engine changes). But you should be able to use the exact same images without any changes to modules.

[copper rose]

I couldn't get it to work when I tried a while ago but things may have changed. However, it's not a single registry mirror. We have prod, non-prod (local), edge (in ci) etc. So having the ability for the modules to configure that kind of makes sense. Also telling consumers to change their docker settings to be able to run dagger is not fun. Having a hard time getting some devs to even install it.