#Painful optional input parameters

1 messages · Page 1 of 1 (latest)

gaunt spruce
#

👋 I'm curious to know if the community has come with different approaches (or the core team can share its views) on the pain of functions with many flags.

We need to onboard hundreds of engineers into dagger and in order to make it as frictionless as possible, we've pre-built modules that account for building specific project types. Modules that are somewhat complex and will take a bunch of inputs.
Ideally, we're looking into something like:

# initialize the project with a config file with default values
$> dagger call -m gh.com/daggerverse/go-project init --output go-project.yaml
# user to modify go-project.yaml to accommodate the project needs
# call the main entrypoint to the module to test/build/publish/deploy
$> dagger call -m gh.com/daggerverse/go-project run --src . --config go-project.yaml

(We're eagerly waiting for this PR which will simplify even further the calls

Onboarding into the functionality that we've provided for the rest of the engineering team is then as simple as those 2 commands and adding a dagger call into your .github/workflows/workflow.yaml

Rationale is, if users need to fiddle with dagger calls to understand what they need to call and what not, what inputs to pass and which don't, we assume resistance in migrating into it.
We understand that with this approach, we're removing capabilities from end-users at first.

TLDR; We're abstracting the inputs into a yaml file, generated and validated through our module which is focused for developers and abstracts the core modules logic into a simple interface.
Are we crazy? 😅
Is there any functionality we've missed or is in the works for the short term that can alleviate the above?

#

Also, any other references/PR/posts/threads would be helpful too 🙂

gloomy apex
unborn pumice
#

TLDR; We're abstracting the inputs into a yaml file, generated and validated through our module which is focused for developers and abstracts the core modules logic into a simple interface.
Are we crazy? 😅
Is there any functionality we've missed or is in the works for the short term that can alleviate the above?

Hey Nadir, I don't think this is crazy, its an interesting approach. Just to make sure I got it right, do you have a module that is being used a shared library across other modules that knows how to parse your config? If so, I think that is pretty cool, I can imagine large orgs doing something like this even if there were some additional ergonomic improvments that Jeremy linked to above.

slow plover
#

Also: did this configuration file format exist already, or did you design it specifically for the needs of this Dagger integration?

gaunt spruce
# slow plover Are you able to share an example of `go-project.yaml`?

Sure. This is the initial version. We expect this to expand with a few more (10+) configs as we keep covering the existing use-cases of our current pipelines.

project:
    name: ""                # Project name. Would be used as fallback for 
    version: file:VERSION   # Version to use when interpolating tags
    artifact: docker        # Type of artifact this Pipeline needs to generate
    os: ""                  # Operating system for artifacts generated (binaries / docker images)
    arch: ""                # Architecture for artifacts generated (binaries / docker images)
golang:
    path: .                 # Path in the repo for the project (support for monorepo)
    test-args: []           # Arguments to `go test`
    check-args: []          # Arguments to `go staticcheck`
    build-args: []          # Arguments to `go build`
docker:
    publish: false          # Publish docker image to production registry
    image-name: ""          # Image name (defaults to project.name if unset)
    tags: []                # Static tags. project.version-$commitSha-$datetime would be appended
    entrypoint: ""          # Docker image entrypoint
debian:
    publish: false          # Publish debian package (name of package == project.name)
    release: ""             # Release for debian package
gaunt spruce
# slow plover Also: did this configuration file format exist already, or did you design it spe...

This has been designed explicitly for the dagger integration. We expect this config file to grow to a few more parameters. We want to keep it as short as possible though!
We are still experimenting on how to drive the adoption across the org and will get some developers to start testing this approach over the next couple of weeks to gather some feedback.

This config feeds a module that allows a developer to run a pre-configured pipeline.
They can call this module functions without ever specifying a single input (and once the context PR is merged, we'd just inject the src folder and look for the file in a default location unless specified otherwise)

$> dagger functions            
Name                   Description
build-debian-package   Build a Debian package containing the executable
build-docker-image     Build a Docker image containing the executable
build-executable       Build project and return the executable
config                 Retrieve project configuration
run                    Run all pipeline steps
validate               Run tests and checks on the project

This is our abstraction over common functionality (docker / debian / golang / git) that standardizes a pipeline at our org (for a specific project-type (Golang based)).

Users can consume this function with a config file, without ever having to run dagger install / dagger init.
They can simply modify their .github/workflows/workflow.yaml and run the dagger-action calling this module run

Power users can still run dagger init ci and consume the same core modules that the abstraction is consuming, and craft their own custom pipelines. We just have a lot of devs that may not feel that confortable/interested in CI 😉

gaunt spruce
# unborn pumice > TLDR; We're abstracting the inputs into a yaml file, generated and validated t...

We would like it to be an independent module to be reused. For now it's part of the same module that defines the pipeline. We're looking at how to abstract it.
The module defines the structure / defaults / comments / etc.

Our core modules won't know about this config. They are "normal" dagger modules with as many inputs as needed, as they are intended to be consumed programmatically (although very usable through the CLI too... 🙂 )