#Managing ignore parameter (pre-call filtering) complexity in Dagger module

1 messages · Page 1 of 1 (latest)

hushed raptor
#

Hi, I'm using Dagger in my IaC Homelab repo for CI checks. I've created a custom Golang module with functions to build, test, and lint across the various languages and frameworks I'm using.

The composability of Dagger functions is probably my favorite part of Dagger. Dagger transparently handles building the dependency graph, caching results, running independent operations in parallel, etc.

When it comes to managing ignore parameters however, the process is quite manual. If I have function B that depends on function A, and I add or update the ignore patterns for A, I have to remember to make the same changes to B. If I don't, I could have a situation where calling B results in different paths being passed to A than if I had called A directly. This potentially breaks B.

As my Dagger module gets more complex, my ignore patterns have gotten unweildy. I'm to the point where I'm considering running some tool to analyze the ignore patterns and dependency graph and generate the patterns for me. Is there an easier way to handle this?

limpid marlin
# hushed raptor Hi, I'm using Dagger in my IaC Homelab repo for CI checks. I've created a custom...

Hello! Sorry about this... you're not the only to have experienced this issue.

We've been working on a set of major improvements. Collectively we call them "modules v2" because the goal is to give more control to module developers, while removing complexity and boilerplate for module users.

Some of those improvements are still in development, but one of them is already available that I think you will like: workspace API.

Your module can now receive a "magic" argument of type Workspace, which allows reading the filesystem of the current workspace dynamically. In other words, you can do the equivalent of ignore filters, but as actual calls eg. `Workspace.directory("./my/src", include: [..], exclude: [...]).

#

@hushed raptor can you share a snippet of one of your functions with those pesky ignore filters ? You don't have to paste the whole thing. I'll try to reply with the workspace API version

#

I don't think we have docs for that API yet - cc @woven depot @novel flame is there a cool example somewhere? I have a dogfooding branch but it's not ready yet

novel flame
hushed raptor
#

I went through the Dagger docs and did find something obvious I was missing. A lot of my 'meta' functions just run other functions in parallel and collect the results. For example, I have a Lint function that simply calls LintGo, LintPython, LintYaml, etc.

I didn't realize I could be using dagger check with globbing to achieve the same effect. Instead of running dagger call lint, I could have been running dagger check 'lint*' this whole time. facepalm I think I wrote off using checks when I first started using Dagger because I thought my functions would need required flags. I never ended up adding any required flags.

Removing these meta functions is going to almost completely relieve me of ignore path management headaches. Switching to using Workspaces would be nice just for gitignore: true alone though. Thank you both for the response!

limpid marlin