What's the current recommended way to

[formal burrow]

Looking into this.

[hazy obsidian]

I do more or less the same:
https://github.com/Excoriate/daggerverse/actions/runs/11163672809/job/31031180374
It works, but maybe there's a better approach.

[formal burrow]

@strange bobcat Started a thread here.

daggerverse ➤ ls -1 | xargs -I {} bash -c "pushd {} && dagger publish && popd"

that works, for example.

but this doesn't with dagger v0.13.5 or v0.12 or v0.11 or v0.10

~~daggerverse ➤ ls -1 | xargs -I {} bash -c "DAGGER_MODULE={} && dagger publish"~~

whoops, shouldn't have the && in there

Error: failed to get configured module: no dagger.json found in directory . or any parents up to git root

[strange bobcat]

ok, checking this now 🙌

k, as Jeremy is correctly highlighting above, seems like DAGGER_MODULE doesn't play very well with Dagger Publish. I'll send a PR to fix it. In the meantime @brisk patio , any chance you can change your step to use the working-directory option instead of using the DAGGER_MODULE var?

[brisk patio]

Yep, I can give that a try

[strange bobcat]

ok, this works for me

I accidentally copy/pasted Jeremy's snippet but it has a small error

DAGGER_MODULE is not correctly defined

[formal burrow]

whoops, you're right, I shouldn't have had the && in there

ls -1 | grep -v LICENSE | xargs -I {} bash -c "DAGGER_MODULE={} dagger publish"

[strange bobcat]

@brisk patio it seems to me that it's working correctly?

i.e: this is the last GHA job I see for your repo: https://github.com/sagikazarmark/daggerverse/actions/runs/11300127431 which is for commit: 14ff058bbf8e1ced23cfeca1a68b1635e0e414bb

if I look for your modules in Daggerverse, I see that version exist for all of them:
i.e:
https://daggerverse.dev/mod/github.com/sagikazarmark/daggerverse/trivy@14ff058bbf8e1ced23cfeca1a68b1635e0e414bb

the thing about Daggerverse, is that the "last" version of a module (that's if you don't specifiy a SHA) is the one tagged. That's why if you navigate https://daggerverse.dev/mod/github.com/sagikazarmark/daggerverse/trivy, you'll get redirected to the last commit that has a tagged version

cc @formal burrow we should change the UI to display the "tagged" and "not tagged" version of the module

[brisk patio]

Check my quarto module, that one is way behind of the latest tag

[formal burrow]

I’ll add to the list 👍

[brisk patio]

Chances are actions logs are lost at this point :/

I believe I even tried running dagger publish locally, but if the exact tag isn’t checked out, dagger publish doesn’t work

But I have to verify that

[strange bobcat]

ok, I see.. the commit is crawled (https://daggerverse.dev/mod/github.com/sagikazarmark/daggerverse/quarto@14ff058bbf8e1ced23cfeca1a68b1635e0e414bb), but it doesn't seem to be picking the tag

[brisk patio]

Yep

Here is the last action run for that release: https://github.com/sagikazarmark/daggerverse/actions/runs/10930874986/job/30344725080

I don't know if it matters, but there is a chance that multiple publish request may be triggered by CI at the same time. If there is some sort of locking that prevents crawling the same module in parallel, that could be a reason why it sometimes works, sometimes doesn't.

[strange bobcat]

hmm I think the issue is that maybe for some reason the module gets published before the tag is created somehow. I can quickly verify this by forcing a module re-crawl now and check if the tag gets created

[brisk patio]

That is possible: publish runs on main as well.

[strange bobcat]

yep, that seems to be the case @brisk patio

just re-crawled https://daggerverse.dev/mod/github.com/sagikazarmark/daggerverse/quarto@126b5fbbdad70dbf2a8689600baec2eb78c05ef4

and the tag appears now

so the module seems to be crawled before the tag exists in the repo somehow

@formal burrow shall we add --recrawl to dagger publish?

[brisk patio]

Well, I usually work through PRs, so the tagged commit may certainly be published when it hits main. I usually tag after merging which triggers another job, another publish.

I could just disable publishing on main

[strange bobcat]

makes sense. We can add a --recrawl flag to publish so you can force the update when needed

[brisk patio]

But dagger call would still "publish" it

[strange bobcat]

yes, correct

[brisk patio]

So i don't think that would help

[brisk patio]

Sounds good!

[formal burrow]

Feels more like we could do this automatically if we checked for tags when a publish was called again. We should be checking for remote tags anyway and not relying on a local tag that consumers may not be able to use if tags aren’t pushed.

[strange bobcat]

this is tricky.. we intentionally made the recrawl endpoint private since crawling is not a cheap operation. That pretty much requires cloning the repository and then running a bunch of introspection queries. We can't also check for tags in a lightweight way since we're relying in the git protocol for that as it's not realistic to implement tag checking against each VCS hosting provider... Thinking about options here 🤔

[formal burrow]

Yep. Would be worried about exposing and needing to rate limit, etc

[strange bobcat]

I think there's something else we can do here. Will continue checking this week

[brisk patio]

@strange bobcat @formal burrow do you have any updates on this? Thanks!

[strange bobcat]

hey Mark! not updates. Next on my to-do list 🙏 . Will very likely get fixed this week

[strange bobcat]

@brisk patio should be fixed now. Funny enough this should was supported in the past but it seems like we added a regression which prevented the tag to be assigned to the published module

[brisk patio]

Does that mean no changes are required on my side?

[strange bobcat]

Yep, correct