Bootleg release

[willow basin]

@civic plover so the tricky part is getting the companion container in a registry somewhere, and shipping a CLI with that ref hardcoded, right?

[civic plover]

yeah, I suppose we could always just do something ad-hoc there

like, if we don't want to use official registry.dagger.io

[willow basin]

good forcing function for disentangling the release spaggheti, in order to put it back together as a perfectly organized zenith module

So we should make it a goal to make this bootleg release with no dependency on our real release infra - if we use any of it, it means we failed to disentangle it

[willow basin]

@turbid python I know you're 3 hours ahead... I just finished my meetings for the day. How much time do you have to try this?

cc @ionic viper @naive knoll 🙂

[turbid python]

I can come back to this in 2 hours-ish if that works!

[naive knoll]

@willow basin seems like we just publish the image to ghcr with creds, right? Then we could use GH API or better, Ali's gh cli module to push up the CLI binary, no? https://cli.github.com/manual/gh_release_upload

[willow basin]

https://tenor.com/view/pokemon-pikachu-eevie-race-pokemon-race-gif-22458610

[willow basin]

yeah I’m thinking docker hub to start, probably easier and more standard (and forces us to parametrize everything) but let me see if I can disentangle the spaghetti first

[naive knoll]

cool. happy to take some subtasks if you can break off some.

[willow basin]

@civic plover are you familiar with the cli/engine entanglement? I’ll start with grep of the whole codebase 😁

[naive knoll]

Super curious. If I want a CLI, I just go build -o bin/dagger ./cmd/dagger

[willow basin]

url might be hardcoded…

[naive knoll]

got it

[willow basin]

ie not parametrizeable in the build. maybe requires forking the code? mm

@naive knoll want to pair on it? im free in 15 mn

[willow basin]

(starting on dev audio, feel free to join if you want 🙂

[willow basin]

We got it to work!!

[civic plover]

awesome!

[willow basin]

@turbid python I pushed the module but got stuck on the bugs last night. Do you want to try pushing a bootleg release somewhere?

and getting someone to try it?

[turbid python]

yeah sure, dockerhub?

[willow basin]

that was going to be my choice but your call

btw I didn’t wire in the version string (cli always gets built as ‘devel’ so it always pulls tag main of the worker image)

[turbid python]

is this a good place to pick up from https://github.com/shykes/daggerverse/blob/main/dagger/examples/publish-zenith.gql

[willow basin]

yes

This worked yesterday - definitely start by making sure it works for you 🙂

(sorry I want nothing more than to work on this today, but pushed back a bunch of meetings to focus on zenith/daggerverse this week, and my bill is due today 😛 )

[turbid python]

looks like it works after a dagger mod sync

looks like next step might be to do a multiplatform publish? Publish looks good 🙂

[willow basin]

The next thing I want to fix, is that hacky FromZenithBranch field, I wanted to add various ways to get the source you want (from tag, from directory, etc) but adding a Src *Directory field to my custom type caused problems. Is that a known issue @civic plover ? Maybe fixed now, I'll try again this afternoon

[civic plover]

might be fixed now - I've had success placing a *Container on a struct at least: https://github.com/vito/dagger-gitutil/blob/93e46a5a20172b1373223e445ba18bc37e251569/main.go#L15

[willow basin]

yeah that works fine for @open sparrow too with inline python

which is why i was surprised

[turbid python]

cool! 🚀

[turbid python]

if anyone wants to try that, here's the darwin/arm64 binary. Make sure to unset _EXPERIMENTAL_DAGGER_RUNNER_HOST and probably kill your existing dev engine

[naive knoll]

cd random-dir-with-no-dagger-env-vars
mv ~/Downloads/dagger-zenith .
xattr -d com.apple.quarantine ./dagger-zenith
chmod +x ./dagger-zenith
echo '{container{from(address: "alpine"){trivyScan}}}' | ./dagger-zenith query -m github.com/jpadams/dagger-module-trivy

👍

[turbid python]

https://giphy.com/gifs/4KF85OSbyjVOfyjksJ

[naive knoll]

CONTAINER ID   IMAGE                             COMMAND                  CREATED         STATUS         PORTS     NAMES
decdadb42c47   kylepenfound/zenith-engine:main   "dagger-entrypoint.s…"   6 minutes ago   Up 6 minutes             dagger-engine-70b23725f5bd8e91

[turbid python]

til xattr -d com.apple.quarantine ./dagger-zenith, that's slick

[civic plover]

is that what prevents the 'Killed' issue?

[turbid python]

No, that's still unknown. I suspect that might be from getting a cached binary possibly. This is apple's quarantine thing if you download a binary from somewhere. The command is a way around having to go through the 'i'm an adult' dialog

yeah i just tested something with the killed issue from ./hack/dev.

• I ran ./hack/dev, then ./bin/dagger version - got the killed issue
• rm -r bin
• ./hack/dev again, ./bin/dagger version all good

[civic plover]

[turbid python]

something something overwriting/cache/timestamps 🤷

oh important note, before step 1 i had an existing bin/dagger

[willow basin]

oooh

I keep hitting that too

(have to overwrite with go build each time)

[turbid python]

Let me know if deleting the old binary first seems like a consistent fix

[willow basin]

Resurrecting this thread... Shall we get that bootleg release out? I have some free cycles

[naive knoll]

I used this bootleg from @turbid python
#1154286253286641755 message

which starts up kylepenfound/zenith-engine:main

but yeah, would be great to make that official and nightly with a GHA and cron

(as well as multi-platform/arch)

[willow basin]

I feel like I'm going crazy... Can't find the message !

But great, in that case is the next step to set up a nightly build somewhere, and change the README to do that?

[naive knoll]

@willow basin I've got a repo where I'm trying to get a nightly (or more frequent) release happening using a linux/amd64 dagger-zenith binary to bootstrap, but the current builds can't run your daggerverse/dagger module. When we get that working again, I'll continue.

https://github.com/jpadams/shykes-dagger-zenith-builder/

[willow basin]

Thanks for the report, I'll check after the meeting streak

[willow basin]

Weird, I just double-checked and it works for me

[willow basin]

@naive knoll does this work for you?

dagger query -m github.com/shykes/daggerverse/dagger <<-EOF
query test {
 dagger {
  engine {
   fromZenithBranch {
    source {
     entries
    }
   }
  }
 }
}
EOF

[naive knoll]

trying it out

[naive knoll]

This runs on a ./hack/dev engine and CLI, but not on a CLI created by shykes/daggerverse/dagger

So I did a dagger mod sync on that dagger module and got some differences in generated code. Going to try with that.

I can get things to run locally, just not in GitHub Actions, for some reason.

[naive knoll]

#1156815908271038474 message @civic plover

@willow basin when I build a linux/amd64 cli with daggerverse/dagger it's 3 MB smaller than one built with ./hack/dev (from a linux/amd64 container) and doesn't ever get off the ground. Hmm...less sure about this now
https://github.com/jpadams/shykes-dagger-zenith-builder/actions/runs/6334728876/job/17204793424
(hangs waiting for engine start)

The ./hack/dev-built one actually sorta starts
https://github.com/jpadams/shykes-dagger-zenith-builder/actions/runs/6334590841/job/17204430826

[turbid python]

@naive knoll having a look, let me know if you've made any progress!

[naive knoll]

Want to collab/pair a bit?

Maybe you can show me where I've gone astray

[turbid python]

dev-audio?

[turbid python]

Hey @willow basin mind if I push up a commit to your daggerverse/dagger with a fix for the image build?

[willow basin]

go for it