#trying to push to a loacl registry is failing due to credentials

1 messages · Page 1 of 1 (latest)

lofty trail
#

Hi Folks

func (it *PfTools) Publish(
    ctx context.Context,
    registry *dagger.Service,
) (string, error) {

    addr, err := registry.Endpoint(ctx)
    if err != nil {
        return "", err
    }

    prodImage := dag.Container().
        From("alpine").
        // WithServiceBinding("registry", registry).
        WithExec([]string{"apk", "add", "curl"})

    _ = prodImage

    // publish to ttl.sh registry
    pubaddr, err := prodImage.Publish(ctx, fmt.Sprintf("http://%s/app/tools:latest", addr))
    if err != nil {
        return "", err
    }

    return pubaddr, nil
}


dagger call publish --registry=tcp://localhost:5000

error:

Stdout:
invoke: input: container.from.withExec.publish resolve: failed to export: failed to push http://ss98c1kcd4ctg:5000/app/tools:latest: invalid reference format

above is my function and how i am calling it. i have a registry2 container running in docker compose outside of dagger. i am trying to push images to this registry

when i remove the http prefix from the address this is what i get

invoke: input: container.from.withExec.publish resolve: failed to export: failed to push ss98c1kcd4ctg:5000/app/tools:latest: failed to do request: Head "https://ss98c1kcd4ctg:5000/v2/app/tools/blobs/sha256:ee4ed94bf39fc6fd0f447de438e6c6b2eea8ea0eb840fa4a790bea9e448ddaf0": dial tcp: lookup ss98c1kcd4ctg on 10.87.0.1:53: no such host
forest fable
#

ok, this is tricky as publish happens within the engine container and doesn't have the same DNS config as the other containers

#

@lofty trail the "easiest" workaround for this is to start your local registry connected to the engine and use localhost to publish.

ref: #1271583698365579395 message

#

LMK if that's a possible approach for you

#

I'll open an issue so we can support this better 🙏

#

cc @slender flax

lofty trail
#

thats amazing thanks for all your help ! i will go ahead and see if i can do the work around

lofty trail
# 
debug = true
insecure-entitlements = ["security.insecure"]

[registry."local.engine.registry.io"]
  mirrors = ["localhost:5000"]

above is my engine.toml

func (it *PfTools) Publish(
    ctx context.Context,
    // registry *dagger.Service,
) (string, error) {

    // addr, err := registry.Endpoint(ctx)
    // if err != nil {
    //     return "", err
    // }

    // curl -X GET http://local.registry.io:5000/v2/_catalog
    // curl -X GET http://local.engine.registry.io/v2/_catalog

    prodImage := dag.Container().
        From("alpine").
        WithExec([]string{"apk", "add", "curl"}).
        Terminal().
        // WithServiceBinding("registry", registry).
        WithExec([]string{"apk", "add", "curl"})

    _ = prodImage

    // publish to ttl.sh registry
    pubaddr, err := prodImage.Publish(ctx, "local.registry.io:5000/app/tools:latest")
    if err != nil {
        return "", err
    }

    return pubaddr, nil
}
# 
    container: Container!
    Container.from(address: "alpine"): Container!
    Container.withExec(args: ["apk", "add", "curl"]): Container!

dagger / $ curl -X GET http://local.engine.registry.io/v2/_catalog
curl: (6) Could not resolve host: local.engine.registry.io
dagger / $

this is what i get when i do curl from inside the container

#

i also tried the old etc/hosts trick by adding a dns entry that pointed to my localhost ip. i also tried using my 192.168.a.b address both failed 😦

#

this is what happens when i publish using the above i.e i dont stop in the terminal

forest fable
#

mirrors works diferently @lofty trail. if you followed my example in the link above on how to connect the registry container into the engine, using localhost:5000 should work directly

lofty trail
# forest fable `mirrors` works diferently <@223431830173515778>. if you followed my example in ...

got it ... now ftr i need to push the image to the registry once if it does not already exist . this is why i need to be able to publish preferaly using sdk. I am still scratching my head but technically what i am doing now is

1. make sure dagger engine is already running
2. using go (outside of dagger sdk/functions ) check if a dir in the file system is present ?
  if not call the `build-tools` dagger function so that the `tools` binaries/containers are built
3.1. publish resulting artifacts to a registry from inside the `build-tools` dagger function
3.2 if 3.1 is not possible export tp a local folder instead of using publish
4. start the registry using go again after 3 is completed
  publish the containers from 3 to it
  stop it
5 start the registry in the same network as dagger container again
6 use the published binaries from inside dagger funnctions from here on

the whole point of this exercise is to do 3 only once preferably as it takes long time to do . so this is supposed to speed up the build. my assumption is that there is maybe a much better way to acheive the same result

i appreciate all your help thank you !

#

also from the docs it would be nice if the docs outlined why we have to do all this dance. as in what are the constraints . i did not know it at first but i am realising now that you cant push to a non secure registry if its not 127.0.0.1 would be nice if the docs stated those constraints at least in the custom registry section

forest fable
#

👋 couple of questions about this.

First of all, is there a reason for not running all this steps within a Dagger function itself? Do you have a hard requirement of an external Go app that orchestrates all this? For what you're describing above seems like you could achieve all you need by only using Dagger.

Now, addressing each step independently.

  1. Related to my comment above, if you orchestarte everything with Dagger this step is "free" since Dagger will check for this automatically and it will start the engine if it's not running.
  2. This could be handled withing Dagger pipeline also, this way you don't have to have a Go program calling into dagger functions. Take into account that functions can be only called via dagger call currently so you won't be able to call Dagger functions via a Go program yet.
    3-6. Do you need a registry at all? Seems like if you run everything within Dagger, you can construct your image and run it in your pipeline without the need to use a registry as a bridge between your Go program and your Dagger pipeline.
lofty trail
#

so the idea for is for each developer to run a registry locally only do the step that takes the 3hrs once. all the resulting binaries will be stored in this local repository from there on

forest fable
#

That's why I'm saying that you don't really need a registry for this

lofty trail
#

you mean the cloud offering ? or out of the box ?

forest fable
#

No, the OSS Dagger has local caching

lofty trail
#

ok having a face palm moment then 🙂

forest fable
#

That's the power of Dagger

#

Let me give you an example

#

what's your SDK of choice, Go?

lofty trail
#

go indeeeed

forest fable
#

so given this Dagger function for example

func (m *Mymodule) Test(msg string) *dagger.Container {
    return dag.Container().From("alpine:latest").
        WithExec([]string{"echo", msg}).
        WithExec([]string{"sleep", "20"}) // imagine this is a super long task
}
#

if you call dagger call test --msg foo the first time, it'll take 20s

#

now if you call it again afterwards, it will run immediately

#

because everything is cached and you didn't change any inputs

lofty trail
#

oh i see ...

forest fable
#

now if you call dagger call test --msg bar, you'll see that it will take 20s again because you invalidated the chain before the sleep

#

this works the same way as Dockerfile cache layers if you're familiarized with that

lofty trail
#

indeeeed ...

#

so all i have to do is willy nilly pass containers around ... nice !

#

ok one last question if i may ...

#

unrelated

forest fable
#

Dagger Cloud gives you a distributed cache so you can share the same cache across your team

#

so each developer doesn't have to wait 3hs the first time they run their stuff

forest fable
lofty trail
#

indeeeed i love that ui !

#

now ... so far my only issue is that i cant depend on a shared private repo so far. i need to share some code between my pipelines

forest fable
#

could you expand? 🙏

lofty trail
#

i have a private go repo.

#

i am unable to access it from my dagger functions basically

#

unless i am missing something it seems impossible

forest fable
#

inside Dagger?

lofty trail
#

no i want to use an exported go function (not dagger function) from my dagger pipeline

#

daggerfunc --> private gorepo.exported func

#

the exported thing is anormal go const, function

forest fable
lofty trail
#

oh coooool !

forest fable
#

cc @harsh charm and @vivid carbon to make sure I'm being accurate here

forest fable
#

in the meantime I guess you can go mod vendor your private dependencies

#

until this is out

lofty trail
#

ok i never used go vendor until now

forest fable
lofty trail
#

yeah good point

#

really appreciate all your help here .. you are amazing !!!!

#

can i export dagger functions from a vendored repo ?

forest fable
#

that's because the dagger CLI does some bootstrapping to make sure the Dagger engine is started and configured so your functions can be correctly initialized and called

lofty trail
#

looks like even vendoring does not work 😦

#

its refusing to even accept the replace directive in the go.mod file ... super weird

vivid carbon
lofty trail
#

hi @vivid carbon i am using github access tokens. that part is definitely working fine

#

also thank you for jumping on this !

#

so far i tried replace , and go.work as well as using vendoring. when i do dagger develop thats when it fails i cant proceeed past that

#

i am just realising now ... during the dagger develop how does it get my credentials ? i usually have my ssh keys added at all time so i am hoping its able to pick up my credentials

vivid carbon
# lofty trail i am just realising now ... during the `dagger develop` how does it get my crede...

Ok, so Dagger currently does forward the SSH_AUTH_SOCK to the engine when the env var is present, BUT, the Go toolchain relies on git rewrite rules to convert the Go ref of your private repo into an SSH ref.

And, we currently do not forward it, mostly because of some design decisions on our end: https://github.com/dagger/dagger/issues/7787

As I've been working on that subject lately, could you please add a comment to restart the conversation ? If we all agree on that, the implementation wouldn't be too hard IMO

--

Now I am curious on your process, you talk about the github access tokens, do you use git rewrite rule to have your Go toolchain rely on that or it's just rewrite rules to convert the ref into an SSH one ? Or do you use git credential manager ?

#

We do support go proxies, so a temporary hack would be to rely on this feature to authenticate against this private repo in the meantime

For your information, I am starting to work on https://github.com/dagger/dagger/issues/7990 this week. The design decision is to also make it compatible with the Go toolchain to the PAT token, probably with the git credential manager

lofty trail
#

i am curious. does this explain why vendor and the replace directive both fail ?

#

on my desk or building normal code i use my ssh-agen to keep my ssh keys loaded. this way i can clone any private repo and build any private repo from my shell

#

i dont do anything special with git rewrite rule to have your Go toolchain this is the only tidbit in my ~/.gitconfig that is maybe close to what you are mentioning. is this what you mean ?

[url "ssh://git@github.com/<mycompany>/"]
  insteadOf = https://github.com/<mycompany>/
vivid carbon
lofty trail
#

i would be more than happy to test this ... i cant wait 🙂

vivid carbon
#

Could you advance without that in the meantime? 😇

lofty trail
#

i am doing some horrific stuff .. like putting the go code in an embed and generating it on the each module that needs it .. yuckkkk .. lol

vivid carbon
vivid carbon
lofty trail
vivid carbon
#

That's why the vendor and rewrite did not work exactly yes