#Registry Auth with buildkit?

1 messages · Page 1 of 1 (latest)

hardy scroll
#

Trying to do some search through the discord, but can't seem to find anything. Any examples of registry auth for cloak and/or go sdk?

sage wedge
#

It’s not currently available in cloak. I know @nova prairie, @tardy pulsar, and @unborn umbra are aware of it, but I’m unsure of the timing for implementation.

unborn umbra
sage wedge
brave rain
#

@glass nest just something to think about for docs - maybe a quick guide or just add in the FAQs?

glass nest
glossy burrow
#

Hi I think i'm running into a similar issue. Here's my output, it's complaining about buildkit config. My company blocks pulling from public repos and proxies docker images via Artifactory so I would want buildkit to go there. However, I am not sure why buildkit is trying to pull golang:1.18.2-alpine. Is that for use of Dagger engine?

Building with Dagger
#1 resolve image config for company.registry.com/golang:latest
#1 DONE 4.2s

#2 mkfile /Dockerfile
#2 DONE 0.0s

#3 mkfile /main.go
#3 CACHED

#4 [internal] load metadata for docker.io/library/golang:1.18.2-alpine
#4 ERROR: failed to do request: Head "https://registry-1.docker.io/v2/library/golang/manifests/1.18.2-alpine": tls: first record does not look like a TLS handshake
------
 > [internal] load metadata for docker.io/library/golang:1.18.2-alpine:
------
input:1: container.from.withMountedDirectory.withWorkdir.exec build shim: golang:1.18.2-alpine: failed to do request: Head "https://registry-1.docker.io/v2/library/golang/manifests/1.18.2-alpine": tls: first record does not look like a TLS handshake

Please visit https://dagger.io/help#go for troubleshooting guidance.

I am trying to follow the getting started guide. Would I have to configure buildkit in some way? Or can I tell dagger engine to use my company proxy?

#

I am authenticated to my company repo via docker login however since dagger is trying to use the public docker registry that auth doesn't work.

nova prairie
# glossy burrow Hi I think i'm running into a similar issue. Here's my output, it's complaining ...

Yeah. Dagger does respect whatever docker login you make

However, we do build a container on the fly using golang (injected inside the Exec — it’s responsible for gathering stdout/stderr):

https://github.com/dagger/dagger/blob/main/core/shim/cmd/Dockerfile

GitHub
dagger/Dockerfile at main · dagger/dagger

A programmable CI/CD engine that runs your pipelines in containers - dagger/Dockerfile at main · dagger/dagger

#

@glossy burrow Do you have to pull images from a different registry? Or just some kind of config?

#

[for monday] /cc @tardy pulsar

glossy burrow
#

Thank you for getting back to me. Yes, we have to pull images via Artifactory using my company domain. So for example, instead of golang:alpine we'd use company.domain.com/golang:alpine. It essentially proxies the same public dockerhub image via Artifactory so that we control rate limits and caching etc.

#

I would assume this is a common pattern in large enterprises.

#

It's not a 100% blocked when pulling from local machines. For example, dagger didn't have any trouble pulling the moby/buildkitd image without the company.domain.com/moby/buildkitd but our CI systems will not be able to pull that image.

nova prairie
#

Thanks, duly noted. We’ll come up with a solution. Thanks for reporting!

vernal kayak
#

Yes, I have the same issue in my company.

vernal kayak
#

I don't know if it's a good way. But I have made something like this:

func awsLogin(profile, region, registry string) error {
    var out bytes.Buffer

    cmd := exec.Command("aws", "ecr", "get-login-password", "--region", region, "--profile", profile)
    cmd.Stdout = &out

    if err := cmd.Run(); err != nil {
        return errors.Wrap(err, "Could not get the aws login")
    }

    token := out.String()

    out = bytes.Buffer{}
    cmd = exec.Command("docker", "login", "--username", "AWS", "--password", token, registry)
    cmd.Stdout = &out

    if err := cmd.Run(); err != nil {
        return errors.Wrap(err, "Could not authenticate docker on AWS")
    }

    return nil
}

And after I call it like this:

func main() {

    ctx := context.Background()

    client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stdout))

    if err != nil {
        fmt.Fprintln(os.Stderr, "Could not connect to dagger engine: ", err)
        os.Exit(1)
    }
    defer client.Close()

    err = awsLogin("my-profile", "eu-west-3", "id-of-my-account.dkr.ecr.eu-west-3.amazonaws.com")

    if err != nil {
        fmt.Fprintln(os.Stderr, "Could not connect to AWS ECR:", err)
    }

    src := client.Host().Workdir()

    output := build(client, src)

    os.Mkdir("dist/", os.ModePerm)

    _, err = output.Export(ctx, "dist/")
    if err != nil {
        fmt.Fprintln(os.Stderr, "Cannot export golang.output: ", err)
        os.Exit(1)
    }

}
glossy burrow
#

This didn't work for me 😦 it's still trying to pull from docker.io

vernal kayak
#

hum, after in the build function I did something like this:

dagger.Container().From("id-of-my-account.dkr.ecr.eu-west-3.amazonaws.com/public/my-image:latest")
glossy burrow
#

I understand how the artifactory proxy works and we have it internally set up and working. My From is using that proxy. However, this particular problem happens before we get to the point of pulling the container we define. Dagger itself pulls a golang container docker.io/library/golang:1.18.2-alpine for it's internal use. I have no control over the prefix registry for that pull hence the issue.

I was able to sort of work around it by running my own buildkit container and passing in a config as shown here - https://docs.docker.com/build/buildkit/configure/. This is my config.toml

debug = true

[registry."docker.io"]
    mirrors = ["mycompany.domain.com"]

However, now I'm running into the other issue i was afraid of, PROXY. Dagger engine does a apk add file git into that golang image but that fails to work because my company's corporate proxies are not available within that image. I can't figure out a way to add those.

I'm certain Dagger engine can help alleviate most of these concerns by allowing users to pass through certain env vars and registry config to the default buildkit image. FYI @nova prairie

Would it be better if I create an issue to track this? Or is the dagger team already planning based on one?

glossy burrow
#

@merry atlas I assume you'll be interested in this too as you worked on the corp proxy issue for the CUE sdk

glossy burrow
nova prairie
#

Yes please! I think #3726 is about HTTP proxy rather than registry mirror

#

@glossy burrow If you can open an issue that would be amazing

Are you looking for mirrors support explicitly (e.g. for every .From() transparently) or just a way to fix the golang issue?

glossy burrow
#

Thank you @nova prairie !

My issues are twofold. One is the above mentioned corp proxy.

Second is the fact that the docker image pulled by Dagger engine is not able to be proxied via private registry. At this point i'm just looking for a way to fix the golang image issue as I have a workaround for the .From() as I mention in my issue.

I have opened a new issue for the second - https://github.com/dagger/dagger/issues/3829

I'm afraid both these issues go hand in hand. They will both need to be fixed for me to get unblocked. Please let me know if I can help in any way 🙂

nova prairie
#

@glossy burrow Thanks a lot. We’ll look for a proper solution

In the meantime, to get you unstuck, it’s a bit hacky but you could use a modified version of dagger locally to stop that FROM golang from happening.

If this is something you’d like to try out I can guide you through

#

The process would be this:

  1. git clone https://github.com/dagger/dagger.git

  2. git checkout sdk/go/v0.4.0

  3. Edit core/shim/cmd/Dockerfile -- replace FROM golang:1.18.2-alpine with your corporate proxy

  4. In your project using dagger, edit go.mod and add these lines:

replace github.com/dagger/dagger => <local path to dagger repository>
replace dagger.io/dagger => <local path to dagger repository>/sdk/go
#

That won't fix the corporate proxy, unfortunately

glossy burrow
#

Awesome! I will test that tomorrow. I can probably squeeze in the http proxies to the same Dockerfile right?

nova prairie
#

Unfortunately, no. That Dockerfile is only for a small binary in the system

#

However, for proxies, are the env variables the only thing that needs setting?

In which case, you could do that manually for now:

c.Container().
  From("alpine")
  WithEnvVariable("HTTP_PROXY", os.Getenv("HTTP_PROXY")).
  Exec(dagger.ContainerExecOpts{Args: ["apk", "add", "bash"})

Would this work or is there more magic to it?

glossy burrow
#

Well, for the golang:alpine image that Dagger Engine pulls, I can't pass in env vars 😦

#

I can do what you suggested for any .From() image but I can't do that for the initial Dagger Engine pull, which is where it's failing

#

Actually, I'm honestly not sure why git and file are needed for that docker image? Can't we just remove it?
RUN apk add --no-cache file git
It's just running a go build and copying a binary to scratch if I'm reading it correctly.

nova prairie
#

Oh then yes, you can put the http proxies in that Dockerfile

#

It will be built like a regular Dockerfile

#

I think file and git are required by go build. But if it builds fine without those, then it's definitely not needed

glossy burrow
glossy burrow
glossy burrow
nova prairie
#

Of course, you can try it out but there’s a few steps involved

From our cloned repo, do

  1. ./hack/make engine:dev

  2. That should give you a export DAGGER_HOST=… command to run in order to use the dev engine

  3. to use the updated Go SDK in your project, add to your go.mod: replace dagger.io/dagger => <path to dagger repo>/sdk/go

glossy burrow
#

Hey @nova prairie , I tried running ./hack/make engine:dev but that process is still trying to pull an image from docker.io.

#1 resolve image config for mycompany.registry.com/golang:1.19-alpine
#1 DONE 0.5s

#2 mkfile /Dockerfile
#2 DONE 0.0s

#3 mkfile /main.go
#3 CACHED

#4 [internal] load metadata for docker.io/library/golang:1.18.2-alpine
Error: input:1: container.from.withEnvVariable.withWorkdir.withMountedDirectory.exec build shim: golang:1.18.2-alpine: failed to do request: Head "https://registry-1.docker.io/v2/library/golang/manifests/1.18.2-alpine": tls: first record does not look like a TLS handshake

Please visit https://dagger.io/help#go for troubleshooting guidance.
exit status 1
#4 ERROR: failed to do request: Head "https://registry-1.docker.io/v2/library/golang/manifests/1.18.2-alpine": tls: first record does not look like a TLS handshake
------
 > [internal] load metadata for docker.io/library/golang:1.18.2-alpine:
------

I replaced all golang:1.18.2-alpine references in the repo to mycompany.registry.com/golang:1.18.2-alpine but the issue is still there. I can't figure out where that 1.18.2 reference is coming from.

tardy pulsar
tardy pulsar
# glossy burrow Hey <@707661669819613324> , I tried running `./hack/make engine:dev` but that pr...

Okay it's that issue but it's also that even building the engine itself locally uses the dockerhub golang image.

A different workaround is to prefix all your commands with DAGGER_HOST=docker-image://ghcr.io/dagger/engine:main@sha256:c77257ff049afc2b97786f61b135fe572b7ce53c82ce23d1a0227a1821dd2f1b

You shouldn't even need to do ./hack/make engine:dev then. Just prefix with that env var (or export it) and run whatever you wanted to run.

What's happening is that we publish engine images whenever we merge into our main branch, so I got that image ref from here: https://github.com/dagger/dagger/pkgs/container/engine/51369877?tag=main

And when you run with that DAGGER_HOST env var you'll be telling dagger to use that image instead of the older one. That image was built last night after merging the PR w/ the fix to the shim, so I think it should work for you now.

#

(I haven't fully caught up on this whole thread so let me know if I'm missing something here)

glossy burrow
#

wonderful! Let me try that

glossy burrow
#

Nice! That worked! That's also a great way to test the latest engine. Thank you @tardy pulsar !

glossy burrow
#

Hey @nova prairie , now that the shim is included in the engine, there is one remaining image that needs to be potentially proxied. The dagger engine image that gets downloaded by the sdk. Do you have any thoughts on how that can be handled? So essentially,

ghcr.io/dagger/engine:main should be customisable to mycompany.domain.com/dagger/engine:main