Mirrors

[pseudo cairn]

To be clear, do you need a container registry mirror or a Python package index?

[static lantern]

with the migration to uv (for python sdk) there is a new container image used (ghcr.io/astral-sh/uv). Now in enterprise environments that is not allowed, like in my case and you need to set container registry mirror. The problem with that is that our mirror requires credentials and I can not change than. I also can not set credentials in engine.toml (which if I understand correctly is just a buildkit config). With buildkit you'd set the mirrors and run docker login ... and that would solve my problem, but that does not work with dagger, at least it didn't in my case. Maybe somebody familiar with the dagger architecture could suggest how I could solve this.

For the time being I might be able to solve this with a nginx proxy to container registry mirror which also logs you in.
I think I'm not going to be the only one with this problem and would be nice if there would be a way to address this in dagger someway.

[pseudo cairn]

@winged flare, do we support this?

[winged flare]

We have support for authenticating to registries in a few ways: https://docs.dagger.io/faq/#how-do-i-log-in-to-a-container-registry-using-a-dagger-sdk

I would have thought that it worked transparently for mirrors, but I am not sure if there's integ test coverage for that so not 100% sure off the top of my head. If it doesn't work for mirrors I'd consider that a bug

cc @timid minnow in case you are familiar with this area of "auth stuff" 🙂

[timid minnow]

Hey @static lantern
I'm taking some cycles to investigate this, creating an issue and investigating it 👼 If you logged in via docker login, it should normally authenticate against your mirror, but happy to dig deeper on the stack, to understand where it might fail

[timid minnow]

https://github.com/dagger/dagger/issues/9014 Feel free to add context so that I can repro it faster 🙏

[static lantern]

Sorry for a delay from my side, I've been busy with other stuff.

I'm logged in into the mirror and I can do docker pull without any issues, uv images get pulled down via docker cli. Then running the dagger command fails because it can not pull from our mirror. This iswhat I'm doing.

engine.toml is:

debug = true
trace = false
insecure-entitlements = ["security.insecure"]
[registry."docker.io"]
  mirrors = ["OUR_MIRROR_DOMAIN"]
[registry."ghcr.io"]
  mirrors = ["OUR_MIRROR_DOMAIN"]

This command is working:

> docker login -u $USERNAME -p $USERNAME OUR_MIRROR_DOMAIN
> docker pull OUR_MIRROR_DOMAIN/astral-sh/uv:0.4.30

And dagger call fails

> cd to/dagger/project/with/python/sdk
> dagger functions
see attached log file

[static lantern]

I'm wondering if there is a way to change which uv image is used. in a similar way as you change base-image via pyproject.toml ... https://github.com/dagger/dagger/blob/6152c829bf7bdd053025c1821d19c439d4ed3a39/sdk/python/runtime/discovery.go#L423

[pseudo cairn]

There isn't, but I know there's several users that successfully use a mirror for ghcr.io (github container registry), even specifically for this particular image. Not entirely sure about what's the authentication methods they use though. What I remember more was aroud setting up the custom certificates.

[static lantern]

I got now this weird error where docker pull ghcr.io/astral-sh/uv works ... But when I run dagger pipeline it fails

I've whitelisted ghcr temporary

[pseudo cairn]

@timid minnow, were you able to look into it? Is there any more information you need from @static lantern to help you repro?

[static lantern]

I needed I can also just on a call if that helps

[pseudo cairn]

@static lantern, what's your timezone?

[static lantern]

CET

[pseudo cairn]

Ok, that's tough because Guillaume is on PST so it could be too late for you when he starts his day.

[static lantern]

no worries, I'm usually up late in the day (also work with PST timezome)

[timid minnow]

Hey, sorry I was willingly ignoring the pings to focus on something

A sync would totally work, I can accomodate 🙏

When would you be available, tomorrow ?

[static lantern]

Sure would tomorrow 21.00CET work for you?

[static lantern]

@timid minnow ^^^

[timid minnow]

Hey @static lantern just saw it 🙏. It seems I am still on time 👼 , in an hour ?

[timid minnow]

Thanks for pinging, do not hesitate ahah, otherwise I don't see

[static lantern]

Awesome. See you soon

@timid minnow my kids are sleeping (finally! :P) if you want we can already jump on a call. feel free to just call me

[timid minnow]

Awesome, give me 2 mins 🙏

wanna go on dev-audio ?

[static lantern]

sure

just not sure I know how to do this. I'm using discord via browser