Private mirror for ghcr.io

[slow rivet]

@serene raft, what I meant is that it appears to me that you've setup things correctly in #python message if you got to the Container.withEnvVariable(name: "DAGGER_UV_IMAGE", ... step, which will only change if you use the tool.dagger.uv-version setting in pyproject.toml and won't reflect any mirror settings, so isn't a barometer on whether your mirror is working or not.

It should have failed earlier, in the Container.from(address: "ghcr.io/astral-sh/uv:0.4.9@...") step. Or does it actually work if you don't setup the mirror, and you're just trying to route it as a best practice within your company?

Can you please find the part that looks like this if you run with dagger -vvv and mirror setup:

    ✔ Container.from(address: "ghcr.io/astral-sh/uv:0.4.9@sha256:d8a3dbcb14b1e20a5e8baafe8dc9981e9bec88a3eef6b24f5392770015502071"): Container! 0.0s
      ✔ resolving ghcr.io/astral-sh/uv:0.4.9@sha256:d8a3dbcb14b1e20a5e8baafe8dc9981e9bec88a3eef6b24f5392770015502071 0.0s
      ✔ cache request: pull ghcr.io/astral-sh/uv:0.4.9@sha256:d8a3dbcb14b1e20a5e8baafe8dc9981e9bec88a3eef6b24f5392770015502071 0.0s

It's possible for example that Container.from() shows ghcr.io while the steps underneath reveals your mirror's domain.

Disclaimer: never tried this, but trying to figure it out with you!

@Chad pointed out that you need the same sha into the mirror. You may also try setting up a different uv version than the default so it doesn't add that sha, that should be easier for you to test:

# pyproject.toml
[tool.dagger]
uv-version = "0.4.8"

Example output:

-    ✔ Container.from(address: "ghcr.io/astral-sh/uv:0.4.9@sha256:d8a3dbcb14b1e20a5e8baafe8dc9981e9bec88a3eef6b24f5392770015502071"): Container! 0.0s
+    ✔ Container.from(address: "ghcr.io/astral-sh/uv:0.4.8"): Container! 1.9s
-      ✔ resolving ghcr.io/astral-sh/uv:0.4.9@sha256:d8a3dbcb14b1e20a5e8baafe8dc9981e9bec88a3eef6b24f5392770015502071 0.0s
+      ✔ resolving ghcr.io/astral-sh/uv:0.4.8 1.8s

[serene raft]

unfortunatelly it does change repository source 😦

docker stop dagger-engine-957f182d61bac893
docker system prune -a -f
dagger functions --progress plain -vvvvvv
208 :         Container.from(address: "ghcr.io/astral-sh/uv:0.4.8"): Container! =
209 :         resolving ghcr.io/astral-sh/uv:0.4.8
208 :       Container.from(address: "ghcr.io/astral-sh/uv:0.4.8"): Container! =
209 :         resolving ghcr.io/astral-sh/uv:0.4.8 DONE [2.0s]
240 :         Container.from(address: "docker.io/library/python:3.12-slim@sha256:8ac54da5710cdd31639bb66f5bc1888948fc2866c0b5b52913b4b33d8252e510"): Container! =
241 :         resolving docker.io/library/python:3.12-slim@sha256:8ac54da5710cdd31639bb66f5bc1888948fc2866c0b5b52913b4b33d8252e510

[slow rivet]

Sorry, I don't follow. The point of adding uv-version to pyproject.toml was to indeed change the image address to pull from, in order to remove the digest, so you didn't have to make sure your mirror had that exact same one.

Did you run that on the custom runner with your mirror config? Can you confirm that the config is being applied? You can for example introduce an error and check that it fails. For example:

[registry."docker.io"]
  mirrors = ["doesnt.exist.com/upstream"]

If the mirror config is indeed being applied, it's likely that Buildkit doesn't actually report it in the logs, so you'll need another way to confirm if it's actually going through it. @tacit plaza, do you know?

@serene raft, can you please confirm if in your environment a non-working mirror config should fail, or does it work anyway but you're just trying to follow best practices within your company? I'm asking just to understand how to know if it's working or not.

Looks like buildkit doesn't report if it's using a mirror: #1275813561435951105 message

[serene raft]

Currently right now, there is no such an image in our internal repository, in the path docker.company.com/upstream/astral-sh/uv:1.2.3 I would expect dagger to fail with error: the image is not available or something similar.
I am using custom engine

echo $_EXPERIMENTAL_DAGGER_RUNNER_HOST
docker-image://docker.company.com/jozef/dagger-custom-engine:v0.13.0

with configuration

docker exec dagger-engine-957f182d61bac893 cat /etc/dagger/engine.toml
debug = true
trace = false
insecure-entitlements = ["security.insecure"]

[registry."docker.io"]
  mirrors = ["docker.company.com/upstream"]

[registry."ghcr.io"]
  mirrors = ["docker.company.com/upstream"]

built like this

FROM docker.company.com/baseimage/ubuntu:jammy AS company-ubuntu
FROM registry.dagger.io/engine:v0.13.0 AS dagger-engine

ENV DAGGER_UV_IMAGE=docker.company.com/upstream/astral-sh/uv:latest

COPY engine.toml /etc/dagger/engine.toml

# Add szn CA certificate
COPY --from=company-ubuntu /usr/share/ca-certificates/company-ca/* /usr/local/share/ca-certificates
RUN update-ca-certificates

ADD Dockerfile /Dockerfile

[slow rivet]

The ENV DAGGER_UV_IMAGE line doesn't do anything, you can remove it.

Pinging @hollow flint too if you know how this works.

[serene raft]

thank you for your support by the way 🙂

and patience as well 😉

[slow rivet]

I'm learning too! Thanks for your patience! 🙂

How do you start the engine?

Is it working for you for docker.io images? You can test it directly (faster) with:

dagger core container from --address=docker.io/python:3.13-rc-slim

(should be an image you know doesn't exist in your mirror)

[serene raft]

engine is started by dagger client, substitution does not work on docker.io either, there is no such image python:3.13-rc-slim in our repository

dagger core container from --address=docker.io/python:3.13-rc-slim
✔ Container.from(address: "docker.io/python:3.13-rc-slim"): Container! 1.9s
  ✔ resolving docker.io/library/python:3.13-rc-slim 1.9s
  ✔ pull docker.io/library/python:3.13-rc-slim 0.0s

[slow rivet]

engine is started by dagger client

But can you show the full command you use? You only showed the Dockerfile to build it.

If it doesn't fail with that, doesn't it look like the mirror config isn't being applied? Or at least you're not running commands on the custom runner.

[tacit plaza]

sorry, just checking this out. if the image doesn't exist in your repository, it'll fetch it from the upstream one

@serene raft if you do a docker logs $engine_container "AND" the image doesn't exist in your repository, you should see the engine attempting to pull it from there initially

[glass turret]

Btw, if you are attempting to get this running offline I went down this path. I got past the astral uv image issue via the mirroring. So got everything to pull okay but then ran into issues with the uv.lock files referencing the internet. I had to then run a custom python sdk to get past that. At which point I didn't need the uv copying into the python image (since my custom image I overrode in the base image of the pyproject.toml had uv build it into it) so I yanked that bit from the runtime. All said and done I did get it working..but there is likely more work ahead of you once you get your mirrored images into the local registry.

With that said it looks like the uv.lock file in the codegen project has since been removed with 0.13 I think? I might be able to attempt this again without using a custom sdk, but I haven't tested it.

[slow rivet]

Glad you made it work but would also like to understand your use case a bit more to make it easier in next versions. The sdk/codegen/uv.lock was removed because it's now a member of the parent workspace, so it was basically merged in sdk/uv.lock. I'm not sure what your issue was though. uv has an offline mode, is that what you needed? Also, uv's version has gone through quite a version bump (0.2.x to 0.4.x), so there's several improvements there.

[glass turret]

So first what I did is create an image like so:

FROM python:3.11-slim
RUN pip install uv
RUN apt-get update && apt-get install -y curl bash

COPY pip.conf /root/.config/pip/pip.conf

ENV UV_INDEX_URL=https://<private repo url>/repository/pypi-all/simple

ENV UV_NATIVE_TLS=true

COPY ca-certificates.crt /usr/local/share/ca-certificates/

ENV REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt

Some of this is overkill I know because of the custom-ca certs should already be added via the engine but leaving nothing to chance and I am installing things I don't need long term... The pip.conf has a reference to the private pypi mirror. So I haven't refined it. So then I make that my base image via the pyproject.toml. I then remove the uv.lock files and things work. If I did not remove the uv.lock files the codegen step would fail attempting to access the internet. So if uv has an offline mode maybe I can adjust the base image to enforce that? Remember that this has to work not just for my module but for the codegen step...which also uses python and uv... and not something I have control over via the module AFAIK.

[slow rivet]

There's a couple of things I want to do to clean this up:

• Centralize codegen, pre-build it, whatever.... so it doesn't have to be installed at runtime
• Remove the uv.lock from the generated sdk dir

Then you should just need to add index-url = "..." to [tool.uv]. I'm not sure how you'd manage offline = true because the dependencies need to be primed in the uv cache or available locally for it to work.

But let me get this straight. Even with UV_INDEX_URL in the base image it doesn't work unless you delete uv.lock?