Dagger on offline network

[young rain]

Has anyone run dagger on a completely offline network.

I've gotten really far on this and can point dagger at an offline docker registry and an offline go proxy. However when my modules go to load the python sdk they're trying to load quite a lot of stuff off of github.

Any advice?

[modern solstice]

hey Mike, we don't officially support this use-case but there's been several threads in Discord and issues in github about users trying to make Dagger work in airgapped environments. Even though we want to support it, we don't have the bandwidth right now to do a full assessment for that work.

Having said that, if you'd like to give it a try, we can give you pointers and expedite some engine changes that will allow you to make progress.

cc @exotic warren since I recall has been involved in this topic in the past

[young rain]

got it.
I've looked at the past help on this and im pretty far

I can load dagger by using an offline go proxy and i can run built in jobs the only advice i could use is on packaging dependencies with my custom modules written with the python sdk - right now when i hit the pythonSdk call (which you can see by setting verbose to about 10 ) you can see it reaching out to github for 10+ code packages as it runs a generate call.....I think i just need these to be packaged so that this call is cached in advance

[young rain]

@modern solstice i do actually have this working - ill have a github example online and post it

[modern solstice]

@young rain so you've got the whole thing working offline?

happy to help promoting a blogpost 🙏

[young rain]

@modern solstice i made a mistake and spoke too soon 🙁

I'm at the very end of the process though - the last thing that happens when you run a self developed module is that it loads all the sdk dependencies. it seems like i can grab these from the uv.lock file in the module ....i'm just trying to figure out if i can provide them to dagger

[young rain]

@modern solstice sorry it took me a little bit to get back to this I had a few other things going on....im pretty sure now i have it working offline. If you guys are interested I could do a more thorough writeup and post my setup on github.

First you need three different proxies setup

1. A go proxy with the following dependies preloaded

• The dagger.io go module for your specific engine of dagger
• The dagger.io python sdk (since i also want python) for your specific version of the dagger engine

2. A pip proxy with the following dependencies preloaded

• uv-build
• The dependencies in the uv.lock file of the dagger python sdk
• Any other pip packages you want in a pipeline

3. A docker proxy will the following images loaded

• Anything you want to use in a pipeline
• These images:

    "ghcr.io/astral-sh/uv:0.9.8"
    "python:3.13.9-slim"
    "python:3.13-slim"
    "alpine:latest"
    "python:alpine"

I run all these proxies in a docker compose file to keep it simple.

**Then the dagger engine must be run wiht custom configuration:
**

1. engine.json must be setup like this

  "registries": {
    "docker.io": {
      "mirrors": ["172.17.0.1:5000"]
    },
  "172.17.0.1:5000": {
      "mirrors": [],
      "http": true,
      "insecure": true,
      "ca": []
    }
  }

}

Where 172.17.0.1:5000 is pointing at my docker registry.
2. You must run the engine docker container with the following env

Then you must properly setup your python sdk modules (im using python) to point at the pip proxy

Finally in pyproject.toml of my python sdk modules i need the following line

url = "http://172.17.0.1:3000"
default = true

Where 172.17.0.1:3000

Is the pip proxy

You should then be able to run the dagger tui but you need to source this env in your shell pointing to your custom running container:

export _EXPERIMENTAL_DAGGER_RUNNER_HOST=docker-container://dagger-engine-custom