How to run a dagger pipeline written in the GO SDK in a K8s Pod? | Dagger | Page 1

graceful pebble Jan 14, 2023, 5:07 AM

#

I love dagger. Have even made a YouTube video on the previous version. I'm planning to cover the new sdk (go and GraphQL) in my next video.

Just one question... How to run a dagger pipeline written in go on Kubernetes... In a pod.

tender cradle Jan 15, 2023, 1:51 PM

#

👋 here's some brief information about it: https://github.com/dagger/dagger/blob/main/core/docs/d7yxc-operator_manual.md#connection-interface

Even though we don't have a full example yet, the steps should be roughly the following:

Start a k8s pod with the dagger engine
Deploy your Dagger pipeline in a pod that has a role which can access pod/exec resources.
Set the _EXPERIMENTAL_DAGGER_RUNNER_HOST variable with the kube-pod:// scheme as described in the link above.

graceful pebble Jan 22, 2023, 4:36 PM

#

That seems like a bit of an anti-pattern in Kubernetes.

Ideally i would want a runner and executor kinda a setup where we have the dagger daemon (runner) running as a long lived process. The runner would spin up Kubernetes jobs to run the steps

tender cradle Jan 22, 2023, 4:39 PM

#

graceful pebble That seems like a bit of an anti-pattern in Kubernetes. Ideally i would want a ...

We know, we don't have an operator if that's what you're looking for. We'll eventually get there, but that's not currently a main priority.

graceful pebble Jan 22, 2023, 4:41 PM

#

tender cradle We know, we don't have an operator if that's what you're looking for. We'll even...

I see. Thanks for the info.

obsidian forge Jun 19, 2025, 8:59 PM

#

tender cradle 👋 here's some brief information about it: https://github.com/dagger/dagger/blob...

I have the Dagger engine running in my cluster and I am using the _EXPERIMENTAL_DAGGER_RUNNER_HOST env var on my workstation to run the Dagger CLI. So far so good...

Now I would like to create a pod that can run a Dagger AI agent (aka a Dagger module written in Go) but I am not sure how to do that. I am thinking that for the POD's image I would create a custom image that has the Dagger CLI and Kubeclt installed as well as a copy of my module sourc, then I would call dagger run as the pod's startup command. And since the pod is running in the same cluster as the "agent" I would be sure to set _EXPERIMENTAL_DAGGER_RUNNER_HOST environment variable, also this is why I think kubectl is needed.

Does that sound right? If so is there an existing container image for this or do I have to create it?

tender cradle Jun 19, 2025, 11:13 PM

#

The helm chart puts the Dagger engine Unix socket in a volume by default. If you create a pod with the Dagger CLI and set the _EXPERIMENTAL_DAGGER_RUNNER_HOST to unix:///$sockpath that will make it work

#

So no kubectl is needed

#

@obsidian forge

obsidian forge Jun 19, 2025, 11:55 PM

#

Awesome, I will give that a try and report back, thanks

obsidian forge Jun 20, 2025, 12:29 AM

#

Quick followup question. I plan to run the Dagger CLI in a separate POD from the Engine so I am unclear where it will get the value for the env var $socketpath for "unix:///$sockpath", what am I missing? Also I don't I need to mount a shared volume to use a socket?

In the engine pod I see the following but no volume

  containers:
  - args:
    - --addr
    - tcp://0.0.0.0:8080
    - --addr
    - unix:///run/dagger/engine.sock

#

I found the hostPath that is set by the Engine. I guess I need to make sure my pod is on the same host and mount that same host path. I will give that a try untill I hear otherwise...

obsidian forge Jun 20, 2025, 1:25 AM

#

Good news that worked, I was able to connect the Dagger CLI to the Dagger Enginve via a socket. Here is my Deployment manifest if others are interested:

As a test I shelled into the pod, manually installed the Dagger CLI and then ran the dagger shell, it works!

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: sre-ai-agent
  name: sre-ai-agent
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sre-ai-agent
  template:
    metadata:
      labels:
        app: sre-ai-agent
    spec:
      serviceAccountName: sre-ai-agent
      containers:
      - name: main
        image: ubuntu:latest 
        command:
        - /bin/bash
        - -c
        - | 
          while true; do 
            date; echo 'ZZZzzz..'; sleep 1m; 
          done
        env:
        - name: "_EXPERIMENTAL_DAGGER_RUNNER_HOST"
          value: "unix:///run/dagger/engine.sock"
        volumeMounts:
        - name: dagger-engine-socket
          mountPath: /run/dagger
      volumes:
        - name: dagger-engine-socket
          hostPath:
            path: /run/dagger-dagger
            type: ""

#How to run a dagger pipeline written in the GO SDK in a K8s Pod?