#can you mount host dir to golang module container to be accessed by module code?

1 messages · Page 1 of 1 (latest)

potent heart
#

from what i understand from this doc about code execution

When implementing Dagger Functions, you are free to write arbitrary code that will execute inside the Dagger Module's container.

we can utilize golang to do things inside the container. But what I can't figure out is, how can i mount the folder on my host (for example: code checked out with a gihub action in previous step) to perform actions inside the reposiotry folder.
eg: here I'm trying to load a git repository using pkg "github.com/go-git/go-git/v5"

func (m *CI) Apply(
    ctx context.Context, 
    src *dagger.Directory,
    ) error{
    
    o := &Options{
        Dir:         ".",
    }
    err := o.Run()
    if err != nil {
        log.Logger().Errorf("failed to run: %v", err)
        return err
    }
    return nil
}

// Options the options for the command
type Options struct {
    Dir           string
    ...
}

func (o *Options) Run() error {

    if o.repo == nil {
        o.repo, err = git.PlainOpen(o.Dir)
        if err != nil {
            return errors.Wrapf(err, "failed to open git repository")
        }
    }
}```
Execution Environment | Dagger

Containerization

lament raptor
#

@potent heart Once you're executing the Dagger function written in Go, you're inside of a Go runtime container sadboxed from the host (e.g. GHA Ubunutu runner). So the git.PlainOpen(o.Dir) is going to try to open a directory inside of the Go runtime container, not from the host.

potent heart
#

is there an option for me to mount the dir to the container?

limpid lichen
#

You'll need to pass any host resource into the dagger call as an argument to your function. For the above example, dagger call apply --src ./src

potent heart
#

can you run go functions inside container (or the module container)?

potent heart
#

I'm running the code with dagger call apply --src ./src but i can't figure out how to mount the dir to the container that runs the modules code.

potent heart
#

can you mount host dir to golang module container to be accessed by module code?

lament raptor
potent heart
# lament raptor If we take a step back, what would you like to do with the `dagger.Directory` of...

certainly, src variable holds the current git repository (not an application code, but a complex configuration repo). git.PlainOpen() opens the config repo and does some checks on the commit history and based of certain criteria (k8s cluster configuration) does some operations like create new files and update files which will be merged back to the original branch.

I feel like if we have the ability to do things like mount dir, add secrets and env vars (yet to find out which of these are actually configurable in module container) we could use the sdk language (in this case go) to do things like make authenticated call using client sdks (for gcp, aws, k8s) using hosts configurations.
in this case, I also want to mount ~/.kube dir from host to call k8s cluster.

lament raptor
#

I see. Yes, today you can pass secrets and env vars (and directories, etc) in as function params (many improvements in the works: https://github.com/dagger/dagger/issues/7647, https://github.com/dagger/dagger/issues/6112). You're passing in a snapshot/copy and not establishing a bi-directional bind mount.

So you can take in a directory (including a remote git repo ref), alter it, and return it from your function where you can use the CLI to write the directory to the host.

potent heart
# lament raptor I see. Yes, today you can pass secrets and env vars (and directories, etc) in as...

I get that we can pass these when we create a container, but what puzzles me is if we can pass these secrets and env vars (and directories, etc) to the container that runs module code (from my understanding module code gets it own runtime container). IF possible how?
for example as i mentioned above, I need to connect to the kubenetes cluster with module code. to create that clientset for the kubernetes/client-go sdk I need to call the helper function below (which is part of the module code).
util.go

import (
    "context"
    "fmt"
    "k8s.io/client-go/kubernetes"
    "k8s.io/client-go/rest"
    "k8s.io/client-go/tools/clientcmd"
    "os"
    "path/filepath"
)

func createClient(configDir string) *kubernetes.Clientset{
    var config *rest.Config
    var err error

    // Use the default config file if available
    if configDir == ""{
    configDir = filepath.Join(os.Getenv("HOME"), ".kube", "config")
    }
    if _, err := os.Stat(configDir); err == nil {
        // Load configuration from the kubeconfig file
        config, err = clientcmd.BuildConfigFromFlags("", kubeconfig)
        if err != nil {
            fmt.Printf("Error loading kubeconfig: %v\n", err)
            return
        }
    }

    // Create a new clientset
    clientset, err := kubernetes.NewForConfig(config)
    if err != nil {
        fmt.Printf("Error creating clientset: %v\n", err)
        return
    }
    return clientset
}

this will fail to authenticate unless we mount the ~/.kube dir (with credentials already created to the desired cluster) in my local machine to the ~/.kube in module running container.

limpid lichen
#

I have a few use cases where I need to mount multiple aws profile credentials, ssh keys and secret env vars. This is what works for me:

// pass the dir to your dagger call <func> --awsDir ~/.aws
WithMountedDirectory("/root/.aws", awsDir)

// pass the dir to your dagger call <func> --sshDir ~/.ssh
WithMountedDirectory("/root/.ssh", sshDir)

// set the env var GITHUB_TOKEN on your shell
// pass the var to your dagger call <func> --ghToken env:GITHUB_TOKEN
WithSecretVariable("GITHUB_TOKEN", ghToken)
#

So you could do the same with a kubeconfig, and then after mounting it, exec the application that requires it.

lament raptor
#

@alpine spire do you get the impression that the OP is trying to exec golang code in the golang runtime container? That's what it seems like to me.

#

I guess the approach in a monorepo could be to have the project that you want to do build/test in the main repo, then a dagger directory for the module, and maybe another directory for some special golang processing code that is expecting to be run on a "host".

Then in the Dagger module you'd create a golang container called foo, and pull in dagger.Directorys, for example the main repo (excluding uneeded stuff) as src and the special golang code as helper.

You could write your native non-Dagger golang helper to look for the src directory in the foo container, modify it, and emit an output directory. Then the Dagger module could extract the output dir and return it as a dagger.Directory.

lament raptor
#

But at that stage, you may need to put that helper logic in a Dagger Module to work properly, I'd think.

lament raptor
#

Here's a working example @potent heart. Might not be what you inteded 😆 You can just run the following if you have dagger. The module and directories are pulled dynamically from github.

dagger -m github.com/jpadams/test-go-helper \
  call foo \
    --src https://github.com/jpadams/test-go-helper\#main:src \
    --helper https://github.com/jpadams/test-go-helper\#main:helper \
  directory --path "/output" \
  file --path LICENSE \
  contents
#

Inside a golang container, it takes in a directory of src, processes it with a golang helper program in helper and outputs to /output. Then with the dagger cli, I pull out the directory and the file I'm interested in and look at its contents. In this case my helper takes the files in src and reverses the text and dumps the in output 🙂

potent heart
lament raptor
#

You can use the regular old Dagger API of course, but here we're using a golang program like it's a helper script to process some things loaded into a container like a little "host" 🙂

potent heart
#

thank you @lament raptor I will try this out. Are there any plans to allow these feature on the runtime container?

lament raptor
#

Here there is not strict sandboxing to the host and you can't share modules across languages, etc

#

There is an issue about getting the best of both worlds.

#

Please add your input 👆 if appropriate 🙂

potent heart
#

also just out of curiosity. I tried the following code to install a copy of the current module in the container.

type ExampleProjectClusterConfigurations struct{}

// Returns a container that echoes whatever string argument is provided
func (m *ExampleProjectClusterConfigurations) Apply(
    ctx context.Context, 
    src *dagger.Directory,
    // +optional
    pullRequest bool) (*dagger.Container){

    s := dag.CurrentModule().Source()


    portStr, ok := os.LookupEnv("DAGGER_SESSION_PORT")
    if !ok {
        return nil
    }
    sessionTokenStr, ok := os.LookupEnv("DAGGER_SESSION_TOKEN")
    if !ok {
        return nil
    }
// build app
    builder := dag.Container().
        From("golang:latest").
        WithEnvVariable("DAGGER_SESSION_PORT", portStr).
        WithEnvVariable("DAGGER_SESSION_TOKEN", sessionTokenStr).
        WithDirectory("/src", s).
        WithWorkdir("/src").
        WithEnvVariable("CGO_ENABLED", "0").
        WithExec([]string{"go", "build", "-o", "runtime"}).Terminal()

    return builder
}

getting
dagger / $ /runtime --help
get parent name: Post "http://127.0.0.1:35807/query": dial tcp 127.0.0.1:35807: connect: connection refused

lament raptor
lament raptor
# potent heart also just out of curiosity. I tried the following code to install a copy of the ...

You want dagger in dagger 🙂 So you need to allow nesting 🙂

// build app
    builder := dag.Container().
        From("golang:latest").
        WithEnvVariable("DAGGER_SESSION_PORT", portStr).
        WithEnvVariable("DAGGER_SESSION_TOKEN", sessionTokenStr).
        WithDirectory("/src", s).
        WithWorkdir("/src").
        WithEnvVariable("CGO_ENABLED", "0").
        WithExec([]string{"go", "build", "-o", "runtime"}).
                WithExec([]string{"./runtime", "--help"}, dagger.ContainerWithExecOpts{ExperimentalPrivilegedNesting: true})

    return builder
}
#

In general, though, I don't know that there are any guarantees about running compiled go modules. Again, I'd go to the previous SDK/client model for that.

alpine spire