#Hello, my main module sources are in the

1 messages · Page 1 of 1 (latest)

vale sapphire
#

I am worried this got burried in the noise today. yann, are you able to share a snippet of what you have so far?

twin citrus
#

Sure! Here is how I was able to access host context inside a container with Dagger 0.9.6: https://github.com/yann-soubeyrand/dagger-test/blob/18037e58de00f91df8918966cc7438b724224d9e/ci/main.go#L10-L22. Is it mandatory now to pass the docs directory as argument (for example here https://github.com/yann-soubeyrand/dagger-test/blob/18037e58de00f91df8918966cc7438b724224d9e/ci/main.go#L32) even if it’s in the same directory as the dagger.json file?
Also, here is how I accessed host context directly from my module code: https://github.com/yann-soubeyrand/dagger-test/blob/18037e58de00f91df8918966cc7438b724224d9e/ci/main.go#L28 and https://github.com/yann-soubeyrand/dagger-test/blob/18037e58de00f91df8918966cc7438b724224d9e/ci/vector.go#L23). I found that I can now access it by passing the host directory as arguments and then exporting it inside the current module workdir, but it doesn’t seem so intuitive at first. Is there a better solution?

granite rock
# twin citrus Sure! Here is how I was able to access host context inside a container with Dagg...

One thing that's still possible without needing any explicit passing of directories is access to the "source dir" of the module (which is where the main.go file is, in the case of the Go SDK).

In that case, dag.CurrentModule().Source() will return that as a Directory. If you need to files to be available locally for your module code, that will indeed still require doing an export. The reasoning is that we want each module to start out with a blank workdir that they can populate as they need rather than forcing it to always be the module's source dir.

There's a brief overview of all this in the docs here https://docs.dagger.io/developer-guide/overview/942201/execution-environment, but it could definitely use some fleshing out and more examples I think

clever fulcrum
#

I am in the same boat. So far my dagger builds always lived in a build directory and in root is where all my services are. So far I could tell which dir from host I like to include like "client.Host().Directory("."). With functions I pass them as args now, but I failed to provide default values for it, which would be kind of the same as my old setup. Is it possible to set default paths in go sdk?

twin citrus
#

@granite rock thanks, that confirms what I understood. Having porcelain commands (as discussed on a GitHub issue I read) to be able to access the context (not sure if this is the correct term to describe the repository where the dagger.json file is rooted) from the module code would be great. Also, having a way for a module to watch changes to the context directory would be awesome (for example, I’d love to be able to start hugo serve from within my Hugo module so that it keeps serving up to date web site while I’m editing its sources.

twin citrus
#

I’m surprised, though, that the module working directory (/scratch) isn’t kept between the constructor and the function. I mean, I’m populating the workdir inside my module constructor and I’d like to use it inside my module functions, but the content is not there anymore.

buoyant vigil
#

I'm trying to do something related here too.

I think the correct way to do this is to pass in a Directory as a function argument. That makes it available to the module. Great. But now I have some code in the module that wants to read it off of disk. Before what I did was simply mount the directory. But if I'm already in the function and not using dag.Container(), I don't call WithMountedDirectory. How do I mount the directory I passed in to the module container so that the library I'm using can read it off the container's disk?

#

I believe from above:

  1. Pass in Directory to function. This makes the host's directory available to the function
  2. Call Directory.Export(). This will write to the module's container filesystem
  3. Proceed as normal inside my function.

correct?

vale sapphire
#

I think that right @buoyant vigil but one thing to note is that when I ran into a similar issue I had to make one mental shift before things made sense.

I wanted to export the directory so I could see things working, but for the purpose of my actual pipeline I only needed to have a function return a directory type that I then passed along into another function. So unless there is a specific reason that you need to export in the middle of your pipeline, i'd suggest writing the function in a way that returns the thing you need and trust that it will work as expected while its running even if you cant observe the files moving around outside of the pipeline.

For example this is what my hugo function looks like

// build hugo site and return directory
func (m *Ci) hugo(ctx context.Context, ci *Ci) *Directory {
        // get ref to local project
        path := "/src"
        src := ci.Dir
        outPath := "/src/public"

        return m.Ctr.
                WithMountedDirectory(path, src).
                WithWorkdir(path).
                WithExec([]string{"hugo"}).
                Directory(outPath)
}

Then later, I use the function like this

        // build hugo site
        site := m.hugo(ctx, ci)

        // deploy with git
        output, err := m.deploy(ctx, ci, site)

So going back to your example, perhaps there is a helper function that returns the directory which you can then carry along with you to other parts of your pipeline.

buoyant vigil
#

In my case the client library wants the directory written to disk (load default ~/.aws folder to authenticate).

Here is my working example, a very simple module that takes in a bucket as input and lists the contents of that s3 bucket:

package main

import (
    "context"
    "fmt"
    "log"
    "strings"
    "time"

    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/s3"
)

type S3 struct {
}

// ListBucketContent lists the content of the S3 bucket
func (m *S3) ListBucketContent(
    ctx context.Context,
    bucket string,
    creds *Directory,
    // +optional
    // optional profile to execute the ListBuckets command with
    profile string,

) (string, error) {
    creds.Export(ctx, "/root/.aws")
    var cfg aws.Config
    var err error
    if profile != "" {
        cfg, err = config.LoadDefaultConfig(context.TODO(), config.WithSharedConfigProfile(profile))
    } else {
        cfg, err = config.LoadDefaultConfig(context.TODO())
    }
    if err != nil {
        log.Fatalf("Unable to load SDK config, %v", err)
    }
    client := s3.NewFromConfig(cfg)
    input := &s3.ListObjectsV2Input{
        Bucket: aws.String(bucket),
    }
    paginator := s3.NewListObjectsV2Paginator(client, input)
    var sb strings.Builder // Use strings.Builder to efficiently build the output string

    for paginator.HasMorePages() {
        output, err := paginator.NextPage(ctx) // Use the passed context instead of context.TODO()
        if err != nil {
            return "", fmt.Errorf("error listing bucket content: %w", err)
        }
        for _, item := range output.Contents {
            sb.WriteString(fmt.Sprintf("Name: %s, Last modified: %s, Size: %d, Storage class: %s\n",
                *item.Key, item.LastModified.Format(time.RFC3339), *item.Size, string(item.StorageClass)))
        }
    }

    return sb.String(), nil // Return the aggregated string
}
#

Like @twin citrus though, I would also like to load my directory into /root/.aws via the constructor instead of inside of the function. That way I can use that same staged directory in other functions in the same module. It sounds like that is not currently possible?