#File export issues

1 messages · Page 1 of 1 (latest)

night acorn
#

I have a pipeline that runs terraform, which in turn modifies a state file. I want to copy this state file back to the host.

This seems to work, but with repeated calls I'm seeing old or incorrect copies of the state file on the host after run.

I've come up with the following repro:

package main

import (
    "context"
    "fmt"
    "os"
    "time"

    "dagger.io/dagger"
)

type Dagger struct {
    client *dagger.Client
    //cache  *dagger.CacheVolume
    ctx context.Context
}

func toolbox(d Dagger) *dagger.Container {
  return d.client.Container().From("ubuntu:latest").
        WithMountedDirectory("/work", d.client.Host().Directory(".")).
        WithWorkdir("/work").
        WithEnvVariable("CACHEBUST", time.Now().String())
}

func run(d Dagger, cmd []string) error {
    tm := toolbox(d)

    tm = tm.WithExec(cmd)
    out, err := tm.Stdout(d.ctx)
    if err != nil {
        return err
    }
    fmt.Println(out)

    output := tm.Directory(".")
    _, err = output.Export(d.ctx, ".")
    if err != nil {
        return err
    }

    return nil
}

func main() {
    // init dagger client
    ctx := context.Background()
    client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stdout))
    if err != nil {
        panic(err)
    }
    defer client.Close()

    d := Dagger{
        client: client,
        ctx:    ctx,
    }

    run(d, []string{"sh", "-c", `echo xxx1 > hello.txt`})
    run(d, []string{"sh", "-c", `echo xxx2 >> hello.txt`})
    run(d, []string{"sh", "-c", `echo xxx3 >> hello.txt`})
}

So after running, this I'd expect hello.txt to contain:

xxx1
xxx2
xxx3

Instead, it only seems to contain xxx3, despite me copying the file back to the host using Export() after each call to run(). Any tips would be great.

sly wind
#

Hey 👋 What's happening here is you're using a fresh container on each run(). So what you're left with at the end is the result of the final run. What you probably want to do is pass the same container through to each operation and call Export once at the end

fading prism
#

I had the exact same issue.
Solved by using the same container (just like @sly wind suggestion).

Still, I think that the current behavior is a bit non-trivial.

tulip jewel
night acorn
#

Thanks all, I'll try the cache volume and report back. What really stumped me was, I knew I was using a new container each time, and I'd rather do that actually. But I'm exporting the working dir back to the host at the end of each call to Run()... so why doesn't this then get mounted into the next container?

astral prawn
#

@night acorn I think it's because this all runs within the same session. Dagger doesn't re-upload the same path repeatedly within a single dagger.Connect.