Make only a portion of a directory available in the target container

[honest gyro]

I'm using the go sdk, with this xs/.dagger/main.go

package main

import (
    "context"
    "dagger/xs/internal/dagger"
)

type Xs struct{}


func (m *Xs) UploadRust(ctx context.Context, dir *dagger.Directory) (string, error) {
    cleanDir := dag.Directory().
        WithDirectory("src", dir.Directory("src")).
        WithFile("Cargo.toml", dir.File("Cargo.toml"))
    
    return dag.Container().
        From("alpine:latest").
        WithMountedDirectory("/mnt", cleanDir).
        WithWorkdir("/mnt").
        WithExec([]string{"find", "."}).
        Stdout(ctx)
}

Rough details of the xs directory:

~/s/03d..x3n/xs $ ^du -sh  .
 11G    .
~/s/03d..x3n/xs $ ^du -sh ./src
416K    ./src
~/s/03d..x3n/xs $ ^find foo
foo
foo/Cargo.toml
foo/Cargo.boo
foo/src
foo/src/main.rx

When I pass "foo" as the directory, it runs quite quickly and I only see ./src and Cargo.toml

$ dagger develop ; dagger -c 'upload-rust "foo"'
▶ connect 2.3s
...                                                                   ./src                                                                  ./src/main.rx
./Cargo.toml                                                           

When use "." (ie the xs directory), I've not seen it complete. It runs for minutes on the upload step before I interupt it. My hunch is it's trying to upload all 11Gb?

dagger develop ; dagger -c 'upload-rust "."'

[honest gyro]

I might need to be using this: https://docs.dagger.io/api/filters/?sdk=go#pre-call-filtering

This works well:

package main

import (
    "context"
    "dagger/xs/internal/dagger"
)

type Xs struct{}

func (m *Xs) UploadRust(
    ctx context.Context,
    // +ignore=["**", "!Cargo.toml", "!Cargo.lock", "!src/**"]
    dir *dagger.Directory) *dagger.Container {


    return dag.Container().
        From("alpine:latest").
        WithMountedDirectory("/mnt", dir).
        WithWorkdir("/mnt")
}

[molten forge]

yep, this is the way 🙌

[honest gyro]

it'd just be nice to be able to specify the includes and exclusions at call time

[molten forge]

that's a fair request. Can't recall if this has been requested before. Probably @neon galleon knows

[neon galleon]

You can, with shell:

dagger -c 'upload-rust $(host | directory . --include=Cargo.toml,Cargo.lock,"src/**") | terminal'

[honest gyro]

it may also be possible with this: #general message -- or if modules exposed dag.Host: via the same mechansim as the shell example

[molten forge]

modules not exposing dag.Host is intentional since it'd otherwise violate their default sandboxing primitives

[honest gyro]

it sounds like you can escape things though, by using: dag.CurrentModule().XXX() ? (i've still to try that)

[molten forge]

that shouldn't escape the sandbox. CurrentModule gives you contextual functions for that module, but you can't access things from the host, etc