Porting go sdk example tests to python

[steep blade]

With workdir (timeout):

async def test_host_workdir():
    async with dagger.Connection() as client:
        readme = await client.host().workdir().file("README.md").contents()

        lines = readme.value.strip().split("\n")

        assert lines[0] == "# Dagger Python SDK"

Not sure why this is hanging... shouldn't take longer than 10s.
\cc @dapper maple

I'm going to try and list the files.

Ok, I'm seeing it's getting canceled on buildkit's side close to 10s. Trying to play with higher timeouts.

[digital estuary]

By any chance, to you have a bunch of stuff locally?

Like ... node_modules?

but for python

[steep blade]

Ah, here's where I tripped up 😇 Since I don't have provisioning working I have a cloak dev running in the terminal. So it's running from dagger's root, not the sdk's root.

Now it's working 😅

[digital estuary]

😄

yeah I had the same happening to me

I made changes to the docs and had a giant website/node_modules

you can exclude it using the exclude field in workdir?

exclude=["**/node_modules", <python equivalent>]

[steep blade]

I just ran it with ❯ go run ./cmd/cloak dev --workdir sdk/python

Which is what the provisioner would do.

[digital estuary]

@dapper maple had the idea of supporting .dockerignore / .gitignore files to avoid those problems

[steep blade]

So now I have to fix the loop on exec. By the graphql error complaining that I need to alias them, makes me think I may have a mutability issue. Gonna check.

[dapper maple]

yeah i think we could support an ignorefile: String! param assuming they're all roughly the same syntax. it's not too difficult

[steep blade]

So, turns out there was only one bug: mutability on the dynamic query builder. Fixed.

[digital estuary]

Awesome 🙂 There’s a few tests related to that in querybuilder_test.go

Mutability hit me a few times

[raw summit]

related: https://github.com/dagger/dagger/issues/3736#issuecomment-1310096027

Take into account that supporting .gitignore is not trivial since it has some quirks like files already being tracked by git not being affected.

cc @urban canyon

[dapper maple]

great point, I was worried about the syntax being slightly different (so a generic ignorefile: wouldn't be compatible between gitignore/dockerignore) but this is clearly even harder due to git semantics. i agree that it seems better to solve that using the existing API capabilities how you suggested (running a git command to figure out the ignored files and then exclude:ing them). or an extension could maybe do this someday

[urban canyon]

Currently trying to rewrite the code segment using exclude and getting this error

func WithoutIgnored(client *dagger.Client, ctx context.Context, dir *dagger.Directory) (_ *dagger.Directory, _ error) {
    gitcontainer := client.Container().
        From("alpine/git").
        WithMountedDirectory(".", dir).
        Exec(dagger.ContainerExecOpts{
            Args: []string{"ls-files", "--ignored", "--exclude-standard", "--others", "--directory", "--no-empty-directory"},
        })

    exitcode, _ := gitcontainer.ExitCode(ctx)
    if exitcode != 0 {
        return nil, errors.New("git exited with non-zero exit code")
    }

    contents, err := gitcontainer.Stdout().Contents(ctx)
    if err != nil {
        return nil, err
    }

    ignoredFiles := strings.Split(contents, "\n")
    ignoredFiles = ignoredFiles[:len(ignoredFiles)-1]

    for _, line := range ignoredFiles {
        fmt.Printf("LINE: %s\n", line)
    }

    dir = client.Directory().WithDirectory(".", dir, dagger.DirectoryWithDirectoryOpts{
        Exclude: ignoredFiles,
    })

    return dir, nil
}

...
#4 [internal] load metadata for docker.io/library/golang:1.18.2-alpine
LINE: .idea/
LINE: node_modules/
LINE: test.txt
#4 DONE 0.7s

#1 resolve image config for docker.io/library/alpine:latest
#1 DONE 1.2s

#4 [internal] load metadata for docker.io/library/golang:1.18.2-alpine
#4 DONE 0.8s

#20 copy / /
#20 ERROR: failed to copy file info: failed to chown /var/lib/buildkit/runc-overlayfs/snapshots/snapshots/596/fs/.idea: lchown /var/lib/buildkit/runc-overlayfs/snapshots/snapshots/596/fs/.idea: no such file or directory

------
 > copy / /:
------

It seems to work if test.txt is the only ignored file, but both folders seem to break something here.

This seems to work

    dir = client.Host().Workdir(dagger.HostWorkdirOpts{
        Exclude: ignoredFiles,
    })

[raw summit]

so it's finally working Benjamin?