#Dagger tries to upload everything in workspace by default

1 messages · Page 1 of 1 (latest)

mild bramble
#

Hi I've created a dagger project as a golang package inside of our monorepo. The monorepo has a go workspace but we do not want to include the dagger package in the workspace. For other packages we have used GOWORK=off to keep them excluded.

We have a simple dagger function that just takes a string input, and doesn't need any directory/fs input. But by default it is still trying to upload/import the whole workspace which is time consuming.

dagger call container-echo --string-arg hi

How can I get it to stop trying to upload everything in the workspace? It's not part of the workspace.

The module is defined in /treasury/test, but it keeps trying to upload /treasury making it take a long time.

#

I've tried this in a dir outside of the repo, e.g. /test and it works well.

rain cairn
#

I think I know what's happening. The default module include and exclude patterns are probably biting you

mild bramble
rain cairn
#

ok, yes I see. Could you try adding this to your dagger.json and trying again?

  "exclude": [
    "**/*",
    "!test/dagger/**/*",
    "!test/dagger.json"
  ],
mild bramble
#

like this?

  "name": "test",
  "sdk": "go",
  "source": "dagger",
  "engineVersion": "v0.13.0",
  "exclude": [
    "**/*",
    "!test/dagger/**/*",
    "!test/dagger.json"
  ]
}```
rain cairn
#

yes please

mild bramble
#

I tried

    "**/*",
    "!test/dagger/**/*",
    "!test/dagger.json"
  ]```
and ```  "exclude": [
    "**/*",
    "!test"
  ]```
rain cairn
#

you're running your dagger call from the /test folder, right?

mild bramble
#

yes

#

/Users/conorpatrick/treasury/test

rain cairn
#

try this instead please:

 "exclude": [
    "**/*",
    "!dagger/**/*",
    "!dagger.json"
  ]
mild bramble
rain cairn
#

oh, there's something odd here

#

can you please move your dagger.json to the top of your repo

#

and change the source field to test/dagger?

mild bramble
#

sure

rain cairn
#

and then chane your exclude again back to:

  "exclude": [
    "**/*",
    "!test/dagger/**/*",
    "!dagger.json"
  ]

edit: just edited this

#

you'll then have to run the dagger call at the root of your repo

mild bramble
#

this works thank you

  "name": "test",
  "sdk": "go",
  "source": "test/dagger",
  "engineVersion": "v0.13.0",
  "exclude": [
    "**/*",
    "!test/dagger/**/*",
    "!dagger.json"
  ]
}```

https://dagger.cloud/conor/traces/d57bc5c912ef86a569240d4fd7a4a1c0
rain cairn
#

now caching will be better since it won't get invalidated with any file that you change in your monorepo

#

it's only looking at the dagger.json and the test/dagger files

#

I'm opening an issue for this

mild bramble
#

nice. it would be helpful to see a basic reference for the dagger.json or examples, i haven't found any on the docs

rain cairn
half tangle
#

noted!

oak imp
#

@rain cairn Won’t they also need 0.13.1 if they want the exclude order to be consistent?

mild bramble
#

i am using 0.13.1, the version in the dagger.json was stale

sage pebble
rain cairn
sage pebble
#

Once that is resolved, will the manual include/exclude still be needed in this case?

#

(ie. once all SDKs stop uploading all go files in the repo when loading a module)

rain cairn
#

which IMO it makes more sense as I described in the issue I opened

sage pebble
#

But by default, the SDK will upload 1) the entire module's source directory (eg. .dagger or . or other), and exclude everything else from the module root and the rest of the repo right?

#

In which case, you only need include if you specifically need to import a go file in the root of your module, and not in the source of your module?

#

(and also not outside your module because that will be out of scope for include/exclude)

rain cairn
sage pebble
#

I'm guessing very unlikely scenario, because if you need to use go files from your own module root, probably you would just use --source=.?

rain cairn
rain cairn
sage pebble
#

right

worldly swallow
#

yeah, we have had to be very specific with include/exclude in our mega repo to avoid these cache invalidations. We still maintain only a single go.mod for 20+ go builds intermixed with python and typescript services