#Host resources by shykes · Pull Request ...
1 messages · Page 1 of 1 (latest)
It's for function arguments that have the // +ignore pragma, so if you pass a directory through the CLI it filters it using those ignore patterns.
When the CLI tries to fetch the value from a flag for a function argument, it passes on the function argument's definition (introspection) in order to get those ignore patterns.
ah makes sense!
but does that mean the pre-call filtering (+ignore) only happens from the CLI? What about from SDKs?
in other words the engine doesn't enforce it?
It happens in multiple places. I think from SDK it's been implemented as well (in the engine), but it brought an issue where directories from the CLI would be filtered twice, I don't remember if Tom worked around that, I'd have to check. The CLI has it too because it puts a Directory object in the argument for the query builder, which by then is agnostic to whether it was filtered or not.
So if I call a function manually, say with dagger query or dagger listen + curl, and I naively pass a directory as argument to a function. if that function has pre-call filtering specified, the filtering will not be applied? ie. the client is responsible for introspecting and enforcing pre-call filtering?
It should get filtered in the engine, but you'll see a performance hit because I think the whole directory is going to be uploaded to the engine, but then filtered before getting saved as the argument value.
The CLI uses Host.directory, while the engine doesn't. It gets a normal Directory.
OK. I thought somehow the engine avoided this with session attachables (ie. the engine only asks the client to upload matching files)
The major filesync refactor came after, so I'm not sure if this could now be simplified or if the problem remains.
I see. Ok it's much clearer now, thanks
For my immediate task, I'll just add include / exclude arguments to HostResource.asDirectory, so that the CLI can pass them in
Yep 👍
No, not really 🙂
Tracking down the double filtering thing, conclusion here is it's not worth it to avoid: https://github.com/dagger/dagger/pull/9955#pullrequestreview-2720330047
Fixes #9490
Signed-off-by: Tom Chauveau tom@epitech.eu
It's applied here: https://github.com/dagger/dagger/blob/main/core/modfunc.go#L163-L163