[SOLVED] Need help with ctr.directory("...").export("...")

[obtuse oasis]

Can someone tell me how I can export a directory on my host? The following code "seems" to work (i.e. no exception and I see the EXPORTING ... and EXPORTED ... message in my console) but didn't.

if ctr:
    print("EXPORTING..................")
    try:
        ctr.directory("/app/src").export("/app/tmp")
        print("EXPORTED..................")
    except Exception as exception:
        print(f"ERROR - EXPORT {exception}")

I'm supposed to have a new /app/tmp folder on my disk but this is not the case.

Thanks

[still schooner]

Are you using a Dagger module?

[still schooner]

Missing an await by the way.

[obtuse oasis]

Thanks Helder!

I feel stupid about your first question: if running dagger call -m xxxx is the answer; no; it's not a module. I've run dagger init a few weeks ago and build a huge Python codebase.

For the missing await, no, not missing because my Python function isn't a async one. Am I forced to convert my function to an async one?

[still schooner]

Yes, only async is supported.

[obtuse oasis]

Unfortunatelly seems not working. I don't have any errors but the /app/tmp folder isn't created.

[still schooner]

It's not enough to add async, you need to run an event loop. How are you making the connection to the Dagger API?

[obtuse oasis]

Did you've a working and updated Python example somewhere ?

[still schooner]

https://docs.dagger.io/api/sdk#custom-applications

[obtuse oasis]

My current code is working i.e. I've created something like 30 (or more) async functions to run python ruff, black, prospector, ... php-cs-fixer, ... phpunit, bats (bash units tests), ... and it's works (thanks Dagger team).

Right now, I'm trying to retrieve the reformated content (the one after Python ruff f.i.) and see if I can export it on my disk

[still schooner]

Do you have Dagger Cloud?

Are you running with or without dagger run?

[obtuse oasis]

No, sorry. I'm refactoring our internal Gitlab CI pipelines and perhaps in a next phase I'll ask my company to look for Dagger Cloud. Right now, I'm running everything on my host (and our self hosted Gitlab)

[obtuse oasis]

dagger call

[still schooner]

Dagger Cloud is for visualization, it's not for running your pipelines. I was asking because it would have to have a trace.

[obtuse oasis]

My current command is dagger call --pipeline-file=/app/.pipeline --source-directory=/app/src --config-directory=/app/src/.config --run-step RUN_FORMAT_PYTHON_RUFF format

[still schooner]

Oh, it seemed like you weren't using modules, from #1342075271884570625 message:

it's not a module

Then that's different.

[obtuse oasis]

Not a module, no. It was also my thought.

[still schooner]

Well, if you're using dagger call then it's a module. What makes you think it isn't?

[obtuse oasis]

If it can help, I've wrote a tutorial on my blog (https://www.avonture.be/blog/dagger-python). I was so happy to discover Dagger.

[still schooner]

I know what's wrong, just looking for the docs for you.

Short story is that since it's a module, it runs in its own container, not in your host. So you're actually exporting into that container and not your machine. To properly export using modules, you should need to create a Dagger function that returns the directory and then export from the CLI. For example, let's say your format function returns a dagger.Directory:

dagger call --pipeline-file=/app/.pipeline --source-directory=/app/src --config-directory=/app/src/.config --run-step RUN_FORMAT_PYTHON_RUFF format export --path=/app/tmp

• https://docs.dagger.io/features/modules
• https://docs.dagger.io/api/custom-functions
• https://docs.dagger.io/api/return-values#directory-return-values

[still schooner]

Still curious, why you thought it wasn't a module?

[obtuse oasis]

I thought dagger call -m xxxx ==> module. In my case I don't use -m so I thought it was not a module (sorry, I feel stupid)

[still schooner]

Not a problem. There's a default for -m, which is to look for a module in the current directory. But dagger call doesn't work without one.

[obtuse oasis]

For the -o flag, I've used it but it for saving the console no ?

[still schooner]

To be clear, dagger call is for calling module functions. dagger core calls core API functions (no module).

The -o <path> option is a convenience that calls export --path=<path>. Same thing.

Oh, sorry.. I'll amend the command

[obtuse oasis]

Ok, I'll continue to investigate the final export -o /app/tmp . I've well used it this morning but got error like export is not a valid flag or something like that; will check again.

[still schooner]

It's either -o /app/tmp or export --path=/app/tmp.

[obtuse oasis]

The correct message is unknown command "export" for "dagger call format"

[still schooner]

That's because format doesn't return a dagger.Directory I bet.

[obtuse oasis]

I've fired dagger call --pipeline-file=/app/.pipeline --source-directory=/app/src --config-directory=/app/src/.config --run-step RUN_FORMAT_PYTHON_RUFF format export --path=/app/tmp

[obtuse oasis]

Oh ? Ok, I'll investigate this... Good direction

[still schooner]

You can do more with that dagger.Directory result:

• .... format entries → lists dirs/files in that result
• ... format terminal → opens a shell in a container (default is alpine) so you can explore it

[obtuse oasis]

My format function was returning str indeed. I'll need to go deeper in my code and change a few classes before being able to return a Directory object;

[still schooner]

You can also turn that str into a dagger.File easily. It also supports export.

[obtuse oasis]

I'm returning strbecause, when fired on the GitLab CI, I need to return a string (the diff coming from the tool). I'm "trying" to make possible to run the function on my host machine and, in that case, it would be nice to get the diff but, too, the changes so my code base is reformated and I can push it again.

I really appreciate your help Helder, I'll jump in my code and do some changes. Thanks!

[still schooner]

return dag.directory().with_new_file("result.txt", result).file("result.txt")

[obtuse oasis]

TREMENDOUS !!! Yes ! That was the root cause. My function was returning a string and I need to return a Directory to achieve my objective.

dagger call --pipeline-file=/app/.pipeline --source-directory=/app/src --config-directory=/app/src/.config --run-step RUN_FORMAT_PYTHON_RUFF format export --path=/app/tmp now works.

THANKS Helder