python code within container block | Dagger | Page 1

devout comet Aug 31, 2023, 5:43 PM

#

hey everyone - I'm doing a live demo today! is it possible to run python code from with the calls in the container? Ideally it would behave in this way:

container()
#magic goes_here
for folder in subfolder:
  .with_workdir(f"/app/{shared_config.base_tg_folder}/{os.getenv('DEPLOY_ENV')}/application/{subfolder}")
.with_exec(command_to_exec("terragrunt run-all apply --terragrunt-non-interactive")

rare blade Aug 31, 2023, 6:31 PM

#

Not sure I understand, can you clarify?

devout comet Aug 31, 2023, 6:47 PM

#

sure - I think its something that is not dagger but python but I can't figure it out. here's what I have that is not working:

async def get_tg_sub_folders(client: dagger.Client):
    container = client.container()
    git_client = client.git(shared_config.clone_url).branch(shared_config.branch)
    project = git_client.tree()
    num_of_subdires = [x[0] for x in os.walk((f"{project}/infrastructure/terragrunt/qa/application/"))]
    if len(num_of_subdires) > 1:
        for folder in num_of_subdires:
            container = getattr(client.container(), "with_workdir")(folder)

    return project, container

#

man discord code look horrible but im trying to run terragrunt in multiple directories (downloaded by the git() dagger method) if they exist.

#

so i need to move to mulitple directories within the container if they exist. but Im having a hard time getting it to work with all the method chaining.

rare blade Aug 31, 2023, 6:54 PM

#

You need to pair with_workdir with with_exec, otherwise you're continuously changing the workdir in the container without doing anything with it.

devout comet Aug 31, 2023, 6:56 PM

#

yeah I fixed that - but the issue right now is that the directory returned by project = git_client.tree() appears to be empty(which it is not)

rare blade Aug 31, 2023, 6:57 PM

#

Have you called await project.entries() to check if it's empty?

devout comet Aug 31, 2023, 6:57 PM

#

oh man... just as you sent i ran a debugger. oof. let me do that.

rare blade Aug 31, 2023, 6:59 PM

#

Oh, I just noticed in f"{project}/infrastructure/terragrunt/qa/application/" that you're trying to use the value from .tree(). When I read it I thought it was a string var with the path prefix or something. Yeah, that won't work since .tree() returns a directory. There's multiple ways to do this, let me think for a second what's the simplest one.

devout comet Aug 31, 2023, 7:00 PM

#

i think you may have just saved the day one moment.

rare blade Aug 31, 2023, 7:00 PM

#

Is it a flat list of dirs, or do you need it to be recursive?

#

If it's flat than .entries() is all you need.

#

For completeness, it should be something like:

subdirs = await project.entries(path="/infrastructure/terragrunt/qa/application")

devout comet Aug 31, 2023, 7:11 PM

#

wow.

#

phew okay. This should be documented - i would imagine for people running TF or TG this is a chunk of pure gold.

rare blade Aug 31, 2023, 7:12 PM

#

It's useful to take a look of what's possible from the reference docs: https://dagger-io.readthedocs.io/en/sdk-python-v0.8.4/client.html#dagger.Directory.entries

#

And there's a cookbook entry for listing directory contents: https://docs.dagger.io/cookbook#list-host-directory-contents

#

And for adding a git repo as a directory in a container: https://docs.dagger.io/cookbook#add-git-repository-as-directory-to-container

devout comet Aug 31, 2023, 7:15 PM

#

Oh I've used directory elsewhere, but never in combination with getattr and you helped me put those peices together.

rare blade Aug 31, 2023, 7:15 PM

#

You don't actually need that getattr, do you still have it?

devout comet Aug 31, 2023, 7:16 PM

#

I do. I need to change into the direcories returned by subdirs

#

and then as you said, run the with_exec

rare blade Aug 31, 2023, 7:16 PM

#

But why the getattr?

#

I mean this line:

container = getattr(client.container(), "with_workdir")(folder)

#

That's the same as

container = client.container().with_workdir(folder)

#

Anyway, each time that's a new empty container. Do you add from_ later on?

#

By the way, does your terragrunt executable accept a directory as an argument? That would be easier than constantly changing workdir and pairing correctly.

#

Another possibility is to make a script that does all of this and add it to the container to run there instead of doing it on the host.

devout comet Aug 31, 2023, 7:21 PM

#

I dont know the answer to that question. But i setup the base container. I'm looking into doing a for loop with it, instead of setting an attribute of the container.

#

no scripts!

#

thats what Im driving away from with our current pipline

rare blade Aug 31, 2023, 7:24 PM

#

So you need to set the base (with from_) before the loop, and add the with_exec in the loop instead of just with_workdir.

devout comet Aug 31, 2023, 7:32 PM

#

it works now. I don't know how to thank you.

#

    if len(subdirs) > 1:
        print(f"subdirectories found {subdirs}")
        for folder in subdirs:
            apply = run_tg.with_env_variable("CACHEBUSTER", str(datetime.now())) \
                .with_workdir(f"/app/{shared_config.base_tg_folder}/{os.getenv('DEPLOY_ENV')}/application/{folder}") \
                .with_env_variable("CURRENT_VERSION_WEIGHT","100") \
                .with_env_variable("NEW_VERSION_WEIGHT","0") \
                .with_exec(command_to_exec(f"terragrunt init -upgrade --terragrunt-non-interactive")) \
                .with_exec(command_to_exec(f"terragrunt run-all plan --terragrunt-non-interactive"))
            apply_out = await apply.stdout()

#python code within container block