#aws-cli in dagger doesn't use EC2 instance role, `docker run` does

1 messages · Page 1 of 1 (latest)

sterile grove
#

Hi. I'm new to Dagger, playing around with the Python SDK. I'm running it on an EC2 instance that has an IAM role attached and I'm using the "aws-cli" image from AWS. Running aws-cli s3 ls using docker run works, but the equivalent from dagger complains it has no credentials. Details:

This is the docker command that works:

docker run --rm -it public.ecr.aws/aws-cli/aws-cli s3 ls

This is fine, it lists my buckets using the IAM role attached to the EC2 instance.

But when I run this Python code:

import sys

import anyio
import dagger


async def main():
    dgr_config = dagger.Config(log_output=sys.stdout)
    async with dagger.connection(dgr_config):
        s3ls = (
            dagger.dag.container()
            .from_("public.ecr.aws/aws-cli/aws-cli")
            .with_exec(["s3", "ls"])
        )
        s3ls_output = await s3ls.stdout()
        print(s3ls_output)


anyio.run(main)

by doing python3.11 dgr-awscli.py it results in:

Unable to locate credentials. You can configure credentials by running "aws configure".

Why? IIRC I can retrieve temporary credentials and pass those to the container via envvars, but is there a better/easier way?

rugged bone
#

In my experience you need to pass the environment variables to Dagger yourself. I'm sure I've seen that this is a design choice - no confusion about what is used from the host, and no risk of code/modules scooping up env vars or files from the host without you knowing/approving that behaviour.

noble violet
sterile grove
#

Got it, thank you both

sterile grove
#

I'll just leave this here, in case someone else comes searching. Python code showing how to pass temporary AWS credentials to a container by assuming a role. Can be used for setting up infra from a dagger pipeline

import sys

import anyio
import dagger
import boto3


def get_aws_credentials():
    sts_client = boto3.client("sts")
    temporary_credentials = sts_client.assume_role(
        RoleArn="arn:aws:iam::123456789012:role/infra",
        RoleSessionName="dagger_infra",
        DurationSeconds=1800,  # defaults to 3600. See docs
        # Policy={},           # nice to have, if practical. See docs
        # SourceIdentity="??", # probably a good idea. See docs
    )["Credentials"]
    return temporary_credentials


async def main():
    temp_aws_creds = get_aws_credentials()
    # The access key ID is passed in clear text because it could be considered
    # not a seret, like a user name.
    dgr_aws_secret_access_key = dagger.dag.set_secret(
        "SecretAccessKey", temp_aws_creds["SecretAccessKey"]
    )
    dgr_aws_session_token = dagger.dag.set_secret(
        "SessionToken", temp_aws_creds["SessionToken"]
    )

    dgr_config = dagger.Config(log_output=sys.stdout)
    async with dagger.connection(dgr_config):
        s3ls = (
            dagger.dag.container()
            .from_("public.ecr.aws/aws-cli/aws-cli")
            .with_env_variable("AWS_ACCESS_KEY_ID", temp_aws_creds["AccessKeyId"])
            .with_secret_variable("AWS_SECRET_ACCESS_KEY", dgr_aws_secret_access_key)
            .with_secret_variable("AWS_SESSION_TOKEN", dgr_aws_session_token)
            .with_env_variable("AWS_DEFAULT_REGION", "us-east-1")
            .with_exec(["s3", "ls"])
        )
        s3ls_output = await s3ls.stdout()
        print(s3ls_output)


anyio.run(main)
#

Things like Terraform, CDK, Pulumi, etc.

rugged bone
#

I need a few key capabilities in my pipelines for AWS-hosted services - auth, s3 (get/set files), ssm (get/set params), secrets manager (get/set secrets), etc etc

#

These feel like they should be 'library' modules, created once and re-used in everything else

sterile grove
#

I’m mostly playing around in my spare time, but I do have something in mind and I will eventually have to build those pieces too if I continue. But why wouldn’t you simply use Terraform/CloudFormation/CDK/Pulumi for that?

#

I was actually looking into how I can pass credentials to Terraform, I’m using it for quick prototyping at the moment. There’s an official Terraform image that I can probably use to do a quick ‘terraform apply’

rugged bone