Best way to convert a dagger.File into a dagger.Secret

[left warren]

Hello, here! I'm working with a TLS private key that get passed as a dagger.Secret through the CLI.

s.with_mounted_secret(mounted_tls_key_path, tls_key)

Now, technically I could also fetch this file from the src directory that gets passed as an argument, and I would like that to be the default behaviour (i.e. tls_key is an optional argument). My solution was to set tls_key to None by default and then do:

if tls_key is None:
    tls_key = src.file(TLS_KEY_PATH_IN_SRC)

Here's where the conundrum arises. The tls_key can now either be dagger.File or dagger.Secret. I could not find an easy way to convert a dagger.File to dagger.Secret, so during mounting I did a conditional like:

s = (
    s.with_mounted_file(mounted_tls_key_path, tls_key)
    if isinstance(tls_key, dagger.File)
    else s.with_mounted_secret(mounted_tls_key_path, tls_key)
)

Frankly, this looks pretty hacky and I believe there must be a better way to do this. I dislike how in this case I treat the tls_key as either a File or a Secret when mounting it into the container, when I want it to be a Secret. Better ideas would be much appreciated!

[tiny surge]

heya 👋

so the reason there's no real easy way to do this conversion is because it's not really safe to do it - by converting a file to a secret manually/etc, the file itself has been cached/etc - which really isn't desirable for a secret

you can still hack it if you like, by making a new secret from the file.contents

but really, the right way to do this is to make sure that secrets are secrets all through dagger if possible

[left warren]

Does this

if tls_key is None:
    tls_key = src.file(TLS_KEY_PATH_IN_SRC)

cache that secret?

I adjusted it to convert into a secret like this:

if tls_key is None:
    tls_key = dag.set_secret(
                 'tls_key', 
                 await src.file(TLS_KEY_PATH_IN_SRC).contents()
               )

I assume this also got cached in plaintext?

This secret in question is not truly a secret, but I'm interested in the mechanism here i.e. what I need to be especially mindful about. Especially if the directories we pass to functions also get cached, then we would need to explicitly leave out the secrets using exclude in dagger.json? The reason why I'm able to be access that tls_key as a dagger.File within the function is because I passed the whole source directory to the functions (including that file).

[tiny surge]

yeah, so at each step, each directory/file/container/etc is essentially able to be cached - including the local directory that's been uploaded

if you have secrets in the filesystem, then yeah, you need to exclude those

cc @spice granite 🤔 i guess this is another reason for the .gitignore support (and a reason to have it apply for module loading, and for host directory loading) - i have a .envrc file in my local directory, that i've just realized is probably getting cached in every build i'm making

tl;dr @left warren - yes, this is really fiddly and tricky at the moment, i think this justifies some changes in dagger to make it easier to reason about this

[left warren]

Thanks for the clarification, that's really good to keep in mind, especially because starting out I was just always passing the whole project directory! Luckily, I didn't pass anything truly sensitive 😄
I'm looking forward to the contextual directory feature, I think it will cut down on the amount of files I'm passing.

[left warren]

@tiny surge I'm curious about one more thing wrt caching. Does calling plaintext on a Secret also get cached or maybe ends up somewhere in logs?

For example, I need to make a GET request with requests while using the basic authentication:

r = requests.get(SOME_URL,
            auth=(username, await password.plaintext()),
        )

password was of type dagger.Secret and was passed as such. Was I correct in doing this? I needed to pass the actual plaintext.