Create a secret from within dagger function.

[vital niche]

Hello.
if I have a binary called generateSecret , I cannot use it in my dagger function (ie creating a dagger container) and create a password that would be of type "daggerSecret" without exposing it in the logs ?

    password, err := myContainerWithSecretGeneratorBinary.
        WithExec([]string{"generateSecret"}).Stdout(ctx)

--> this will print the password...

myContainerWithSecretGeneratorBinary.
        WithExec([]string{"generateSecret"}, dagger.ContainerWithExecOpts{RedirectStdout: "/tmp/password"}).
        File("/tmp/password").
        Contents(ctx)

--> this will also print the password...

what solution do I have ?
(I want to use generateSecret and create a dagger.SecretINSIDE dagger world, not from the outside on the command line when calling the function.... and "of course I would like to do so without exposing the secret in th logs...)

[silent zodiac]

I believe you can use dag.SetSecret("my-secret", "<plaintext>").

BUT be advised, if you generate the secret inside dagger via a dagger API call, there's a risk that the plaintext will leak in the dagger cache. At the moment we don't have a reliable way to avoid this

(this is in addition to the log leaking issue you flagged)

@hot cairn is this right? 👆 or do we have a solution now?

[vital niche]

@silent zodiac thanks for your answer but in order to use dag.SetSecret("my-secret", "<plaintext>") I need to have plaintext which I would like to generate from within a dagger container by executing the generateSecret binary which I cannot without having it leaked/logged

[fresh lance]

Are you using the secret inside Dagger as well? I'd guess so if you strictly need to generate it inside Dagger?

If so you can just mount the /tmp/password file as a mounted secret on another container. You don't need the any of Stdout / Plaintext / Contents and it doesn't leak to output

[vital niche]

thank you@fresh lance Can you show me how ?

[hot cairn]

nope, there's no solution for the moment. I recall there's an issue where Justin and I were brainstorming some possible ideas on how we could achieve this. Looking for it rn

[silent zodiac]

Yes unfortunately I believe that's correct. It's a known issue that we want to solve, but haven't yet (we prioritized integration with external secrets first)

[fresh lance]

Use your second example, redirecting to stdout, but drop the .Contents() so your chain returns the File

Hmm actually unsure if you can do it that way. We mount secret files with input from users

[hot cairn]

https://github.com/dagger/dagger/issues/10376

[silent zodiac]

@hot cairn maybe there's a viable workaround now that we have cache-control?

• To avoid leaks in log: open a nested dagger client dagger.Connect() with logging set to dev-null?
• To avoid cache leaks: set +cache="never" ?

mmm no the inner withExec would still be cached

[hot cairn]

+cache="never" won't cache the function call AFAIK but dagger operations will still be stored in the engine cache

[silent zodiac]

right

nevermind

we're missing a nocache argument in withExec

or maybe a special redirectStdoutSecret arg 🙂

[fresh lance]

This would be useful in general

As an option on https://pkg.go.dev/dagger.io/dagger#ContainerWithExecOpts would be even better

(Possibly what you meant!)

[vital niche]

@fresh lance say what ?

[fresh lance]

I don't think it's possible @vital niche. We accept a *dagger.Secret as an input parameter with file://path/to/file, then mount that as a secret, but you don't seem to be able to do that with files inside, only those passed in with file://

[silent zodiac]

Yes that's what I mean

[hot cairn]

lol, this is along the line of what I discussing with Justin

[silent zodiac]

I guess I should just read that github thread instead of rebooting it here 😛

[vital niche]

Thank you for your answers.
This means that all the code that has been written here and is available as a "library" will actually
log/leak all the "secrets" it generated... correct ?
https://github.dev/dariusj1/daggerverse/blob/ff30733e902cf8d3c7c00a76f80b6381ddd19aae/aws-oidc-auth/dagger/main.go

[fresh lance]

My company uses OIDC for AWS and we do that bit outside of Dagger, then pass the generated credentials inside. I'd be quite interested to see how this shapes up though

[hot cairn]

yes, that will leak secrets. The "best" way to implement this would be through a secret provider currently

or do that outside dagger as @fresh lance is currently doing

having said that, we should push https://github.com/dagger/dagger/issues/10376 forward and find a better way to dynamically hook secret providers into the engine maybe (?)

[fresh lance]

Is there not a quick fix by doing what Solomon suggested and allowing output to be hidden somehow?

Is the issue there that it'll end up cached?

[hot cairn]

yep. I don't think there's a way AFAIK

silently pinging @potent pawn and @drifting onyx and @worldly obsidian in case you might know a way 🧠

[vital niche]

Can someone explain to me why it is not possible to implement a File.toSecret function? (where FIle would contain a password or private key)

[silent zodiac]

There's always the option of someone contributing the missing primitive 😇

[silent zodiac]

The issue is upstream - if there's a File then there's a) a cache entry for that file, and b) a withExec which produced it, leaking logs in the process

Hence the idea of going straight from withExec to Secret

[vital niche]

ok but when we use withExec with dagger.ContainerWithExecOpts{RedirectStdout --> does it sill leak logs in the process ? (I thought it was the stdout or the File.COntentthat were leaking logs...

[silent zodiac]

There are 2 possible sources of leaks (that I know of):

1. Logs
2. Cache

[vital niche]

I was actually only talking about the logs as that is really visible...
I will need to learn about the Cache function...

thank you.

[silent zodiac]

Yes I agree logs are more obviously problematic. Then maybe there's a way to avoid leaking logs, reducing the scope of the problem to only cache leak?

[vital niche]

since I am currently not using cache, that would be a good "temporary" solution for me

[silent zodiac]

Dagger always caches all operations automatically. So the secret's plaintext will be stored in your dagger engine's local state directory (/var/run/dagger or an equivalent path inside the engine container)

[vital niche]

yes but since I am using github action at the end of the action everyting is "erased"...

[silent zodiac]

Right

[vital niche]

(I will try to use the github cache with the var/run/dagger at some point to see if it saves some time, but not now...)

[silent zodiac]

that doesn't work super well anyway, github cache is a little too dumb & slow for this to work as you would hope. But, we are working on something much better 🙂

[vital niche]

can t wait to see all these! keep going with the good work. Any pointers to "how to disable the logs" ?

[worldly obsidian]

I have a terrible, work-around idea, until we get something implemented.

just use symmetric encryption, like aes, and pass the encryption key via a secret.

echo -n protect-this | OPENSSL_PASSWORD=keepitsecret openssl enc -e -aes-256-cbc -pbkdf2 -iter 1000000 -a -salt -pass env:OPENSSL_PASSWORD

and then you can set OPENSSL_PASSWORD as a dagger secret. Then you can use .Stdout(ctx) as your original example.

[hot cairn]

Oh right! Now you reminded me that someone effectively did this to overcome the limitation lol

[worldly obsidian]

hah, I'm glad I'm not the only crazy one out there 😆

[vital niche]

@worldly obsidian thank you but could you explain within a real dagger function ?

[hot cairn]

@vital niche something like this (pseudo-go-code)

func FetchSecret(ctx context.Context, pass *dagger.Secret) *dagger.Secret {
encryptedPass :=  myContanerWithSecretBinary.
  WithSecretEnv("OPENSSL_PASSWORD", pass).
  WithExec([]string{"sh", "-c", "generateSecret | openssl enc -e -aes-256-cbc -pbkdf2 -iter 1000000 -a -salt -pass env:OPENSSL_PASSWORD").Stdout(ctx)

//you can now decrypt encryptedPass in this function using pass.Plaintext(ctx). Doing this will only leak the encrypted secret
}

[silent zodiac]

you could generate the keypair natively inside 'FetchSecret`

[vital niche]

@hot cairn thank you for your answer, but since this function you suggested takes a secret as a parameter it is a chicken and egg problem...
Maybe what @silent zodiac is suggesting is a solution but I do not understand it what he said.

[hot cairn]

@vital niche you can generate a random secret within your function instead of passing it as an argument

something like: (ai generated)

package main

import (
    "crypto/rand"
    "fmt"
    "math/big"
)

// generatePassword generates a random string of a specified length using a cryptographically secure source of randomness.
func generatePassword(length int) (string, error) {
    const charset = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789!@#$%^&*()-_=+"
    password := make([]byte, length)
    for i := 0; i < length; i++ {
        // Use crypto/rand to generate a secure random number for index selection
        randomIndex, err := rand.Int(rand.Reader, big.NewInt(int64(len(charset))))
        if err != nil {
            return "", err
        }
        password[i] = charset[randomIndex.Int64()]
    }
    return string(password), nil
}

func main() {
    length := 16 // Desired password length
    password, err := generatePassword(length)
    if err != nil {
        fmt.Printf("Error generating password: %v\n", err)
        return
    }
    fmt.Printf("Generated secure password: %s\n", password)
}

[silent zodiac]

@hot cairn I didn't even know there was a 😂

[hot cairn]

lol, same. I accidentally typed it. No idea what emoji I was looking for

[silent zodiac]

I thought it meant "I half-like it" 😛