Using llm without setting environment variables

[hasty yew]

• I have a command I want to distribute as a tool for devs.
• I'll vault the llm api key and connection info in azure keyvault.
• I then want to retrieve those to create my llm client. Instead of them having in local environment they'll have access to keyvault by default as part of their role and then the secrets and config will be retrieved but without every setting in their terminal. This might be more obfuscation than security, but keeps it far less likely I need to worry about more setup commands ( pwsh vs zsh) and can just let them run the command and benefit from the llm automation I wrote without more plumbing.

Is it possible without doing everything from scratch to let me enable llm for azure OpenAI and set client connection info inside dagger code without needing it in environment? Sure would simplify distribution/sharing!

[zealous basalt]

@hasty yew the user would need to have at least some sort of key so they're able to access the keyvalut and retrieve the corresponding connection info and creadentials to talk to the LLM

if that's the case, user defaults (https://github.com/dagger/dagger/pull/11034) + secret providers would be the best DX

[hasty yew]

Their azure identity would let them. So I could essentially use azure-cli image or grab token as I did on last module and use to retrieve secret to create client credential without having to ask them for it. That would be the goal. So essentially assume I can retrieve secret, then would user defaults be what I'd need?

My goal was to hand a snippet to someone in a repo that doesn't have dagger, and give me a single command to run and it would self retrieve the keyvault values assuming I've already allowed all the devs to read that vault, and configure the llm client for azure openai at that point.

Note they wouldn't have any .env files, but at the worst case I could add those to the repo with settings if that's really what's needed. Ideally it would be just a command, but I'm ok with .env file if that helps here.

[zealous basalt]

Their azure identity would let them. So I could essentially use azure-cli image or grab token as I did on last module and use to retrieve secret to create client credential without having to ask them for it. That would be the goal. So essentially assume I can retrieve secret, then would user defaults be what I'd need?

are you building your custom CLI? or are you expecing users to directly run dagger call?

[hasty yew]

I'm in devex, so typically I need to assume they don't have any of my tools except dagger. I want them to invoke /diff prompt in repos I'll provide and it will run dagger -m remotemodule/differ call suggest -o .artifacts/differ which will contain git diff report, and an agentout.md that contains the llm summary and suggestion. Copilot will just read that and proceed with the rest. My hope is to that the invocation of this is seamless and one click. The credential retrieveal ideally is self contained in dagger, as they'll have access to the keyvault, but I don't really want to make it obvious that folks might start reusing export LLM_KEY=az keyvault read foo cause it's just asking for folks to do more with it. Having it just self setup in my repo is enough obfuscation for this purpose.

flow

• opens copilot
• /changelog
• prompt told to run dagger ....
• copilot reads the report
• provides suggestions and iterates, no logic to run locally, no scripts, I've handled in my dagger module

very basic usage, but I have more cases this would be great to do as I send something to someone and it can run some llm automation to try and help them and spit out the updates, without any more plumbing than "here devex has a command to run that is self-setup and self-contained"

[zealous basalt]

Gotcha makes sense.

So, user defaults don't really help in this particular case since the .env file with the correct default needs to exist in the developer's machine before making the Dagger call.

Since the only secret that you seem to need is the user's keyvault credentials, adding a --keyvalt=cmd//keyvault_cmd doesn't seem that bad

Ideally it'd be nice to have a specific keyvault provider so you don't depend on a binary being available on the system

[zealous basalt]

Should not be too much effort in implementing one if you're up for it. If you follow the one password or vault providers, it should be straightforward

[hasty yew]

So I'd essentially give a command that has az keyvault retrieve command in that format and it would inject the secret but without exposing on terminal?

That still means they could run adhoc themselves so not ideal. Maybe I'll explore vault provider option then for the future.

Thank you!