#I noticed with 0.18.6 I'm prompted by
marble flare
What’s the exact scenario this happens? Just literally run dagger shell with no args?
echo veldt
yup! And I have env variables like GEMINI_API_KEY that are 1password refs
stuck marsh
Yep, I've noticed the same.
Related, perhaps, on the secrets topic: when I have multiple secrets in op:// as function args, I see three calls to 1password to auth.
Guessed that was the serial secret provider access, but unsure if that's a change and we would have cached that auth for a ttl previously.
marble flare
The cache key for secrets is based on the plaintext value of the secret now (https://github.com/dagger/dagger/pull/10311), which currently requires reading the plaintext when the secret is created. So the plaintext is read earlier than it used to be.
For the LLM api key use case, we could probably get away with creating the secrets more lazily and only on demand
it's definitely weird that you get prompted for the secret before you even do anything w/ llm (if you do at all)
echo veldt
Maybe there's another change other than 10311 changing how those credentials are loaded?
marble flare
No I think 10311 explains what you are seeing. The implementation of llm has always tried to create secrets when the core API is loaded. Nothing changed there. It's just that pre-10311 calling secret didn't require resolving the plaintext, now it does.
halcyon acorn
The part that's mysterious to me is why opening the shell triggers a LLM config lookup in the first place
Is it because of prompt mode?
marble flare
I thought it's because reading the llm api keys happened when the schema for llm was loaded, so it happens at the start of any session since llm is in the core API.
Double-checking that's true though
halcyon acorn
If that's the case, we should probably change it. I don't see why that would be needed. Normally the earliest you need the LLM config is when actually instantiating one (dag.LLM()). And ideally it would be even lazier than that, and only done when an actual request needs to be made (eg. LLM.Sync())
marble flare
eh maybe it's not, the middleware stuff I remembered around llm isn't there now, so i'm out of sync with that
halcyon acorn
Oh it might be a side effect of having to create a throaway instance of the type to pass it as a Typed value in the dagql API?
anyway my first reaction is - surely there's a way to solve this without changing low level secrets caching
marble flare
anyway my first reaction is - surely there's a way to solve this without changin...
yeah that's not in the cards for now, we just need to make whatever is triggering the load of llm api keys lazier
robust shore
the shell silently tries to instantiate $agent=llm(), it helps with testing and agent dev workflows like dagger -c "$agent | tools"
when i added that, i did not know there was an interactive path that might be hit
marble flare
the shell silently tries to instantiate $agent=llm(), it helps with testing and ...
That would probably explain it, is that easy to avoid?
robust shore
ideally it'd be lazy, like you have a agent var bound that only gets evaled when you evaluate it? idk if that's possible... alternatively, right now i only used it to test this
so maybe there's some other way to test that, or if the regression here is worse than the feature we can just comment those tests
could also potentially add a flag that does this. under test i just don't wanna attach a tty, switch to prompt mode, and then switch back to shell to populate $agent
halcyon acorn
@robust shore I think it's reasonable to set $agent=llm(). The cleanest fix would be to move LLM config lookup to Sync rather than NewLLM(). I think I actually have a FIXME to do that, just never got around to it
you should need to lookup config when you instantiate a LLM - only when you actually call the API
marble flare
^ makes sense to me
robust shore
yeah that does sound ideal
@echo veldt you mind opening an issue so i can claim it and fix? i actually don't know the specific env var setup that'd trigger this, is it like GEMINI_API_KEY=op://vault/ref/etc?
echo veldt
<@135620352201064448> you mind opening an issue so i can claim it and fix? i act...
on it! and yes exactly GEMINI_API_KEY=op://Employee/GEMINI_API_KEY/credential
robust shore
shell magic
echo veldt
as an aside I think it's so cool that I can have my dotfiles committed like this https://github.com/kpenfound/dotfiles/blob/master/.zshrc#L207-L230
robust shore
as an aside I think it's so cool that I can have my dotfiles committed like this...
i used to do this for arbitrary api keys in shell with a checked in .envrc like export API_KEY=$(op read XYZ) but it'd prompt me every time i opened a new terminal split
https://developer.1password.com/docs/cli/secrets-environment-variables/ that was years ago and it turns out 1p added op run to solve this problem, pretty slick
echo veldt
oh that's nice (not sarcastic 🤣 )