Let's start a separate thread about AI

[gray spear]

Do you have gloud?

[inland vector]

Yes, I have gcloud installed.

[gray arch]

please share the link, help me install

[gray spear]

https://cloud.google.com/sdk/docs/install

[gray arch]

just installed

[gray spear]

ok, try this:

• unset GOOGLE_APPLICATION_CREDENTIALS
• run gcloud auth application-default login and login with the user the model was created under
• then run the app that accesses the model

[inland vector]

I found this warning

[pulsar verge]

Hi all, didn't follow the whole thread, but assuming the tuned model was made in AI Studio? If so then I think it's associated with your logged in user account and not the gcloud project/service account, so generate calls would need to be oauth authenticated with your user account.

You can add permissions to models using https://ai.google.dev/api/rest/v1beta/tunedModels.permissions/create but there's no UI for this in AI Studio currently.

[gray spear]

Yup, that's what I realized. Trying to use gauth at this point to have them log into their own accounts using application default credentials. Which I'm pretty sure your default library supports.

[gray spear]

And knowing there is an API for it is useful to know! But does have a chicken and egg problem to use it. {:

[pulsar verge]

I think the best working flow is (1) Make model in AI Studio (2) Make OAuth'ed POST request as yourself to add your service account (or everyone if that works for you, you'd need to know the model name to call it but it's obfuscation at best)

Definitely complicated

[inland vector]

Error: 7 PERMISSION_DENIED: Request had insufficient authentication scopes.

[gray spear]

I'm honestly confused by that one, since the library should be requesting the right scopes.

[inland vector]

I followed this advice. https://ai.google.dev/palm_docs/oauth_quickstart

[gray spear]

stand by

it looks like the node library doesn't have listModels

(that call is from the PaLM page, so is obsolete)

[gray arch]

as per this, we might need to create a virtual session to get the access token and then use the access token to set permissions to as our client email. so we can use the tuned model

[gray arch]

hold up, we might need to set permissions before trying to access it

[inland vector]

How do I set it up?

[gray arch]

we would need to create a session to obtain an access token

[gray spear]

Try just accessing the model once you've done the gauth

The gauth should log you into the applicaiton default credentials as yourself. The node library will use these default credentials if no other credentials are provided.

The problem is - the node library doens't have a way to list models.

[gray spear]

Well... this is a new one for me. Trying to log in using gcloud with the training scope is blocked as an unsafe action

[gray arch]

yes im trying for the scope as well

is it training?

[gray spear]

https://www.googleapis.com/auth/generative-language.tuning
or
https://www.googleapis.com/auth/generative-language.tuning.readonly

[gray arch]

just figured

can you give me an example url string

to set permissions

[gray arch]

i was able to login with the specified scope, unable to decipher this. can you provide an example endpoint

@gray spear @inland vector @pulsar verge was able to figure the url below is the data im passing and error as follows

[gray spear]

https://generativelanguage.googleapis.com/v1beta3/tunedModels/YOUR_MODEL_NAME_HERE

[gray arch]

@gray spear got it, i was able to set permissions as well

it was a 200

[gray arch]

@gray spear @inland vector @pulsar verge I was able to create a session with tuning scope and then make call to set permissions, get content , getmodel as per the documentation. REST is fine, but node doesn't work even after updating the permissions. Is REST the possible work around for this. do we need to generate tokens and make REST calls. or will it work, the NODE way

[gray spear]

It should work with Node. If you have permitted your service account, and then use the credentials for the service account, I would expect it to work.

How did you get the REST to use your account permissions?

[pulsar verge]

Sorry, don't know node well. Does the trained model show up in the list (tuned) models equiv. for node?

[gray spear]

There is no trained models list in node, apparently.

(at least from my reading of the documentation and code)

And when I try to use REST, I either don't have the right scope... or it won't allow me to use the scope for security reasons. {:

[pulsar verge]

This will become an official example at some point, but here's the model sharing flow in Python/colab... https://gist.github.com/williamito/1701ae3fae909b40de328d43d347620d

it's written in Python but uses REST calls

[gray spear]

interesting. and it uses the --scopes parameter for gcloud auth application-default login, which is giving me an error. tho I'm not using the client-secret.json. will have to plahy with this more.

[pulsar verge]

The specific steps were adapted from https://ai.google.dev/docs/oauth_quickstart and I'm just parroting, definitely not a Cloud auth expert.

[gray arch]

I achieved this using only tuning scope, in the Postman Client. The workflow would be similar to creating a login session and storing the access token. With the access token present, I was able make api Calls and it is working fine. It seems that REST alone is working. But I would need a NODE work around so I wouldn't need to deal with sessions and tokens

[gray arch]

let's just say, we want to build an API endpoint and want to use this Tuned Model in it to process the input. it appears that all the requests should be authenticated, is it? and it is the only possible work around? @pulsar verge

[pulsar verge]

I haven't used the tuning API much myself, but my understanding is all requests involving tuned models would have to be OAuth authenticated

how you want to design the authentication is up to your business requirements

[pulsar verge]

A small correction to what I said here; if you share the model to everyone the model name IS discoverable if people try, so you should only do this if you actually mean everyone.

[leaden thunder]

have anyone successfully run tuned model with OAuth?

[pulsar verge]

By "run" do you mean making the model, or using generateContent with it? Both should work, though SDK support seems to be a bit behind the API in some places.

[gray arch]

i was able to make authenticated calls to it over POSTMAN to the endpoint and was able to generate content. Tried to develop REST API using NODE+EXPRESS but was unsuccessful. Now im trying with PYTHON+FLASK. Hopefully it works since Gemini API has better Documentation with Python

[leaden thunder]

Great to know. Can u share the URL you call after OAuth? And parameter needed?

[gray arch]

https://generativelanguage.googleapis.com/v1beta/tunedModels/your-model-id:generateContent

[gray arch]

google.api_core.exceptions.PermissionDenied: 403 You do not have permission to access tuned model tunedModels/test-12va0xxxx5ij. //// Got the same error in the python+flask as well. Now this confirms that we can only use tuned models with our own authenticated sessions. @pulsar verge @gray spear

[pulsar verge]

That should go away if you share the model approprately. Not sure if you're planning to have users call the model with their credentials, or e.g. go through your API and your backend uses a service account with your model. I've tested user sharing at least.

[gray arch]

the problem is, i would want to create an api endpoint so it can be accessible by the allowed domain names. I want to be able to call the tuned model from the api logic. but it requires authentication, and only works with the owner's session. now i am obligated to authenticate it every time i make calls to it. access tokens, refresh tokens and persist tokens. ah! what a mess, google should have never published it for production use. I was able to make it work, but it feels like i wasted a lot of time. why would i deploy it and authenticate it logging in and creating a session. this doesn't feel right, and even there is no proper documentation. @pulsar verge

[pulsar verge]

Authentication is Hard, but it's a common challenge. It sounds like you want to send requests authenticated using your server's credentials? If so https://developers.google.com/identity/protocols/oauth2/service-account would be a starting point. If you wanted to use the user's credentials there are many OAuth libraries that can help handle things for you.

[pulsar verge]

This may be more useful if you haven't had to deal with authentication much before https://developers.google.com/workspace/guides/auth-overview

[gray spear]

I've had to deal with auth extensively - and I find this a mess to deal with. {:

(It might be easier if I could set permissions in the UI, however, I will admit. And I'll see if I can coerce it to play with this weekend, perhaps.)

[pulsar verge]

I'm working on more OAuth docs/examples for Colab at least!

[gray arch]

thanks, ill go through this

anyways I was able to deploy my REST API, but the catch is, I would login over the browser to hold the session, and then using the refresh tokens in the Api logic to handle token expiration and making subsequent calls. it feels slightly unprofessional, kind of, just a bit. Finally, some relief over the weekend:)

[gray spear]

This is what a service account should be used for.

[gray arch]

Please share some proper resource or documentation to help make this work

[gray spear]

Well... I'm in the middle of writing some. {:

And part of the problem is that you'd have to permit the service account, and I don't know how to do that yet for the tuned models.

[pulsar verge]

@gray spear I think you would https://ai.google.dev/api/rest/v1beta/tunedModels.permissions/create a permission that includes the service account's email, but I haven't had time to test this specifically yet

[gray spear]

Nod. That was my guess, but I haven't had the chance to test it yet, either.

[pulsar verge]

sudden realization that I wrote all the code snippets I need to test this over the weekend, let me try now 🙂

[pulsar verge]

Okay, I've basically confirmed this works, as I can genai.get_tuned_model() in Python, a tuned model shared with a service account, but it fails with a different model I didn't share

I'm having issues with generating content, but I think I just need more time to understand the SDK

[willow solstice]

this all intresting and i know the issue

[crude mountain]

@gray spear Okay, I'm here. I'm still stuck on the "Set up application default credentials" part. I don't understand where I need to insert this in the code and if I need to replace anything. I'm making a chatbot with Gemini in a messenger and if I mess up something it will be a bit difficult.

[gray spear]

This isn't meant to be "inserted in the code". This is a command you need to run on the command-line. gcloud is a set of tools that interacts with Google Cloud.

[crude mountain]

Win cmd or cloud shell?

By the way, why is it that if the model on the Gemini website and in the code is essentially the same, but the behavior is different?

[gray spear]

Win cmd or cloud shell
Wherever you're trying to run the app that will call the trained model from.

why is it that if the model on the Gemini website and in the code is essentially the same, but the behavior is different?
I'm... not sure I follow what you're saying.
You're asking why you need to use OAuth when it works fine using AI Studio?

[crude mountain]

Maybe it's difficult for me. Do you know if there will be any other ways to use tuned models?

[crude mountain]

Maybe I'm the only one like that...

[gray spear]

I'm trying to understand the question, first. {:

[gray spear]

(And no. I find this difficult and annoying for a variety of reasons. I haven't gotten through trying them out for this same reason.)

[crude mountain]

Ok I can explane simply

For example, I can go to the official Gemini website and if I say "Hello" to him, he can also complete the answer and offer something (for example, to find out something). And if, for example, you take a own version in a telegram bot, then he can respond to "Hello" like "Hello, how are you?" And nothing else

[gray spear]

oh! ok. that's a more general topic and not about OAuth and tuning. {: That's what confused me.

The Gemini chat app has many other models and prompting setup as part of it. So, for example, the temperature has been tuned. The instructional prompt is longer. etc etc. (This is a better discussion for the main channel if you do want to followup more.)

[rotund nimbus]

How exactly do you do this permission creation in Python?

[pulsar verge]

https://github.com/google-gemini/gemini-api-cookbook/blob/main/quickstarts/Authentication_with_OAuth.ipynb has relevant samples to prototype with. Using OAuth in production is beyond the scope of this doc, but also the challenges aren't specific to the Gemini API.

[rotund nimbus]

My print('Available base models:', [m.name for m in genai.list_models()]) works fine. It's the tuned models lack permissions.

Thanks for the last session on permission creation. Will give it a try.

[rotund nimbus]

<Response [404]> and <Response [401]>

[gray spear]

When youlist models, is it showing your tuned model?

[pulsar verge]

Where did you make the tuned model? if it was AI studio, you'll need to oauth to colab using the same account. You can confirm auth is working as expected if you see the tuned model under genai.list_tuned_models().

[pulsar verge]

"list models" and "list tuned models" are different API endpoints, and different methods in python

[gray spear]

sigh... right... thank you.

[rotund nimbus]

No. It shows only the base models. The tuned models return an error with permission.

[rotund nimbus]

I did the tuning in AI studio. Should I retrain them in Colab?

I was following instructions "Get Code" in the AI studio.

[rotund nimbus]

I auth on my local python env

Can confirm that base models are working so the auth should be good. I am guessing it's stuck at Tuned model permissions.

Is this like so by design? I can imagine Google imposes this to avoid naughty finetunings made to the news. But I am doing this for my PhD

[pulsar verge]

If you can't see the tuned model using list_tuned_models then the tuned model was created by a different account than yours.

or at least it was not made by the account you're currently authenticated as

[pulsar verge]

Ah, there's a known issue with Get Code and tuned models; Get Code still uses an api key, but you have to use OAuth for tuned models.

[short bough]

In the "new tuned model" screen in the Ai studio, why is my Tune button greyed out, even though everything is filled out and loaded

[gray spear]

Are you using an exisitng prompt or importing the data?