I can't easily look at the docs at the

[compact nymph]

Huh. I don't remember seeing that, but I'll take a look again. Thanks for the pointer

[pearl flare]

Look in v1beta

[compact nymph]

Right, that's where the REST docs are

[pearl flare]

https://ai.google.dev/api/rest/v1beta/cachedContents

[compact nymph]

I think I looked at that earlier and gave up on it too quickly. I probably just need to spend a bit more time with it and experiment. Thanks again

[pearl flare]

The bit I'm not sure about yet is how to include the id in the request. But I haven't dug in or experimented

[compact nymph]

Do you mean the generated cache id? Yeah, I don't see it in the return type, which I'm a bit surprised to see is the same as the request type on the "create" method

I think it's inside the "name" field, which takes the form "cachedContents/{id}"

Actually, it looks like you supply your own key

[pearl flare]

But it says name is optional.
And there's is nothing in the generateContent request body that refers to the cache name. It uses another term.

[analog star]

I haven't played with it so my guess is that it works like the File API. name is optional and autogenerated if you do not supply a unique identifier yourself.

create should return with a response containing name in the format of cachedContents/{id}

generateContent now accepts "cachedContent": string

Optional. The name of the cached content used as context to serve the prediction. Note: only used in explicit caching, where users can have control over caching (e.g. what content to cache) and enjoy guaranteed cost savings. Format: cachedContents/{cachedContent}

I feel that that last part in the documentation cachedContents/{cachedContent} is a typo and should read more like cachedContents/{id}

[pearl flare]

Thanks for looking up the details - i wasn't really able to at the time. But that's exactly what I meant.

[compact nymph]

FWIW, I was finally able to create a cache using the REST API, though the contents is coming back null so I'm worried something's wrong. More importantly, though, I don't see how to get a model that actually uses the cache. How do you supply the cache to the model using REST calls?

[pearl flare]

Are you using one of the stable models? (Ie the 1.5 -001 models)

And you should supply it as part of the generateContent call

[analog star]

Can you clarify a few things?

• What do you mean when you say "contents is coming back null"

If you successfully created cached content then the response body from the create API call should not be null. And, if there were errors then it would have responded with error messages, not null.

• Are you using free Gemini AI API, paid Gemini AI API, or Vertex AI API

Each has slightly different access and requirements - see additional considerations section - https://ai.google.dev/gemini-api/docs/caching?lang=python#considerations

• Using the cached content

When calling the generateContent API you should only need to add "cachedContent": "cachedContents/{id}" to your request body

Lastly, I did get a chance to test this using REST API calls it all seems to work as expected.

1. Create cached content
2. List cached content
3. Use cached contet in generateContent call
4. Delete cached content

[compact nymph]

Thanks for your reply :). To answer your questions:

1. The reason I'm using the REST API is I'm trying to use the API from Java. That means mapping all the JSON structures to records, creating interfaces for access, and so on. When I did all that (using the Spring framework for networking and JSON parsing/generation), the CachedContent returned by my createCachedContents method (which calls the cachedContents endpoint) is:
   CachedContent[contents=null, tools=null, createTime=2024-06-23T15:23:42.406048Z, updateTime=2024-06-23T15:23:42.406048Z, usageMetadata=UsageMetadata[totalTokenCount=79447], ttl=null, name=cachedContents/m4qhg6ir3lqq, displayName=, model=models/gemini-1.5-flash-001, systemInstruction=null, toolConfig=null]

So the name looks fine and the totalTokenCount inside the UsageMetadata is correct for the file I uploaded. I'm just concerned about he fact that the contents property inside the returned root is null.

You're right, though. I'd be a lot more worried if I'd gotten an error of any kind.

2. I'm current using Gemini API directly, which I believe means I'm using the free tier, though I have a paid account and have used the Vertex API before. I prefer to go directly if possible.

3. When you say that I need to add "cachedContent": "cachedContents/{id}" to the request body, is that in the root of the body, or nested somewhere? Also, did I miss that in the documentation?

Again, thanks for responding. If you have your own REST experiment documented somewhere, I'd love to see it.

Never mind about where to find the cachedContent parameter. I just found it 🙂

[compact nymph]

Also, I think my code is working now. I still have to check it over, but I'm finally over the hump. Thanks again!

[analog star]

I just checked the full response body with my test code and list/get does not return everything in the reponse

response Body: {
  "name": "cachedContents/xxxxxxxxxxxxxx",
  "model": "models/gemini-1.5-flash-001",
  "createTime": "2024-06-24T16:20:16.829619Z",
  "updateTime": "2024-06-24T16:20:16.829619Z",
  "expireTime": "2024-06-24T17:19:50.594288263Z",
  "displayName": "TestCache",
  "usageMetadata": {
    "totalTokenCount": 696162
  }
}```

This is why you're getting null values. Seems like you'll need to make sure display name is informative and manage the unique name/id.

The documentation leaves a lot to be desired.

[compact nymph]

Makes sense. I totally agree about the documentation.

[analog star]

At least you're back on track, hopefully.

[pearl flare]

Somewhere I thought I saw it saying that it won't ever return the cached content.

The documentation leaves a lot to be desired.

Totally concurred. For a first cut at the documentation - this is below average for Google. Tho not much below average.

[compact nymph]

Yeah, I just found that as well. That's probably a good thing, but seeing those nulls in the JSON response was awkward

[analog star]

You're probably right but the documentation is spread all over the place and ever changing.

[pearl flare]

I'm current using Gemini API directly, which I believe means I'm using the free tier,
If you're using an API Key attached to a paid account - it should be the paid tier.

[pearl flare]

Right. I'm not even going to try and find where it said that.

Speaking of Vertex - we've all mentioned it before... but I couldn't find anything in the Vertex documentation saying it was supported. Has anyone seen that?