#Anyone get prompt cacheing working with

1 messages · Page 1 of 1 (latest)

vital isle
#

Just looking to cut cost for things that are not really changing (system prompts)

#

@grim aurora 👆 (feels like a docs issue)

jolly wing
#

I guess you've tried using section headers:
--- LIKE THIS ---

vital isle
#

I'm not following? Do you mean in the prompt?

I literally sent the same lorem ipsum request N times in a row:

  1. when using the chat API it works
  2. when using the reponses API it works ONLY with passing the previous_response_id
#

Also when I send the previous_response_id it continues to increase the tokens (its the same lorem ipsum). Which makes sense since its the "same" convo but I don't want the same convo per-say I just want the cache hits. I thought that would be via the prompt_cache_key

vital isle
#

@jolly wing did you mean the prompt? I can also share the res/req object I see the data is being sent as a system role