Trying to understand how batch API works with system prompts. I have a system role message (which contains instructions on what to do with user role inputs) that has about 500 tokens and it's the same across all API calls. The user role message is different per API call. If I include the system role message on every single call in the batch API, then it'll take up so much token and would be very costly. Is there a way to only mention it once? Can I only mention it once per batch json file? Or are each line in a batch file independent from each other?
#Avoiding system prompt repetition on batch API to save cost
trail ore