Open AI Assistants API Pricing

[alpine pagoda]

Hey, can someone please let me know how to work out the price of 1,000 users sending 1 message each to an OpenAI Assistant via API please?

[cedar prism]

You can find the price in the price page of official website

I remember the price counts like the other api

[crude solstice]

see https://openai.com/pricing under Assistants API. Let me know if you need further help

[alpine pagoda]

thanks @cedar prism @crude solstice

it just says Retrieval $0.20 / GB / assistant / day (free until 03/01/2024)

Doesn't really tell me exactly how much?

Example, how much would 1,000 requests to it asking ("What's the Annual Leave Policy?") cost?

[crude solstice]

@alpine pagoda
When you use an LLM you pay the tokens of text you send, in your case your question and the response tokens per token. For that refer to the pricing on models on the same page. See here to count how many tokens a text is here https://platform.openai.com/tokenizer

[alpine pagoda]

Ok, so 7 tokens on GPT-4 in Assistants API?

I'm so confused

[dusk nimbus]

you did not say how long messages the users send or what the ai responds. user count 1000 * in/out token count per message + message history tokens * model token price

[alpine pagoda]

Ah, so is there any equasion that I can follow to generate an estimate?

[dusk nimbus]

if you use assistants and the users will have a long chat with the bot then you can estimate the cost be your model max size per message * price

[crude solstice]

To get an estimate you will need an average number of tokens for your question and for its response. Then you can, based on the pricings on the page, know how much it will cost on average. It's important to understand that you pay exactly for what you use.

The assistants api has no fees unless you get files or use the code interpreter

[alpine pagoda]

Ah, so it'll charge more if I upload more files for it to recall information from?

[crude solstice]

Yup it costs 0.20$/gb/day

However that starts the 1 march, until then it's frez

Plus the normal tokens as discussed before obviously

[alpine pagoda]

so that's $0.20/gb of data I have uploaded to the assistant or amount that gets queried per day?

[dusk nimbus]

you still pay for the model token usage and assistants fill the whole context all the time and burn money fast 😛

[crude solstice]

It only billes you once but resets every day so you pay it again every day

[crude solstice]

Yeah that’s why I added a message after I noticed it might not be very clear

[alpine pagoda]

@dusk nimbus @crude solstice

so are you saying Assistants API is not cost efficient? What would you reccomend instead?

[crude solstice]

It is. As a fact, open ai handles the messages for you instead of you having to store a context. The pricing for the llms themselves is the same. It’s just that if you want to use retrieval, you need to pay for that as well

[alpine pagoda]

So how would you recommend I go about training an AI model on my companies data for my staff members to ask the AI to get quick answers? Via API?

Most cost efficiency

And easily really

[crude solstice]

I would not advise that if what you want to do is have the model answer from a knowledge base. Fine tuning is more about the form of the answer than it’s actual content. Learn more here https://platform.openai.com/docs/guides/fine-tuning

Retrieval is better imo if you want the model to answer from a knowledge base

[alpine pagoda]

Ah is Assistants the only way to retrieve data in the way I would like?

Or any alternatives that are better in your opinion?

I did try fine tuning but it came out with a lot of irrelevant answers

[crude solstice]

You can also try with embeddings, read more about it here https://cookbook.openai.com/examples/vector_databases/qdrant/using_qdrant_for_embeddings_search

[alpine pagoda]

wow so complex

what's the prise difference between this and assistants please?

[crude solstice]

I don't know, and I don't believe it's easy to calculate. If I believe correctly, this is what assistants do in the background under the hood. If I were at your place I would use the assistants api

but this is up to you to decide

[alpine pagoda]

Ok, so let's stick with OpenAI assistants.

let's say the message sent is "what is the annual leave policy?" - 7 tokens

the reply from the API is let's say we limit it at 100 tokens.

We use GPT-4 with the Assistants API.

How much would that cost with retrieval too?

[crude solstice]

For 1,000 users asking "What's the annual leave policy?" (7 tokens) and getting back 100 tokens in response using GPT-4:

• Input: 7 tokens x 1,000 users = 7,000 tokens. Cost = 7,000 tokens * $0.03/1,000 = $0.21.
• Output: 100 tokens x 1,000 users = 100,000 tokens. Cost = 100,000 tokens * $0.06/1,000 = $6.

Total token cost = $0.21 + $6 = $6.21.

For retrieval, if you're using 1GB of data, it's $0.20/GB/day. So, if we add that, your total would be $6.21 for the tokens plus $0.20 for the retrieval, making it $6.41 in total. You can then change that based on how large the average file would be in your context.

Prices can change, so check OpenAI's pricing page for the latest. Hope that helps clear things up! Let me know if you need more clarifications.

[alpine pagoda]

Ah yes that's amazing!

So, the 1GB of data, say I have 2GB data, will it charge for 2GB's worth if it only needs to read one 5MB document to get the answer?

[crude solstice]

Yes since it needs to embed the entire document I believe

Paid every day

[alpine pagoda]

Ah ok. Thank you!

And say, those 1,000 users messaged on different days would the total token cost still be the same?

[alpine pagoda]

Every day that the assistant is used? Or every day the assistant is there?

[crude solstice]

Every day it's used I believe.

[crude solstice]

You can read more here
https://platform.openai.com/docs/assistants/tools/retrieval-pricing

[alpine pagoda]

Ok, tysm!!!

Final question, at my company before we're able to start using OpenAI, we need to get them to fill a security document, etc... How do I get in contact with them so we can kick this off?

[crude solstice]

You should probably contact the sales team here https://openai.com/contact-sales

[alpine pagoda]

Mh I did try but no reply in a week :(

[crude solstice]

Actually, If the file is uploaded in the assistant itself I wonder if the fee only occurs once

[crude solstice]

you can also try [e-mail]

[alpine pagoda]

Thanks

[crude solstice]

no problem let me know if you need anything else

[alpine pagoda]

@crude solstice one more question, do you know how long user messages are sored in a thread for before they're deleted?

[crude solstice]

According to this post in the openai forum, they last 60 days after the thread's last activity. However, I did not find any official source stating said duration.

[alpine pagoda]

thanks!!

[alpine pagoda]

Sorry, me again @crude solstice.

Is this $6.21 per message to the API? Or is this for all 1k requests?

[crude solstice]

It would be 6.21 of tokens per 1000 requests, so 0.00621$ per request in tokens

[alpine pagoda]

Ah ok, I'm a little confused why you did /1,000 in the original workings out? Would you mind explaining the working out please?

[crude solstice]

I just did that because the original pricings are already in per 1000 tokens, on the pricing page, you will see the pricings per thousand tokens, not per tokens, or else the price would be very small. Same here, it's hard to omagine what 0.00621$ really represent, as opposed of 6.21$

[alpine pagoda]

ah okk

[spring mural]

that kinda reminds me of when i first used the api.. i was waiting for the usage to go up and it just wouldn't and i thought i had free access 👀 🤣

[alpine pagoda]

@crude solstice So this would be correct?

[alpine pagoda]

lol

[crude solstice]

Yup, make sure to count a monthly / daily cost for the file. That cost is constantly deducted, until the file is deleted.

for reference : https://community.openai.com/t/assistant-retrieval-pricing/601158

[alpine pagoda]

I think this change sums it up nicer?

[crude solstice]

yup, you migth wanna add a ~100 token example response for scale, but that is absolutely correct

[alpine pagoda]

amazing thank you! I've got something meaningful to go back with!!