Hi @swift marten. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
#Unexpected High Token Usage in GPT-4o API Response
fading jacinthBOT
Hello,
Based on the information provided in the knowledge sources, there could be several reasons for the unexpectedly high token usage in your GPT-4o API response.
-
Image Size: The token usage for images is based on the detail and tiles, but this is not calculated. The internal resizing of the smallest side of the image is what makes even up to 1024x1024 take 4 “tiles” that are 512x512 in the detail:high mode. For an image 640x640, detail:low will make it only cost 85 tokens with the image being downsized to 512x512 internally. If your image is larger than this, it could result in higher token usage (source (https://community.openai.com/t/743297)).
-
Input Structure: The token usage of input is not calculated based on the sent JSON, but on the language the AI receives, needing the role, name, message, tools to be constructed into the plain text with the overhead of unjoinable container tokens. Then images (source (https://community.openai.com/t/758778)).
-
File Search Tool: If you have the
file_searchtool enabled along with an attached vector store, this could explain the high input tokens (source (https://community.openai.com/t/903755)). -
GPT-4o vs GPT-4 Turbo: It has been observed that GPT-4o uses more tokens by default compared to GPT-4 Turbo. This could be due to the fact that GPT-4o automatically uses the file search, but GPT-4 won’t do it on its own unless you turn on the force to use the file_search (source (https://community.openai.com/t/763075)).