Do the string descriptions in the function calling part only deduct tokens when they are used or are they always deducted?
#Function Calling Token Usage
ivory jungle
pearl solarBOT
Hi @ivory jungle. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
The string descriptions in the function calling part are always deducted as tokens. According to the knowledge source on function calling, functions are injected into the system message in a syntax the model has been trained on. This means that functions, including their descriptions, count against the model's context limit and are billed as input tokens. Therefore, even if the descriptions are not actively used in a particular interaction, they still contribute to the token count.