Assistant API Token Calculations

[versed sparrow]

@lime lynx I am utilizing the OpenAI Assistant API, and I am trying to better predict useage. Specifically how are tokens calculated when using the assistant api? Please be as detailed as possiable. IN addition. Take in this example script. `import OpenAI from 'openai';
import dotenv from 'dotenv';
// Vexstorm Studio Modules
const configuration = require('./configuration/config');
dotenv.config();

const openai = new OpenAI(process.env.OPENAI_API_KEY);

async function createAssistant() {
const assistant = await openai.beta.assistants.create({
name: 'My Assistant',
instructions: 'You are an assistant. Help answer questions and provide information.',
tools: [{ type: 'code_interpreter' }],
model: 'gpt-4-1106-preview',
});
return assistant;
}

async function createThread() {
const thread = await openai.beta.threads.create();
return thread;
}

async function sendMessage(threadId, messageContent) {
const message = await openai.beta.threads.messages.create(threadId, {
role: 'user',
content: messageContent,
});
return message;
}

async function runThread(threadId, assistantId) {
const run = await openai.beta.threads.runs.create(threadId, {
assistant_id: assistantId,
instructions: 'Please provide detailed and accurate responses.',
});

let completedRun;
do {
    await new Promise(resolve => setTimeout(resolve, 1000));
    completedRun = await openai.beta.threads.runs.retrieve(threadId, run.id);
} while (completedRun.status === 'in_progress');

if (completedRun.status !== 'completed') {
    console.log('interactWithAssistant - did not complete properly');
    throw new Error(`Run did not complete successfully. Status: ${completedRun.status}`);
}

return completedRun;

}

async function interactWithAssistant() {
try {
const assistant = await createAssistant();
const thread = await createThread();
await sendMessage(thread.id, 'What is the meaning of life?');
const runResult = await runThread(thread.id, assistant.id);
console.log('Run result:', runResult);
} catch (error) {
console.error('Error in interactWithAssistant:', error.message);
}
}

interactWithAssistant();` how ouwld i amend it, to print out in a console.log, the count of tokens used per response from the api? Please generate full code to realize a way to see token count being returned. Please generate full code to predict token count, based on the users question, (prompt), the instruction to the assistant, and anything else that might increase token count please

[lime lynx]

Hi @versed sparrow. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

---

Based on the information provided in the extracts, it's important to note that OpenAI's API does not directly provide a function to calculate the number of tokens in a given prompt or response for a given model. However, there are ways to estimate this.

Tokens can be thought of as pieces of words. Before the API processes the prompts, the input is broken down into tokens. These tokens are not cut up exactly where the words start or end - tokens can include trailing spaces and even sub-words. Here are some helpful rules of thumb for understanding tokens in terms of lengths:

• 1 token ~= 4 chars in English
• 1 token ~= ¾ words
• 100 tokens ~= 75 words
• 1-2 sentence ~= 30 tokens
• 1 paragraph ~= 100 tokens
• 1,500 words ~= 2048 tokens

To further explore tokenization, you can use OpenAI's interactive Tokenizer tool (https://beta.openai.com/tokenizer), which allows you to calculate the number of tokens and see how text is broken into tokens. Alternatively, if you'd like to tokenize text programmatically, use Tiktoken (https://github.com/openai/tiktoken) as a fast BPE tokenizer specifically used for OpenAI models. Other such libraries you can explore as well include transformers (https://huggingface.co/transformers/model_doc/gpt2.html#gpt2tokenizerfast) package for Python or the gpt-3-encoder (https://www.npmjs.com/package/gpt-3-encoder) package for node.js.

In the context of your script, you would need to implement a token counting function using one of the methods mentioned above. However, the extracts do not provide a specific code snippet for this. You might consider using the tiktoken library to count tokens in your input and output strings, but please note that this is an estimation and might not be 100% accurate.

Here's a rough example of how you might implement this in your script:

// Import the necessary libraries
const Tiktoken = require('tiktoken'); // Assuming there's a Node.js equivalent of Tiktoken

// Initialize the tokenizer
const tokenizer = new Tiktoken();

// In your sendMessage function
async function sendMessage(threadId, messageContent) {
    const message = await openai.beta.threads.messages.create(threadId, {
        role: 'user',
        content: messageContent,
    });

    // Estimate the number of tokens in the message
    const tokenCount = tokenizer.countTokens(messageContent);
    console.log(`Estimated token count for message: ${tokenCount}`);

    return message;
}

Please note that this is a rough example and might not work as expected. You would need to find a suitable Node.js library for token counting or implement your own function based on the rules of thumb provided above.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: