#Function calling

1 messages · Page 1 of 1 (latest)

fair light
#

@wanton ocean does function calling consume extra tokens?

wanton oceanBOT
#

Hi @fair light. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

Yes, function calling does consume extra tokens. When functions are called, they are injected into the system message in a syntax the model has been trained on. This means functions count against the model's context limit and are billed as input tokens. If you're running into context limits, it's suggested to limit the number of functions or the length of documentation you provide for function parameters. It's also possible to use fine-tuning to reduce the number of tokens used if you have many functions defined [source (https://platform.openai.com/docs/guides/function-calling)].

In addition, instructions, chat history, retrieval of data from your stored files, sending information from the code interpreter, and tool calls also consume tokens. The GPT models are stateless, everything has to be sent to the model from scratch every time you make an API call [source (https://community.openai.com/t/553188)].

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@fair light kapa.ai is still learning and improving, please let us know how it did by reacting below

fair light
#

@wanton ocean what happens if the assistant makes a function call, but the function doesn't return anything?

wanton oceanBOT
#

Let me check the knowledge base for you. One moment! :books:

#

If the assistant makes a function call and the function doesn't return anything, you should still submit a response back to the assistant. This is because the assistant expects a return value after a function call. If your real function doesn't have any output, you can submit an affirmative like a string that says "success". This way, the assistant knows that the function has been executed successfully.

Here's an excerpt from one of the forum posts that explains this:

"When the assistant needs to perform an action, it will call what it thinks is the right function from the pool it was given, and supply input into it as defined by the functions parameters (you can mark which ones are required). The state of the run becomes requires_action when this happens.
What you do is get the name of the function that the assistant has called, and grab the data that it inputted. Then in your program, call the real function of the same name and pass in the data that the assistant gave you. This effectively let’s your program take over and handle the processing. If your real function has output that the assistant is going to need, you can submit it back to the assistant when you have it. In fact you have to. So, if there is no output from your real function, you can always just submit an affirmative like a string that says “success”." [source (https://community.openai.com/t/591372)]

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@fair light kapa.ai is still learning and improving, please let us know how it did by reacting below

fair light
#

@wanton ocean explain fine tuning to me

wanton oceanBOT
#

Let me check the knowledge base for you. One moment! :books:

#

Fine-tuning is a process that allows you to customize a pre-trained model for your specific application. OpenAI's text generation models, such as the GPT series, have been pre-trained on a vast amount of text. However, to use these models effectively for a specific task, you can fine-tune them on your own data, which can lead to higher quality results, the ability to train on more examples than can fit in a prompt, token savings due to shorter prompts, and lower latency requests [source (https://platform.openai.com/docs/guides/fine-tuning)].

Here's how fine-tuning works at a high level:

  1. Prepare and upload training data: This is the data that you want the model to learn from. It should be relevant to the task you want the model to perform.

  2. Train a new fine-tuned model: This involves running a training job where the model learns from your training data.

  3. Evaluate results and go back to step 1 if needed: After training, you should evaluate the performance of the model. If it's not satisfactory, you may need to adjust your training data or parameters and train again.

  4. Use your fine-tuned model: Once you're happy with the performance, you can start using your fine-tuned model for your task.

Fine-tuning is currently available for the following models: gpt-3.5-turbo-1106 (recommended), gpt-3.5-turbo-0613, babbage-002, davinci-002, and gpt-4-0613 (experimental). You can also fine-tune a fine-tuned model, which is useful if you acquire additional data and don't want to repeat the previous training steps [source (https://platform.openai.com/docs/guides/fine-tuning)].

#

It's important to note that fine-tuning requires a careful investment of time and effort. Before deciding to fine-tune a model, it's recommended to first attempt to get good results with prompt engineering, prompt chaining, and function calling. Fine-tuning may not be necessary if the right prompts can improve the model's performance [source (https://platform.openai.com/docs/guides/fine-tuning)].

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@fair light kapa.ai is still learning and improving, please let us know how it did by reacting below

fair light
#

@wanton ocean can the data used for fine-tuning be a list of products with their information?

wanton oceanBOT
#

Let me check the knowledge base for you. One moment! :books:

#

Yes, the data used for fine-tuning can be a list of products with their information. The key is to structure your data in a way that the model can learn from it effectively. For example, if you're fine-tuning a model to answer questions about products, your training data might include pairs of questions and answers about those products.

However, it's important to note that the fine-tuning process involves preparing and uploading your training data, training a new fine-tuned model, evaluating the results, and then using your fine-tuned model. If the results are not satisfactory, you may need to adjust your training data or parameters and train again [source (https://platform.openai.com/docs/guides/fine-tuning)].

Also, remember that the token limits depend on the model you select. For instance, for gpt-3.5-turbo-1106, the maximum content length is 16,385, so each training example is also limited to 16,385 tokens [source (https://community.openai.com/t/563147)].

If you're unsure about how to structure your data for fine-tuning, OpenAI provides a step-by-step guide on how to fine-tune models using the RecipeNLG dataset as an example. This guide might give you some ideas on how to structure your own product data for fine-tuning [source (https://cookbook.openai.com/examples/how_to_finetune_chat_models)].

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@fair light kapa.ai is still learning and improving, please let us know how it did by reacting below

fair light
#

@wanton ocean are there any regulations with the assistant api? Like can you only send a specific amount of messages or is there a maximum token-consumption?

wanton oceanBOT
#

Let me check the knowledge base for you. One moment! :books:

#

Yes, there are certain regulations and limitations when using the Assistant API.

  1. Rate Limits: The API usage is subject to rate limits. You can check the updated guidance on rate limits here (https://platform.openai.com/docs/guides/rate-limits) [source (https://help.openai.com/en/articles/5955598-is-api-usage-subject-to-any-rate-limits)].

  2. Retention Policy: There is a retention policy for Threads/Messages. Currently, the retention for messages and threads is 60 days after which they will automatically be deleted [source (https://community.openai.com/t/487558)].

  3. Beta Limitations: As the Assistants API is still in Beta, there are several known limitations that OpenAI is looking to address. These include support for streaming output, notifications to share object status updates, support for DALL·E or Browsing as a tool, and support for user message creation with images [source (https://community.openai.com/t/534201)].

  4. Token Limit: There is a maximum token limit for each model. For instance, for gpt-3.5-turbo, the maximum limit is 4096 tokens per API call. If a conversation has too many tokens to fit within this limit, you'll have to truncate, omit, or otherwise shrink your text until it fits [source (https://platform.openai.com/docs/guides/conversation)].

Please note that these regulations and limitations are subject to change as OpenAI continues to update and improve the Assistant API.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@fair light kapa.ai is still learning and improving, please let us know how it did by reacting below

fair light
#

@wanton ocean what is the chat completion format?

wanton oceanBOT
#

Let me check the knowledge base for you. One moment! :books:

#

The Chat Completions format is a way to structure your API calls when using OpenAI's chat models. It's designed to facilitate back-and-forth conversation with the model.

In the Chat Completions format, you send a list of messages instead of a single string prompt. Each message in the list has a 'role' and 'content'. The 'role' can be 'system', 'user', or 'assistant', and 'content' is the text of the message from the role.

Here's an example of a chat completion API call:

MODEL = "gpt-3.5-turbo"
response = client.chat.completions.create(
 model=MODEL,
 messages=[
 {"role": "system", "content": "You are a helpful assistant."},
 {"role": "user", "content": "Knock knock."},
 {"role": "assistant", "content": "Who's there?"},
 {"role": "user", "content": "Orange."},
 ],
 temperature=0,
)

[source (https://cookbook.openai.com/examples/how_to_format_inputs_to_chatgpt_models#2-an-example-chat-completion-api-call)]

In this example, the conversation starts with a system message that sets the behavior of the assistant, followed by alternating user and assistant messages. The 'system' role is used to set the behavior of the 'assistant', while the 'user' role is used to provide instructions to the 'assistant'.

The Chat Completions format can also be used to simulate a chat between a user and an assistant by formatting the input accordingly [source (https://platform.openai.com/docs/guides/text-generation)].

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@fair light kapa.ai is still learning and improving, please let us know how it did by reacting below

fair light
#

@wanton ocean fine tuning on gpt 4

wanton oceanBOT
#

Let me check the knowledge base for you. One moment! :books:

#

Fine-tuning on GPT-4 is currently available only through an experimental access program. Preliminary results indicate that GPT-4 fine-tuning requires more work to achieve meaningful improvements over the base model compared to the substantial gains achieved with GPT-3.5 fine-tuning. As the quality and safety for GPT-4 fine-tuning improves, developers actively using GPT-3.5 fine-tuning will be presented with an option to apply to the GPT-4 program within their fine-tuning console [source (https://help.openai.com/en/articles/7127982-can-i-fine-tune-on-gpt-4)].

However, it's important to note that fine-tuning the image capabilities of GPT-4 is not supported at this time [source (https://platform.openai.com/docs/guides/vision)].

If you're eligible and interested in fine-tuning GPT-4, you can request access via the fine-tuning UI when creating a new fine-tuning job [source (https://platform.openai.com/docs/guides/fine-tuning)].

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@fair light kapa.ai is still learning and improving, please let us know how it did by reacting below

fair light
#

@wanton ocean what is validation data at fine-tuning?

wanton oceanBOT
#

Let me check the knowledge base for you. One moment! :books:

#

Validation data is a subset of your dataset that is used during the fine-tuning process of a machine learning model. It serves a critical role in preventing overfitting and helping you understand how well your model is likely to perform on unseen data.

Here's how it works:

  1. Model Selection: The validation set is used to estimate the prediction error for model selection. This helps in choosing the best model during the training process.

  2. Model Assessment: Once a final model is chosen, the validation set is used to estimate its prediction error (generalization error) on new data.

In a data-rich situation, the best approach is to randomly divide the dataset into three parts: a training set, a validation set, and a test set. The training set is used to fit the models; the validation set is used to estimate prediction error for model selection; the test set is used for assessment of the generalization error of the final chosen model [source (https://community.openai.com/t/434146)].

The validation file should be the same type of example conversations as you are training on, in the same type of file format. The held-out examples should be of the quality where you could shuffle all your questions randomly and put any 10% of them into a validation file. The validation file lets you see a second benchmark produced during fine-tune: not just how much the learning on the training set has progressed, but how well similar questions are inferred [source (https://community.openai.com/t/452061)].

In the context of fine-tuning with OpenAI, you would provide a validation file in the same format as your training file when creating a fine-tuning job [source (https://community.openai.com/t/421784)].

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@fair light kapa.ai is still learning and improving, please let us know how it did by reacting below

fair light
#

@wanton ocean what is the connection of fine-tuning and the knowledgebase?

wanton oceanBOT
#

Let me check the knowledge base for you. One moment! :books:

#

Fine-tuning and a knowledge base are both methods to customize and enhance the performance of an AI model, but they serve different purposes and work in different ways.

Fine-tuning is a process where a pre-trained model (like GPT-3 or GPT-4) is further trained on a specific dataset to adapt it to a particular task or to improve its performance on that task. The fine-tuning process does not give the model new knowledge per se, but rather refines its ability to generate responses based on the style and content of the training data [source (https://platform.openai.com/docs/guides/fine-tuning), source (https://community.openai.com/t/554222)].

On the other hand, a knowledge base is a structured database of information that an AI model can use to retrieve and provide specific information. In the context of AI models, a knowledge base can be used in conjunction with embeddings-based search to retrieve relevant information for answering within a specific new knowledge domain. This can help the model to generate more informed and up-to-date responses [source (https://community.openai.com/t/386986)].

In summary, fine-tuning is about adjusting the model's behavior based on a specific dataset, while a knowledge base is about providing the model with a source of specific information that it can use to answer queries. They can be used separately or together, depending on the requirements of your application.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@fair light kapa.ai is still learning and improving, please let us know how it did by reacting below

fair light
#

@wanton ocean how can a model be trained on a dataset?

wanton oceanBOT
#

Let me check the knowledge base for you. One moment! :books:

#

Training a model on a dataset involves several steps:

  1. Prepare your dataset: This is the data that you want the model to learn from. It should be relevant to the task you want the model to perform. For instance, if you're training a chat model, you might create a diverse set of demonstration conversations that are similar to the conversations you will ask the model to respond to at inference time in production. Each example in the dataset should be a conversation in the same format as the Chat Completions API, specifically a list of messages where each message has a role, content, and optional name. Some of the training examples should directly target cases where the prompted model is not behaving as desired, and the provided assistant messages in the data should be the ideal responses you want the model to provide [source (https://platform.openai.com/docs/guides/fine-tuning)].

  2. Upload your dataset: Once your dataset is prepared, you need to upload it to the platform where you'll be training your model. For instance, if you're using OpenAI's fine-tuning service, you would upload your dataset through the OpenAI platform.

  3. Train your model: This involves running a training job where the model learns from your training data. The specifics of this step can vary depending on the platform and the model you're using. For instance, with OpenAI's fine-tuning service, you would create a new fine-tuning job through the OpenAI platform.

  4. Evaluate your model: After training, you should evaluate the performance of your model. This typically involves testing the model on a separate validation dataset and measuring its performance using appropriate metrics. If the performance is not satisfactory, you may need to adjust your training data or parameters and train again.

#
  1. Use your model: Once you're happy with the performance, you can start using your model for your task.

Remember that training a model requires a careful investment of time and effort. It's important to monitor the training process and adjust your approach as needed based on the performance of your model.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@fair light kapa.ai is still learning and improving, please let us know how it did by reacting below

fair light
#

@wanton ocean i have a .txt file with a list of products and their information. What can i do to achieve the best results?

wanton oceanBOT
#

Let me check the knowledge base for you. One moment! :books: