#gpt4 vision+function calling

1 messages · Page 1 of 1 (latest)

crude atlas
#

how do i use gpt4 vision with function calling

night gazelleBOT
#

Hi @crude atlas. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

Based on the information provided in the extracts, there isn't a direct example of using GPT-4 Vision with function calling. However, we can infer from the available information how you might approach this.

Firstly, it's important to note that not all models support function calling. According to the Supported Models (https://platform.openai.com/docs/guides/function-calling) section in the OpenAI documentation, GPT-4 does support function calling.

To use function calling with GPT-4, you would typically describe functions in an API call and the model would generate a JSON object containing arguments to call one or many functions. The Chat Completions API does not call the function; instead, the model generates JSON that you can use to call the function in your code.

Here's a Python example of how you might use function calling with GPT-4, based on the OpenAPI specification example (https://cookbook.openai.com/examples/function_calling_with_an_openapi_spec#how-to-call-these-functions-with-gpt):

def get_openai_response(functions, messages):
 return client.chat.completions.create(
 model="gpt-4",
 functions=functions,
 function_call="auto", # "auto" means the model can pick between generating a message or calling a function.
 temperature=0,
 messages=messages,
 )

In this example, functions would be a list of function definitions and messages would be a list of messages to be processed by the model.

As for GPT-4 Vision, the OpenAI Vision FAQ (https://platform.openai.com/docs/guides/vision) mentions that you can find details about evaluations, preparation, and mitigation work in the GPT-4 with Vision system card. However, the extracts do not provide specific details on how to use GPT-4 Vision with function calling.

#

@crude atlas kapa.ai is still learning and improving, please let us know how it did by reacting below

crude atlas
#

does this have information post 1106?

restive veldt
#

i think it failed this one too

#

"Currently, GPT-4 Turbo with vision does not support the message.name parameter, functions/tools, response_format parameter, and we currently set a low max_tokens default which you can override."

crude atlas
# restive veldt i think it failed this one too

im a difficult customer haha. i knew the vision model doesn't support function calling directly, i was hoping it would offer a smart work around, like using another model with function calling to call a function which uses the vision model to answer a specific question about an image and feed the results back to the function calling model.

#

i was just curious how this new bot responds to certain questions