#Async Function Calling

1 messages · Page 1 of 1 (latest)

uncut stag
#

Why does client.beta.threads.runs.submit_tool_outputs_stream() return nothing and client.beta.threads.runs.stream() returns the actual response from OpenAI Assistant API.

buoyant bluffBOT
#

Hi @uncut stag. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

The client.beta.threads.runs.submit_tool_outputs_stream() and client.beta.threads.runs.stream() are two different methods used for different purposes in the OpenAI Assistant API.

The client.beta.threads.runs.stream() method is used to create a stream of events from a specific run of an assistant. This stream can be used to listen for various events that occur during the run, such as when a new message is created, when a tool call is done, or when the run requires action. This method returns a stream of events, which is why you see a response from the OpenAI Assistant API.

On the other hand, the client.beta.threads.runs.submit_tool_outputs_stream() method is used to submit the outputs from tool calls when a run requires action. This method is used when the run status is "requires_action" and the required action type is "submit_tool_outputs". This method does not return a response because its purpose is to submit data to the API, not to retrieve data from it.

Here is an example of how these methods might be used together:

let stream = openai.beta.threads.runs.stream(thread_id, { assistant_id })
.on("event", async ({ event, data }) => {
    if (
        event === "thread.run.requires_action" &&
        data.status === "requires_action" &&
        data.required_action?.type === "submit_tool_outputs"
    ) {
        const tool_outputs = await Promise.all( ... );

        stream = openai.beta.threads.runs.submitToolOutputsStream(thread_id, data.id, {
            tool_outputs,
        });
        return new Response(stream.toReadableStream());  // THIS!!!
    }
});
return new Response(stream.toReadableStream());

In this example, the stream() method is used to listen for events from the run, and when the run requires action, the submitToolOutputsStream() method is used to submit the required tool outputs.

#

@uncut stag kapa.ai is still learning and improving, please let us know how it did by reacting below

uncut stag
#

rewrite this using the python sdk

#

@buoyant bluff rewrite this using the python sdk

buoyant bluffBOT
#

Let me check the knowledge base for you. One moment! :books:

#

Sure, here's how you might use these methods together in Python:

from openai import OpenAI

openai = OpenAI("your-api-key")

# Create and poll a new run
run = openai.beta.threads.runs.create_and_poll(
    thread_id=thread_id,
    assistant_id=assistant_id,
)

# Check the status of the run
if run.status == 'requires_action' and run.required_action.type == 'submit_tool_outputs':
    # Prepare your tool outputs here
    tool_outputs = [...]

    # Submit the tool outputs
    run = openai.beta.threads.runs.submit_tool_outputs(
        thread_id=thread_id,
        run_id=run.id,
        tool_outputs=tool_outputs
    )

In this example, the create_and_poll() method is used to create a new run and poll for its status. If the run requires action, the submit_tool_outputs() method is used to submit the required tool outputs.

Please note that you'll need to replace "your-api-key", thread_id, assistant_id, and tool_outputs with your actual API key, thread ID, assistant ID, and tool outputs respectively.

Sources:

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@uncut stag kapa.ai is still learning and improving, please let us know how it did by reacting below

uncut stag
#

@buoyant bluff can you rewrite this code with async and AsyncEventHandler?
from typing_extensions import override
from openai import AssistantEventHandler

class EventHandler(AssistantEventHandler):
@override
def on_event(self, event):
# Retrieve events that are denoted with 'requires_action'
# since these will have our tool_calls
if event.event == 'thread.run.requires_action':
run_id = event.data.id # Retrieve the run ID from the event data
self.handle_requires_action(event.data, run_id)

def handle_requires_action(self, data, run_id):
  tool_outputs = []
    
  for tool in data.required_action.submit_tool_outputs.tool_calls:
    if tool.function.name == "get_current_temperature":
      tool_outputs.append({"tool_call_id": tool.id, "output": "57"})
    elif tool.function.name == "get_rain_probability":
      tool_outputs.append({"tool_call_id": tool.id, "output": "0.06"})
    
  # Submit all tool_outputs at the same time
  self.submit_tool_outputs(tool_outputs, run_id)

def submit_tool_outputs(self, tool_outputs, run_id):
  # Use the submit_tool_outputs_stream helper
  with client.beta.threads.runs.submit_tool_outputs_stream(
    thread_id=self.current_run.thread_id,
    run_id=self.current_run.id,
    tool_outputs=tool_outputs,
    event_handler=EventHandler(),
  ) as stream:
    for text in stream.text_deltas:
      print(text, end="", flush=True)
    print()

with client.beta.threads.runs.stream(
thread_id=thread.id,
assistant_id=assistant.id,
event_handler=EventHandler()
) as stream:
stream.until_done()

buoyant bluffBOT
#

Let me check the knowledge base for you. One moment! :books:

#

Sure, here's how you might rewrite the code using async and AsyncEventHandler:

from typing_extensions import override
from openai import AsyncAssistantEventHandler

class EventHandler(AsyncAssistantEventHandler):
    @override
    async def on_event(self, event):
        if event.event == 'thread.run.requires_action':
            run_id = event.data.id
            await self.handle_requires_action(event.data, run_id)

    async def handle_requires_action(self, data, run_id):
        tool_outputs = []

        for tool in data.required_action.submit_tool_outputs.tool_calls:
            if tool.function.name == "get_current_temperature":
                tool_outputs.append({"tool_call_id": tool.id, "output": "57"})
            elif tool.function.name == "get_rain_probability":
                tool_outputs.append({"tool_call_id": tool.id, "output": "0.06"})

        await self.submit_tool_outputs(tool_outputs, run_id)

    async def submit_tool_outputs(self, tool_outputs, run_id):
        async with client.beta.threads.runs.submit_tool_outputs_stream(
            thread_id=self.current_run.thread_id,
            run_id=self.current_run.id,
            tool_outputs=tool_outputs,
            event_handler=EventHandler(),
        ) as stream:
            async for text in stream.text_deltas:
                print(text, end="", flush=True)
            print()

async with client.beta.threads.runs.stream(
    thread_id=thread.id,
    assistant_id=assistant.id,
    event_handler=EventHandler()
) as stream:
    await stream.until_done()
#

In this version of the code, the async keyword is used to define asynchronous functions, and the await keyword is used to pause execution of the function until the awaited task is complete. The AsyncAssistantEventHandler class is used instead of AssistantEventHandler, and the async for statement is used to iterate over asynchronous iterators.

Please note that you'll need to replace client, thread.id, and assistant.id with your actual OpenAI client, thread ID, and assistant ID respectively.

Sources:

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@uncut stag kapa.ai is still learning and improving, please let us know how it did by reacting below

uncut stag
buoyant bluffBOT
#

Let me check the knowledge base for you. One moment! :books:

#

Sure, here's how you might rewrite the code using async:

from openai import OpenAI

openai = OpenAI("your-api-key")

async def main():
    response = await openai.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": "What’s in this image?"},
                    {
                        "type": "image_url",
                        "image_url": {
                            "url": "<https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",>
                        },
                    },
                ],
            }
        ],
        max_tokens=300,
    )

    print(response.choices[0])

# Run the main function
import asyncio
asyncio.run(main())

In this version of the code, the async keyword is used to define an asynchronous function, and the await keyword is used to pause execution of the function until the awaited task is complete. The asyncio.run() function is used to run the top-level function main().

Please note that you'll need to replace "your-api-key" with your actual OpenAI API key.

Sources:

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@uncut stag kapa.ai is still learning and improving, please let us know how it did by reacting below

uncut stag
#

@buoyant bluff how to do chat completions with AsyncOpenAI as the client

buoyant bluffBOT
#

Let me check the knowledge base for you. One moment! :books:

#

To perform chat completions with AsyncOpenAI as the client, you can use the chat.completions.create() method in an asynchronous context. Here's an example:

import asyncio
from openai import AsyncOpenAI

async def main():
    client = AsyncOpenAI(
        api_key="my_api_key",
        base_url="<https://<MY_AZURE_OPENAI>.openai.azure.com/openai/deployments/<MY_DEPLOYMENT>",>
        default_query={"api-version": "2024-02-01"},
    )
    response = await client.chat.completions.create(
        model="my_deployment",
        messages=[{"role": "user", "content": "What is Python?"}],
    )

    print(response.choices[-1].message.content)

# Run the main function
asyncio.run(main())

In this example, an instance of AsyncOpenAI is created with the necessary parameters. The chat.completions.create() method is then used to create a chat completion. The await keyword is used to pause execution of the function until the awaited task is complete. The asyncio.run() function is used to run the top-level function main().

Please note that you'll need to replace "my_api_key", <MY_AZURE_OPENAI>, <MY_DEPLOYMENT>, and "my_deployment" with your actual API key, Azure OpenAI endpoint, Azure OpenAI deployment name, and deployment model respectively.

Sources:

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@uncut stag kapa.ai is still learning and improving, please let us know how it did by reacting below

uncut stag
#

@buoyant bluff Using this code, how can I get the arguments from function calling and use it in get_current_temperature?
from typing_extensions import override
from openai import AsyncAssistantEventHandler

class EventHandler(AsyncAssistantEventHandler):
@override
async def on_event(self, event):
if event.event == 'thread.run.requires_action':
run_id = event.data.id
await self.handle_requires_action(event.data, run_id)

async def handle_requires_action(self, data, run_id):
    tool_outputs = []

    for tool in data.required_action.submit_tool_outputs.tool_calls:
        if tool.function.name == "get_current_temperature":
            tool_outputs.append({"tool_call_id": tool.id, "output": "57"})
        elif tool.function.name == "get_rain_probability":
            tool_outputs.append({"tool_call_id": tool.id, "output": "0.06"})

    await self.submit_tool_outputs(tool_outputs, run_id)

async def submit_tool_outputs(self, tool_outputs, run_id):
    async with client.beta.threads.runs.submit_tool_outputs_stream(
        thread_id=self.current_run.thread_id,
        run_id=self.current_run.id,
        tool_outputs=tool_outputs,
        event_handler=EventHandler(),
    ) as stream:
        async for text in stream.text_deltas:
            print(text, end="", flush=True)
        print()

async with client.beta.threads.runs.stream(
thread_id=thread.id,
assistant_id=assistant.id,
event_handler=EventHandler()
) as stream:
await stream.until_done()

buoyant bluffBOT
#

Let me check the knowledge base for you. One moment! :books:

#

In the provided code, the function get_current_temperature is being called, but the arguments from the function call are not being used. To use the arguments from the function call, you would need to extract them from the tool object in the handle_requires_action method.

Here's how you can modify the handle_requires_action method to extract and use the arguments:

import json

async def handle_requires_action(self, data, run_id):
    tool_outputs = []

    for tool in data.required_action.submit_tool_outputs.tool_calls:
        if tool.function.name == "get_current_temperature":
            # Extract the arguments from the function call
            function_args = json.loads(tool.function.arguments)

            # Use the arguments in the get_current_temperature function
            output = get_current_temperature(**function_args)

            tool_outputs.append({"tool_call_id": tool.id, "output": output})
        elif tool.function.name == "get_rain_probability":
            tool_outputs.append({"tool_call_id": tool.id, "output": "0.06"})

    await self.submit_tool_outputs(tool_outputs, run_id)

In this modified version of the method, the json.loads() function is used to parse the arguments from the function call into a Python dictionary. The ** operator is then used to unpack the dictionary and pass the arguments to the get_current_temperature function.

Please note that you'll need to define the get_current_temperature function to accept the arguments and return the desired output. The function could look something like this:

def get_current_temperature(location, unit="fahrenheit"):
    """Get the current temperature in a given location"""
    # Your code to get the temperature goes here
    ...
    return temperature