misty kite
Ork
Nov '23
Hello everyone,
I’ve recently started working with the new version of the OpenAI library for Python, and I’m a bit stuck on implementing asynchronous calls properly. I’m not entirely sure how to apply async with the latest features of the openai library.
Could someone please provide a detailed explanation or example of how to use the async functionalities in the new library?
night vergeBOT
Hi @misty kite. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:
Sure, I can provide some guidance based on the information available in the extracts.
In the latest version of the OpenAI Python library, the acreate method has been removed. Instead, you can use the AsyncOpenAI class to make asynchronous calls. Here's an example of how you can use it:
from openai import AsyncOpenAI
client = AsyncOpenAI()
response = await client.chat.completions.create(
model="gpt-4",
messages=messages,
tools=functions,
temperature=0.0,
tool_choice=None
)
This code was found in a forum post (https://community.openai.com/t/478439) on the OpenAI community site.
If you're using Azure, you can use the AsyncAzureOpenAI class in a similar way. Here's an example:
from openai import AsyncAzureOpenAI
client = AsyncAzureOpenAI()
await client.chat.completions.create(...)
This information was found in a Github issue (https://github.com/openai/openai-python/issues/712) on the OpenAI Python API library.
Please note that you might encounter some issues when using the AsyncOpenAI class at scale. For example, creating new clients per request might lead to a memory leak, and reusing a global AsyncOpenAI client across requests might lead to httpx.PoolTimeouts. This information was found in another Github issue (https://github.com/openai/openai-python/issues/874) on the OpenAI Python API library.
If you're using Windows and encounter a segmentation fault, you can try setting the event loop policy to WindowsSelectorEventLoopPolicy as a workaround:
asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
This information was found in yet another Github issue (https://github.com/openai/openai-python/issues/791) on the OpenAI Python API library.