Moderation API async best practices

[frosty nacelle]

My understanding of the content policy for the OpenAI API is that it applies to both generating AND sending inappropriate content to a model, which would violate OpenAI's policies and potentially lead to suspensions/bans.

This recent OpenAI cookbook article (https://cookbook.openai.com/examples/how_to_use_moderation) suggests using the moderation endpoint asynchronously with the other endpoints as a best practice(reduced latency, yada yada).

Wouldn't that potentially allow inappropriate content to flow through to one of the models in the event a completions call finished before a call to the moderation endpoint? If so, wouldn't it be better to always synchronously call the moderation endpoint first before passing any information to one of the models to aide in prevention of violating the policy?

[velvet jackal]

Hi @frosty nacelle. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

---

You raise a valid concern about the potential for inappropriate content to reach the language model (LLM) when using the Moderation API asynchronously. The OpenAI Cookbook indeed suggests using the Moderation API asynchronously to minimize latency, but it also acknowledges the potential for increased costs and the risk of inappropriate content reaching the LLM.

In the example provided (https://cookbook.openai.com/examples/how_to_use_moderation#1-input-moderation), the function execute_chat_with_moderation is designed to run the LLM's get_chat_response and the check_expression moderation function in parallel. If the moderation gets triggered (returns True), it sends back a placeholder response, otherwise, it sends back the LLM response. This is done to balance the benefits of reduced latency with the potential for increased expenses and the risk you mentioned.

However, if you are more concerned about strictly preventing any inappropriate content from reaching the LLM, you might indeed prefer to use the Moderation API synchronously before passing any information to the models. This approach is demonstrated in the Input Moderation Workflow (https://cookbook.openai.com/examples/how_to_use_moderation#workflow) where the Moderation API is used to check user input for any potentially unsafe content before it is sent to the LLM.

Here's the relevant code snippet:

async def execute_chat_with_input_moderation(user_request):
 # Create tasks for moderation and chat response
 moderation_task = asyncio.create_task(check_moderation_flag(user_request))
 chat_task = asyncio.create_task(get_chat_response(user_request))

 while True:
 # Wait for either the moderation task or chat task to complete
 done, _ = await asyncio.wait(
 [moderation_task, chat_task], return_when=asyncio.FIRST_COMPLETED
 )

If moderation task is not completed, wait and continue to the next iteration

if moderation_task not in done:
await asyncio.sleep(0.1)
continue

In this workflow, the moderation task is checked first, and if it's not completed, the function waits and continues to the next iteration, effectively preventing the chat task from proceeding if the moderation task flags the content.

Ultimately, the choice between synchronous and asynchronous use of the Moderation API depends on your specific needs and priorities.
----------------------
Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:
----------------------

@frosty nacelle kapa.ai is still learning and improving, please let us know how it did by reacting below

[frosty nacelle]

@velvet jackal I thought you caught me, because I often have bad reading comprehension. The example you've referenced (https://cookbook.openai.com/examples/how_to_use_moderation#workflow) does not demonstrate synchronously calling the moderation endpoint, handling the response, and then sending making a chat request with the passing content. It makes both calls at the same time asynchronously... so potentially bad content could reach a model endpoint before the moderation endpoint returns a result. This would only stop follow-up content in the same conversation

Technically you answered my question, but I'm giving you a thumbs down because the referenced example code does not accurately address the content policy concerns discussed.

[velvet jackal]

Let me check the knowledge base for you. One moment! :books:

---

I apologize for the confusion. You're correct that the example provided in the OpenAI Cookbook (https://cookbook.openai.com/examples/how_to_use_moderation#workflow) runs the moderation check and the chat response generation asynchronously, which means they start at the same time. This could potentially allow content to reach the language model before the moderation check is completed.

If you want to ensure that no content reaches the language model before passing the moderation check, you would indeed need to call the moderation endpoint synchronously before making a chat request. This would involve waiting for the moderation check to complete and only proceeding with the chat request if the content passes the moderation check.

Unfortunately, the provided knowledge sources do not include an example of this synchronous approach. However, the concept would be similar to the asynchronous example, but without creating the chat task until after the moderation task has completed and confirmed the content is appropriate.

I appreciate your feedback and I'm sorry for any misunderstanding. Your concern about strictly adhering to OpenAI's content policy is completely valid and important.

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon: