#What model do I use for AI moderation?

1 messages · Page 1 of 1 (latest)

crude fern
#

What model do I use for AI moderation?

royal sparrowBOT
#

Hi @crude fern. I'm kapa.ai, your friendly support bot. I will try to answer your question by searching through the knowledge base. :wave:

#

OpenAI provides several models for AI moderation. These models are designed to check whether content complies with OpenAI's usage policies. They provide classification capabilities that look for content in the following categories: hate, hate/threatening, self-harm, sexual, sexual/minors, violence, and violence/graphic.

The models available for moderation are:

  • text-moderation-latest: This model points to text-moderation-007 and is automatically upgraded over time to ensure you are always using the most accurate model.
  • text-moderation-stable: This model also points to text-moderation-007. If you use this model, OpenAI will provide advanced notice before updating the model. The accuracy of text-moderation-stable may be slightly lower than for text-moderation-latest.
  • text-moderation-007: This is the most capable moderation model across all categories.

These models take in an arbitrary sized input that is automatically broken up into chunks of 4,096 tokens. In cases where the input is more than 32,768 tokens, truncation is used which in a rare condition may omit a small number of tokens from the moderation check.

Here is an example of how to use the moderation API:

from openai import OpenAI
client = OpenAI()

GPT_MODEL = 'gpt-3.5-turbo'

You can find more information about these models and how to use them in the OpenAI documentation (https://platform.openai.com/docs/models) and the OpenAI Cookbook (https://cookbook.openai.com/examples/how_to_use_moderation).

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@crude fern kapa.ai is still learning and improving, please let us know how it did by reacting below

crude fern
#

please give me a short python script for moderating the returned result of a function

#

@royal sparrow please give me a short python script for moderating the returned result of a function

royal sparrowBOT
#

Let me check the knowledge base for you. One moment! :books:

#

Sure, here's a short Python script that uses OpenAI's moderation API to moderate the output of a function. This script is based on the examples provided in the OpenAI Cookbook and OpenAI Community Forum.

from openai import OpenAI

client = OpenAI()

def function_to_moderate():
    # This is your function that generates some text
    return "Your generated text goes here."

def moderate_output(text):
    # This function uses OpenAI's moderation API to check the text
    api_response = client.moderations.create(input=text)
    response_dict = api_response.model_dump()
    is_flagged = response_dict['results'][0]['flagged']
    return is_flagged

# Generate some text
generated_text = function_to_moderate()

# Moderate the generated text
if moderate_output(generated_text):
    print("The generated text was flagged by the moderation API.")
else:
    print("The generated text passed the moderation API.")

In this script, function_to_moderate is a placeholder for your function that generates some text. The moderate_output function uses OpenAI's moderation API to check whether the generated text is flagged as inappropriate. If the text is flagged, the script prints a message indicating that the text was flagged. Otherwise, it prints a message indicating that the text passed the moderation API.

Please note that you need to replace "Your generated text goes here." with the actual text generated by your function.

You can find more information about how to use OpenAI's moderation API in the OpenAI Cookbook (https://cookbook.openai.com/examples/how_to_use_moderation) and the OpenAI Community Forum (https://community.openai.com/t/530068).

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@crude fern kapa.ai is still learning and improving, please let us know how it did by reacting below

crude fern
#

@royal sparrow what is the pricing of the moderation models?

royal sparrowBOT
#

Let me check the knowledge base for you. One moment! :books:

#

The Moderation endpoint and content filter provided by OpenAI are free to use for OpenAI API users. The usage of these tools does not count towards your monthly usage limits. However, it's important to note that the moderation endpoint is free to use only when monitoring the inputs and outputs of OpenAI APIs. Other use cases are currently not allowed.

You can find more information about this in the OpenAI Help Article (https://help.openai.com/en/articles/4936833-are-the-moderation-endpoint-and-content-filter-free-to-use).

Note: you can ask me a follow up question by @ mentioning me again :speech_balloon:

#

@crude fern kapa.ai is still learning and improving, please let us know how it did by reacting below

crude fern
#

@royal sparrow how much is the moderation when moderating external text

royal sparrowBOT
#

Let me check the knowledge base for you. One moment! :books: