regal helm
This came up in a discussion thread about the use of a public Discord bot. I wish to ask to gather some information and to clarify the rules. OpenAI includes usage policy outlined here: https://openai.com/policies/usage-policies in the usage policy, it clarifies that some content is inappropriate, and that the model should not be used for that purpose, or the platform for that matter, and that OpenAI will take actions on users (including API users) whom violate this usage policy.
My question comes through this potential scenerio -- antispam
If we use a public endpoint, like Discord, what stops a developer from getting their account banned if a public Discord user sends malicious messages in violation of the usage policy, and the developer's OpenAI token submits the request to the OpenAI API?
Will we for example as API users, put our accounts at risk to use OpenAI for spam detection, malicious user flagging / punishment and raid control (where the API would recieve copies of potentially inappropriate message contexts) to classify them as spam/not spam.
In this use-case, submitting a system prompt attached to a message context that asks over the API to identify whether or not a message is spam, harmful, malicious, or not in accordance with the user policy, and return a boolean response (true or false), that leads to a message deletion or the application of punishment (for example, handing out server mutes or simply deleting messages) if the prompt returns "true".
I feel like without provisioning for acceptable use case to provide potentially inappropriate message content with the intent to identify / remove it, a developer puts their account at risk by allowing a bot to be used on a public discord server
My idea was a spam filter for Discord using OpenAI's LLMs, but I have concerns about that. Because doing so involves sending potentially harmful data in, to get a clear identifier out.
Is this allowed / not allowed, what should be the approach?
