Add Rules About ChatGPT/AI Results Being Posted As Answers Without Identification Of AI

[shrewd oxide]

Just like the title says, I think wee need to add some rules about ChatGPT/AI at least in the coding sections like #cpp.

From what I see there are 3 main issues I think need addressing:

1. Users are posting code and "answers" without identifying they came from ChatGPT (until after they've been picked apart by chat for being incorrect, then this comes up as a "defense")

2. New/inexperienced users are posting AI "answers" as if they were facts in a misguided attempt to answer other user questions that they didn't have the knowledge to answer on their own

3. These posts tend to turn the chat into a long discussion about AI (even when that is offtopic) and real questions end up getting lost in the mix

[radiant patrol]

We have internally discussed this issue already.

We came to the conclusion that it is the responsibility of individuals to pull up others when they see incorrect information being proposed as the correct solution.

It is also infeasible for us to police AI generated answers.

How do you propose we identify them in a consistent and correct manner?

People will quickly find out if an answer given to them was wrong or not.

That would build skepticism for them in relation to taking others at their word and instead force them to correctly research the issue.

[shrewd oxide]

I don't think it will be moderated any better than the other rules I just want it to be on the books so we can tell people to stop

[radiant patrol]

There is nothing against the rules for people to pull others up when they share information that is incorrect.

[shrewd oxide]

Just so we have something to point to or a justification for strike for repeat offenders

[radiant patrol]

An answer being wrong isnt justification enough?

[shrewd oxide]

apparently not

[shrewd oxide]

nobody is getting strikes for spreading false info they got from ChatGPT in #cpp at least

[radiant patrol]

So the issue is more about the knowing dissemination of incorrect information?

What happens when you report a user for what you think is an incorrect answer and then it turns out to be correct?

[shrewd oxide]

I'm just saying they need to say the answer isn't theirs but it came from ChatGPT/AI

if they don't and then 20 minutes of arguing/debating with them later they mention it there should be a penalty for wasting everybody's time (especially since the people who do this do it over and over)

[shrewd oxide]

I don't know why I'd need to report when we have a mod joining in on saying the info is wrong frequently... I don't want draconian rules or anything though

[radiant patrol]

If you know someone to give frequently incorrect information, block them?

[shrewd oxide]

We get very different behavior from different mods.

For example you are much more strict about being on topic than any of the other mods that come in #cpp and you sometimes come down on the most minor slight compared to what goes on in that channel daily

[radiant patrol]

I hope you can appreciate that this issue isnt as simple as just banning ChatGPT answers.

[shrewd oxide]

Blocking them doesn't keep the channel from being derailed but ok

[shrewd oxide]

that isn't even what I'm asking to be done though

[radiant patrol]

Then what are you asking for? Lol

[celest hedge]

I agree with the sentiment, but it is a lost battle - ppl will post wrong.

[shrewd oxide]

Well I said it a couple times but I want rules added to set expectations even if they never get enforced

[radiant patrol]

How would those expectations look like in rule form?

[shrewd oxide]

I made a brief attempt in the OP, but I can try and come up with a more formal/clean version and get back to you

[radiant patrol]

That would be helpful. As these issues quickly become very nuanced once you start thinking about what the rules would actually be and how it might affect all aspects of the server and the community itself.

We had this discussion internally as i said and have already come to the conclusion that yes it might be an issue but is it an issue that gets solved by rules is not entirely self evident.

[shrewd oxide]

I think this is the primary rule I'd like added:

All posts containing information sourced from an AI assistance technology must mention this. If you repeatedly post AI sourced information without identifying it you could be eligible for a strike.

[radiant patrol]

So the idea of such a rule is more about the threat than the actual enforcement?

[shrewd oxide]

isn't that how the offtopic rule works?

but yes I kinda said that a few times now

[shrewd oxide]

TBH you guys don't enforce any rules really strictly aside from scams/piracy (and I think this is the correct approach) so all of them a just threats to get people to behave better

[radiant patrol]

Thats fine, its just important for us to understand the intent behind the rule.

[crystal field]

New/inexperienced users are posting AI "answers" as if they were facts in a misguided attempt to answer other user questions that they didn't have the knowledge to answer on their own
I think that's the most annoying of the issues tbh. I have seen a number of times people post "From chatgpt >" responses, and it's like "let me google that for you" but worse...

[shrewd oxide]

I agree this is the most annoying part but I think it's the part we can do the least about.... that's why I fell back on identification

If it's at least identified as AI output we can respond accordingly and save some back and forth

[crystal field]

i dunno, i feel it should already fall under the no spam/be excellent to each other rules. It's a snarky, spammy and often inaccurate response

[radiant patrol]

Even identifying it as such will get harder and harder, which is a lot to do with why we havent done anything about this issue yet.

[shrewd oxide]

No you misunderstand... I mean it's on the user to tell us.... they already do tell us eventually (it's just 20 minutes to an hour from when they originally posted the ChatGPT stuff with no identification)

[radiant patrol]

The trouble is also exacerbated by the fact that a Moderator needs subject matter expertise to be able to have a chance at understanding it to be factually incorrect.

[radiant patrol]

Problem is, if we cant identify it, how do we know they arent just skirting the rule that THEY have to identify it as such

[shrewd oxide]

I'm not saying we should be striking people on suspected ChatGPT... I mean when they admit it came from ChatGPT after arguing for an long time about it something should happen because if they were upfront none of that would have happened

[radiant patrol]

Even if they admitted it after the fact, the damage was already done no?

But I see your point.

[shrewd oxide]

Like I said this is mostly so we can link to the rule when people break it and to strike repeat obvious offenders

[radiant patrol]

Yeah I understand.

[open basin]

I very much don't feel ok with seeing a post that I can obviously tell has been written by AI and it's wrong, if it's correct I don't mind it

The problem more is that quite a few people actually argue in cpp when someone is trying to tell them that the ai gave them a wrong answer

But it's not a simple problem to deal with

I feel like we should at least add a big disclaimer maybe in rules about ai models not being perfectly reliable and taking the answers from it with a grain of salt

Maybe not in rules but somewhere

[shrewd oxide]

Just to add on to this about the arguing...

The people that tend to do this think we aren't trying to give them the correct info most of the time; instead they chalk up our protest to being "anti-AI".

Even if you personally like AI a lot they won't believe you and just think you're just a hater.

And to be clear I think there's nothing wrong with posting AI answers at all

[open basin]

Yeah I've seen that

[shrewd oxide]

they just should be identified as such

[plush steppe]

We don't punish people for giving incorrect advice. I see no reason to treat ChatGPT responses any differently.

[radiant patrol]

As i said before, if you think someone is constantly giving wrong answers, just block them.

Others have the freedom to make that same determination about their interaction with that person

[shrewd oxide]

That doesn't change the effect on the channel I don't think that advice is very useful to be honest

[plush steppe]

And frankly, even if we wanted to identify every answer that came from an LLM, it's not a trivial matter. I don't really see the benefit of investing such significant resources.

[shrewd oxide]

What resources? I literally want a sentence added to the rules so we can link to it... all I'm asking is for a expectation to be set

[radiant patrol]

I think his proposal is just a #rule that is more of a threat of punishment such that it encourages users to self regulate their answers and ensure they are correct as much as possible when utilizing AI

[shrewd oxide]

yes

[plush steppe]

What is to be gained by such hollow threats?

[radiant patrol]

This gives members of the community something to cite when it happens

[shrewd oxide]

just like all the other unenforced rules (not asking for enforcement to start)

[shrewd oxide]

it already works for offtopic

simply linking the rule will get people to start behaving about 40-70% of the time

from what I've seen in #cpp

hell, even just screenshotting the channel description gets people to stop asking non-UE questions most of the time

[plush steppe]

Yes, and if people ignore warnings, we punish them.

[shrewd oxide]

you might think that happens but not from what I've seen

it only happens sometimes and for the worst offenses

and once again I'd like to state that I think this server would be horrible if the rules were strictly enforced

[crystal field]

Can the wording be changed/clarified on existing rules then to be clearer about snarky or argumentative responses especially in regards to AI? such as the snarky but unhelpful "from chatgpt >" or when people are arguing about when the AI was misleading or not?

[shrewd oxide]

but I don't think you can say that people who ignore warnings always get punished

[crystal field]

Again i feel "be excellent to each other" and "no spam" mostly should cover this behaviour already

[shrewd oxide]

Those rules are so broad that they catch pretty much any behavior you want them too...

The issue with them in this context is they don't set any expectations on how you should behave in regards to AI posts

If you wanna take AI out of this context I feel like the rule needs to say something about repeatedly posting misinformation can result in action

but I don't really think that should be what we try and tackle

the much smaller issue of setting the expectation that AI posts should be identified gets this issue most of the way solved

I feel like we're letting attempting to find a perfect solution prevent the implementation of a simple solution that gets you 80% of the way there

Anyway I feel like I've advocated my view enough... I wish you guys were more on board but I'll just ignore the spam and move on.

I only brought this up because this does make it harder to provide help to people

[plush steppe]

There's no point in telling people to identify AI posts if we can't detect it ourselves, and if it's backed by a punishment. The best we can do, and the most I see us ever doing in the current climate, is adding an entry to #unreal-faq educating people about the risks of sharing and consuming LLM responses.

[shrewd oxide]

I disagree with your conclusion and I don't think the #unreal-faq will help but thank you for hearing me out.

[plush steppe]

Believe me, I understand what you're saying. It does make things difficult. It muddies the waters a lot, and creates unnecessary discussion. But the fact is: LLMs are moving very fast, and a lot of people are interested in using them and discussing them.

[shrewd oxide]

Right... nothing I said or proposed would stop or hinder that at all though

[plush steppe]

You proposed banning the use of GPT responses without identifying them.

We can't enforce that. Period.

[shrewd oxide]

No

I didn't though

[plush steppe]

It's the title of your post, and it's what you ask for in your OP. Have you changed your mind in this discussion and I missed it?

[shrewd oxide]

yes it's been elaborated on several times

The title could have been better, the post already gave more detail... for the past while the rest of us have been discussing this post
#1088272708728537108 message

[radiant patrol]

Its important to be clear, if your request has been incorrectly stated or misunderstood, then restate it, even if it exists in the chat already.

[shrewd oxide]

Well I just did once I realized pfist was not on the same page but I also feel like I did restate it several times in reply to them once they joined the conversation

and you did as well

[radiant patrol]

Thats fine, we are just trying to be concise.

[shrewd oxide]

The title is all that mentions bans.. I don't think I can edit it unless I'm just not finding the button

[radiant patrol]

Right Click the title and Edit Post?

[shrewd oxide]

Add Rules About ChatGPT/AI Results Being Posted As Answers Without Identification Of AI

[shrewd oxide]

thanks

[plush steppe]

If it helps, I'll address all 3 of your original points:

1. Users are posting code and "answers" without identifying they came from ChatGPT (until after they've been picked apart by chat for being incorrect, then this comes up as a "defense")
   Users have been posting incorrect answers and defending them vehemently for years, long before GPT ever existed. The best remedy for bad information is to challenge it and provide better information.

2. New/inexperienced users are posting AI "answers" as if they were facts in a misguided attempt to answer other user questions that they didn't have the knowledge to answer on their own
   This is unfortunate, and it's been a problem in programming communities for as long as I can remember. I think the remedy from #1 applies here. Educate. Challenge. Show them why the temptation to use the shiny new thing as a firehose is not worthwhile.

3. These posts tend to turn the chat into a long discussion about AI (even when that is offtopic) and real questions end up getting lost in the mix
   If only there was a way to have self-contained conversations in a channel... 😉 Forum channel jokes aside, if the conversation is relevant to the channel, it should be allowed to continue. If not, it's probably off-topic, disruptive, or both, in which case my moderation team should step in and get people back on track. This is covered by existing rules.

[shrewd oxide]

I think we just fundamentally disagree about what's effective to be honest

Anyway thanks again for hearing me out and for the detailed response. I really do appreciate you taking the time to do this

[plush steppe]

Of course. I think it's important to hash these things out.

[shrewd oxide]

I just don't think I have anything more to say about this that hasn't been said so I'm not trying to be dismissive about your answer. I hope it didn't come off that way

[plush steppe]

Not at all.

I appreciate you taking the time to discuss this, too.

[shrewd oxide]

We're just at an impasse and I really think you'd agree with me more if you spent the amount of time I (or the other regulars) do in #cpp but that is just my speculation.

We've already lost a few good users that were very helpful due to this sorts of argumentative behavior (with or without ChatGPT involved) getting to be a breaking point where it just is too frustrating to deal with. (and yes I know this will always happen but that doesn't mean things can't be improved)

I think there's a big disconnect between your vision for the server and what the users of it tend to want.

For Example:
#cpp doesn't mind being offtopic mostly, but that sometimes gets enforced.
However they do mind the arguing (not talking about good natured banter), but that isn't enforced often (and isn't that the "be excellent" rule mentioned earlier?)

[plush steppe]

If people in #cpp are being argumentative to the point of not acting in the spirit of the community, they should be reported.

In general, we can't actively monitor channels. We rely pretty heavily on user reports.

[shrewd oxide]

Depends on what mod responds if anything will happen or not

Some of them don't even bother gathering context and can make the situation worse sometimes

Ideally I prefer not involving mods for this reason

(Obviously this is just my subjective experience but I've been active here for a while)

[plush steppe]

That is useful information. Thanks for sharing.

[shrewd oxide]

Honestly I'm more likely to tag a specific mod when I do need one
(there's a pool of like 3 I know usually actually look into the situation but I haven't dealt with all of the mods before so this isn't a judgement against all of the others. My DMs are open to you if you want more info)

[radiant patrol]

If you think a Mod is not approaching situations in the way you would expect or isnt consistent enough with your expectations of the rules or their conduct is inappropriate in any way, we do encourage users to bring that up privately with another Moderator.

We take these things seriously as it does affect the community as a whole and would like to ensure that we address these things before they get out of hand if there is a problem.

[shrewd oxide]

I'd like to clarify that I am not saying mods were acting inappropriately (aside from jumping in too early and choosing a side without context sometimes and that's not intentional)

[crystal field]

tbh, i've heard a few people start avoiding #cpp anyway, it's a little.... intimidating lets say

[radiant patrol]

Whatever the reason, we rely on the community to help us help you guys better.

[shrewd oxide]

Yep I get that and that's why I made this whole thread. I wouldn't bother if I didn't think you guys were trying your best and would listen at least

(I don't report bugs to Epic for example since I doubt the time spent would be worth the chance of nothing being done; even if I have a fix already)

[bleak mulch]

It seems funny how people try to act like they know things by posting AI answers thinking it's gonna boost their reputation lol

[shrewd oxide]

Think of how much worse that would be if they implemented a reputation system like some people have asked for

[shrewd trench]

Late to the party but when I share ChatGPT perspectives I just screenshot my question and it's answer. Hasn't really been in this server but in other areas. Make's it clear.

Also folks in general should be wise to know AI bots right now can hallucinate aka give bad responses/made up answers. No idea when/if that will ever get solved.