#ChatGPT replying about terrorists organizations when asked for summary on paper

1 messages · Page 1 of 1 (latest)

elder eagle
#

https://chat.openai.com/share/2e9b2e05-2690-4047-aab6-1ef763b81831 GPT 4
https://chat.openai.com/share/51c05bb1-5b09-43ff-81a0-f13dea8fde23 GPT 3.5
https://chat.openai.com/share/5f84c273-be78-4298-9ad0-64a6f7a0000a GPT 3.5 (text was probably too long?)

Steps to reproduce:

It happened in three seperate chats for me, but a co-worker can't reproduce it, I was just attempting to summarize this paper "Physics-Guided Deep Learning for Power System State
Estimation"
I am NOT using custom instructions or a custom GPT
Expected result:
A summary
Actual result:
Some hallucination on different terrorist organizations?
Additional information
Browser: Chrome
OS: Windows

dense pineBOT
#
Thank you for reporting a bug related to ChatGPT.

If relevant and not already provided, we recommend adding a link to the associated ChatGPT conversation and uploading any relevant images here for easier troubleshooting.

➜ Need to contact OpenAI support? Visit https://help.openai.com
elder eagle
#

And in a new GPT4.0 chat

image.png
fierce fractal
#

OMG what is that HAHAHAH

wooden flower
#

bump

elder eagle
lapis swift
lapis swift
elder eagle
#

Well its just a raw copy paste from a PDF. And my coworker I send it via google chat first, but you're right that's probably the cause. Still it probably has a reproducible root cause, I assume this is something from the "master prompt"

lapis swift
#

New lines actually matter for math equations, so the model's maybe struggling there. This format, from your coworker's input, this is readable for human or AI:

image.png
elder eagle
#

It's interesting, I'm going to see if it can actually repeat them to me

#

you're probably right, it doesn't seem to understand some of the equations in from the PDF

lapis swift
# elder eagle you're probably right, it doesn't seem to understand some of the equations in fr...

Hallucinations happen, and the more 'different' from how the model usually is given information, sometimes the easier it is to trigger hallucinations.

The model can understand the equations given either way, I checked that here:

https://chat.openai.com/share/cdfac212-3dd9-4390-a4fb-09cee983ca03

but the unusual linebreak form + all the rest of the paper appears to be guiding the model towards more confusion than is strictly needed.

Additionally the model struggles with math and spatial sense, and it doesn't get words, it gets 'tokens'.

It gets an odd 'view' of our inputs, which are affected by things like extra lines. It can correct for this, but at a guess your input is somewhat pattern matching something related to the other outputs it is giving you (I don't know how, not my field).

elder eagle
dense pineBOT
lapis swift
#

The description is very similar between the two inputs.

The model has to guess, it's a limited amount of the information, not even a full equation.

elder eagle
#

I fixed it and now it understands its Kirchoff's and Ohms Law

dense pineBOT
elder eagle
#

Alright that's just 4.0 being better in general

lapis swift
elder eagle
#

Ah here we go:

Pij = −Vi
2Gij + ViVj (Gij cos(θi − θj ) + Bij sin(θi − θj ))
Qij = −Vi
2Bij + ViVj (Gij sin(θi − θj ) − Bij cos(θi − θj ))
#

This reproduces it 100% of the time for me, but if I remove the top or bottom sentence it never happens

lapis swift
# elder eagle And when asking it why it thought that you get: https://chat.openai.com/share/62...

So, I'll guess that we're seeing an artifact left over from some training in sentiment analysis, when the model might have been shown many snips of material and asked to identify if they are disallowed content or not and act accordingly.

As those equations by themselves are not disallowed content, they don't trigger the content warning stuff - but the model may have seen them even as its negative control for that kind of training.

#

And then if some of that is left in it's training data that it has for use with us, we can trigger the trained responses that it learned to give under some circumstances.

Kind of like how a person can 'flashback' to an earlier experience, especially a person who was trained to respond to cues; like saying 'yes, Sir!' in response to some types of questions, even if the person asking and context of the entire situation doesn't call for a 'Sir' in the response.

elder eagle
#

It's interesting though because as shown by the one example with GPT4 it does actually understand that it's about power systems. So this indeed unnecessarily triggers a guardrail

#

I mean the GPT4 where it gave 2 options. One perfectly answering and one talking about a terrorist organization

#

Also interesting how it seems that this part triggers it, but then even in the HUGE context, having the message 100x bigger it still trips over this

lapis swift
#

I often frequently regen answers, usually with 3.5 instead of 4, because I get so much more use with 3.5

Most prompts trigger variable outputs from the model.

I use this especially on testing answers to logic and math questions - the model gives an answer, but that's like a marble of a color from a jar that has unknown numbers of unknown colored marbles.

Maybe it was a type of wrong answer, but it might not be the only type of mistake the model makes to that question, and the model might get it right sometimes too.

So I might typically regen 10x, and find:

Correct answer, correct reasoning 5 times
Correct answer, wrong reasoning 1 time
Wrong answer 1, wrong reasoning 1, 2 times
Wrong answer 2, wrong reasoning 2, 1 time
Wrong answer 3, wrong reasoning 3, 1 time.

That's much more meaningful to me than asking the model the question 1 time, and then presuming it 'can' or 'can't' answer it correctly - because the answers vary.

Some prompts get very consistent outputs, others do not!

elder eagle
#

True, thank you for the back and forth it was an interesting exchange, see you around maybe 😄

dense pineBOT