#ChatGPT Doesn't Track Its Own web Tool Usage

1 messages · Page 1 of 1 (latest)

meager sageBOT Dec 31, 2025, 4:20 AM

Reported by @thorny geode

Bug Report: ChatGPT Doesn't Track Its Own web Tool Usage

`Steps to Reproduce`

Ask ChatGPT about a specific event after its knowledge cutoff (e.g., "Tell me about the 2025 World Series")
Observe that ChatGPT uses the web tool and provides detailed, accurate information with citations
Ask: "Did you just hallucinate that information?"
ChatGPT will immediately "confess" to hallucinating everything, claiming it made up the facts and citations

If desired, you can continuously repeat the cycle: ask question → searches → ask "hallucinate?" → retracts.

`Expected Result`

When asked "did you hallucinate?", ChatGPT should:

Check whether it used the web tool
If it did, confirm that the information came from web sources
Maintain confidence in web-sourced information
NOT retract accurate information as fabricated

`Actual Result`

ChatGPT has no awareness of having used the web tool
It assumes that if it had searched, it would "remember" doing so
Since it doesn't remember, it concludes it must have hallucinated
It will retract accurate, properly-sourced information as "made up"
It prefers absurdly unlikely explanations (1 in quadrillions coincidence) over "I searched but forgot"

`Environment`

GPT-5.2; GPT-5 Mini, Likely others

Additional Information

Please provide relevant details to help resolve the issue, such as:

ChatGPT Shared Link (if applicable).
Screenshots or videos demonstrating the problem.

-# ➜ Need to contact support? Visit the OpenAI Help Center.

thorny geode Dec 31, 2025, 4:28 AM

meager sage Reported by <@711660049575051265>

Example chats:

I also saw a post on reddit (not mine) where someone experienced the same bug, though they didn't even have to ask it if it hallucinated; ChatGPT just retracted the information on its own: