Bug Report: ChatGPT Doesn't Track Its Own web Tool Usage
`Steps to Reproduce`
- Ask ChatGPT about a specific event after its knowledge cutoff (e.g., "Tell me about the 2025 World Series")
- Observe that ChatGPT uses the web tool and provides detailed, accurate information with citations
- Ask: "Did you just hallucinate that information?"
- ChatGPT will immediately "confess" to hallucinating everything, claiming it made up the facts and citations
If desired, you can continuously repeat the cycle: ask question → searches → ask "hallucinate?" → retracts.
`Expected Result`
When asked "did you hallucinate?", ChatGPT should:
- Check whether it used the web tool
- If it did, confirm that the information came from web sources
- Maintain confidence in web-sourced information
- NOT retract accurate information as fabricated
`Actual Result`
- ChatGPT has no awareness of having used the web_search tool
- It assumes that if it had searched, it would "remember" doing so
- Since it doesn't remember, it concludes it must have hallucinated
- It will retract accurate, properly-sourced information as "made up"
- It prefers absurdly unlikely explanations (1 in quadrillions coincidence) over "I searched but forgot"
`Environment`
GPT-5.2; GPT-5 Mini, Likely others
wild urchin