#ChatGPT Project Files Access incorrect data

1 messages · Page 1 of 1 (latest)

midnight condorBOT
#

Reported by @golden otter

Bug Report: ChatGPT Project Files Access incorrect data
`Steps to Reproduce`

Ask ChatGPT to copy text sections from an uploaded project file to a new canvas document.

`Expected Result`

Text from uploaded project file should be copied word for word to new canvas file.

`Actual Result`

Text from unknown origin copied to new canvas file. Unable to ascertain where this text came from.
Unreliable responses to queries.

`Environment`

Windows 10 Desktop app, Windows 10 Chrome Browser

#
Additional Information

Please provide relevant details to help resolve the issue, such as:

  • ChatGPT Shared Link (if applicable).
  • Screenshots or videos demonstrating the problem.

-# ➜ Need to contact support? Visit the OpenAI Help Center.

woeful dove
#

I don't know if this is a bug but it's certainly annoying. That is, this might just be how it works, not something broken in defined functionality.

#

This tooling doesn't actually copy text from files. I believe it vectorizes files and then puts its best interpretation of that text (not the actual text) back into the chat/canvas. Sometimes it's 100% correct and other times not.

#

It certainly processes the text - if you ask for code from a text file it doesn't pull in the comments, probably because during embedding/vectorization it didn't consider comments relevant.

#

I used a Custom GPT to prepare for a driver's license exam - as much a test of the GPT's ability as it was intended to be useful for me. You're right, it didn't use the exact text. It would make up rules (hallucinate), guess, do net searches, combine details that did not result in legally correct responses, and when asked to provide a specific reference to text to support its responses it almost always failed and said "yup, you're right, that's not in there".

#

This is one of the reasons why we shouldn't use this tooling for legal or medical or other important information. It's just not trustworthy enough yet.

#

There's another issue with project files that I've documented a few times here. Files are cached by name. So if you upload "info.txt", modify it, delete the existing file, and upload the new file, you might still get responses based on the original file. This is based on file caching and scheduled removal of unused files. To address this I version my file names: "info01.txt", "info02.txt". Then in prompts I specifically request info from the info02 file, etc. It's a pain but necessary at the moment until OpenAI publishes a statement that indicates this is no longer necessary.