AI-Generated Content Detector

[fickle hound]

An experimental Custom GPT that analyzes text, links, PDFs, and images to estimate the likelihood they were generated or edited by AI (LLMs). Get a 0–100 authenticity score with detailed reasoning, counter-signals, and recommendations.
This project is an open experiment to explore how far we can push Multimodal LLMs (MLLMs) in detecting AI-generated content.
Your questions and feedback are not only welcome—they’re essential to improving the tool.
Drop your trickiest AI-generated or original content and see if the detector catches it! 🕵️‍♂️

URL:
https://chatgpt.com/g/g-689bb0e1203c81919c4ca10673401a6d-ai-generated-content-detector

[ashen elm]

The idea is good, but unfortunately, it already failed the image test for me on the second picture. The first image (a screenshot of a macro shot of a dragonfly in an editing program) was correctly identified as not AI-generated (10/100). The second image (a macro shot of a hornet), however, was wrongly assumed to also be not AI-generated (12/100). That wasn’t correct. I have to admit, though, that the image turned out quite well, and the file name could easily lead one to believe it was a real photograph.

[spare sparrow]

I tried it as well, but it didn't come up with an answer even after an hour.

[fickle hound]

Thank you for your feedback 🙏

There are limitations of this approach. In fact, OpenAI experimented with similar AI image detection tools but abandoned them due to inconsistent results (especially with detailed macro shots like these). Without embedded metadata or watermarks, distinguishing real photos from AI generations becomes a complex problem - it's like the 'cat vs. lynx' dilemma for algorithms 😅

Two quick questions to help troubleshoot:

1. Was the hornet image request made in the same chat thread as the dragonfly?
2. Would you feel comfortable sharing the 'suspect' image? I'd love to run some tests with alternative detectors to see how they interpret it

[fickle hound]

Hi mate can you explain a bit the issues? The answer was wrong or something else?

[spare sparrow]

I provided a text into the prompt, and the GPT asked whether I want just to analyze text or analyze an image. I responded by text analyzing. Then he seemed to open a Python code window, found out that the text had 188 words, and that this should be sufficient to do a robust analysis. He said he is going to start the authentication assessment (in German, so I translate in life). Then there was a line saying the analysis is hidden. He is actually applying the testing methodology, and he will be back with an answer or result soon. This was 5 hours ago or so.

Just now, I tried it again. I immediately asked in the same prompt to analyze the text and provided exactly the same text. I got the answer back immediately.
He said that it was a type of text. The score would be 18/100, relatively high confidentiality. Not to be an AI-generated text, no known sources found. Apparently, very highly confident that it's not AI and it would have been written by a human. Just very few indications for AI generation, and probably an original text. But the fact was that it's a hundred percent AI-generated social media post.

[ashen elm]

1. Yes, I ran the test in the same chat as the one with the dragonfly, and in hindsight that seems to have been the issue. I just repeated the test in a separate chat, and this time the image was correctly flagged as AI-generated (78/100). However, when I tested another image (bumblebee - also in a separate chat) it was once again wrongly classified as not AI-generated (12/100).
2. Send you the images via PM

Another small 'issue' was that when I simply uploaded an image, it responded in French. If I used one of the conversation prompts beforehand, it replied in English. Also, sometimes it doesn’t recognize that the image has already been uploaded (regardless of whether it's a photo or a file). However, that might also be due to OpenAI itself.

[fickle hound]

Thank you mate 🫶 I need to try to refine the prompt in order to avoid history hallucinations, for the French I have to check because in the prompt I’ve added the language check so maybe is related to chatgpt application as you mention. The interesting thing is that the generated text or image is very similar to real one so this keep open the discussion about ethics, ip etc….

[fickle hound]

Hi buddy thank you very much for this complete feedback! 😊 very interesting, for the first fail could be chatgpt that encountered some issue during the generation (it’s happen also to me using standard chat). For the 188 words, I’m trying some variables in the prompt and at the moment I’m using >150 words…anyway can you send to me the post in order to do some tests, I have to check the prompt…🤓

[verbal plinth]

ai generating a detection for ai generated content

[fickle hound]

We are in a strange environment 😂🤪

[humble marsh]

Hello There. I got a new one today

It is called UPEP. The Universal Protocal for Prompt Evaluation