People broadly categorize AI into narrow ai, general AI, and superintelligent AI. Right now I don't think you could realistically call GPT-4 "narrow" - you can take a new task that is definitely not in the training data and throw it at GPT-4 and it'll usually be successful. It doesn't feel like general AI either though, because there are a lot of fields where it falls severely short of what even an average human can do. What qualities does an AGI need to have to be an AGI?
#How good does an AI have to be before we call it AGI?
solemn matrix
lyric kite
Where does it "fall severely short of what even an average human can do"?
It seems to me that the restrictions it currently has (no inner monolgue, limited memory, nearly no sensory input etc.) are somewhat unfair disadvantages. But even with those, it surpasses the average human in general.
Show me a human that can improvise a theatre sketch, explain basic terms of rocket design, write a poem about curtains, and fix a bug in a chunk of code, each in a few seconds.
Taking the idea of "general intelligence" that people had in mind when the term was coined, I think we have already mostly achieved that. I guess it depends on whether you compare it with the best human in every possible domain, or with an average human and see how much worse or better it performs on average across many tasks. It cannot definitely not everything better than every human, but it can do most things better than most humans.
solemn matrix
Where does it "fall severely short of what even an average human can do"?
It se...
Even with the official GPT-4 vision addon, it has pretty bad problems doing spatial reasoning. It's also pretty well known how bad it is at doing even simple math problems without having access to external tools like the Wolfram Alpha plugin. And in cases where it makes mistakes, it seems to have pretty bad problems recognizing and rectifying its mistakes without hand-holding from the user. I've noticed that last one particularly in programming tasks; it'll do something like make up a library which doesn't exist, or it'll use outdated methods that don't work in up-to-date libraries for whatever it's trying to do, and even when told "hey, that doesn't work" it struggles to grasp it and understand that it should approach the problem differently.
Broadly I'd say that I agree with you, I think we're most of the way to AGI with GPT-4. I think that long-term memory and better non-text inputs are going to be critical to taking it to truly human-level in all intellectual areas though.
lyric kite
I agree with you on the spacial reasoning.
I would add that it's also really bad at any tasks which require internal monologue or "thinking things through" before blurting out an answer, for example, even playing tic tac toe. If you prompt it to go through the rules for each step explicitly, it works, otherwise it makes stupid errors. However, this is solvable by giving it an internal monologue and allowing it to think about the answer before commiting.
On the other hand, I use it daily for coding, and I basically never come across the problem you describe. I only encountered an outdated syntax once, and that was for something that had changed afterthe knowledge cutoff date, so it's not GPT's fault. Out of curiosity, what programming language do you use? I mostly use python, javascript, SQL and other very widespread languages and had basically no problems.
I think that long-term memory and better non-text inputs are going to be critical to taking it to truly human-level in all intellectual areas though.
I agree.
solemn matrix
I agree with you on the spacial reasoning.
I would add that it's also really bad...
per the programming issues, I basically only use it for Python. The vast majority of the time it works great. This is primarily an issue with obscure or lesser-known libraries though, stuff that presumably doesn't appear very much in the training data. I wish I had some examples to show here but it is fairly rare that I encounter this.
I've noticed similar in my tests of using it for less often used programming languages too though. If you ask it to do really atypical tasks with programming languages that are still well-known and well-documented it seems to cause issues for it. One thing that springs to mind is when I asked it to write a web application in C++ that had a registration and login page that used SQL for data storage. It produced code full of really severe SQL injection vulnerabilities which I found very strange, being that it's well aware of how escaping user input for SQL works and well aware of how C++ works, but it seemed to struggle with combining these two pieces of knowledge. After I pointed the issue out it worked fine, but that still sticks out in my memory as an odd moment.
junior violet
Even with the official GPT-4 vision addon, it has pretty bad problems doing spat...
skill issue 👎 👎
edgy glen
radiant ginkgo
I think it's safe to say it needs to at least exist beyond that short period of time it takes for a single API request to finish. It also needs to be able to perceive/see the world and have thoughts or ideas that are independent of input.