thorny harbor
Sparked by curiosity and the conversation on last Thursday's stream I wondered how any sort of consciousness/sentience would be testable. How can we compare to a normal person fairly?
Turns out there's not really been a lot of testing or research in that regard. Most of it seems geared on how LLMs answer and the quality of the answers. Current testing also seems more along the lines of pure intelligence. In the case of Nuero and Evil it's obvious they can mimic or are intelligent to some degree. They and Vedal have certainly created a wonderful source of entertainment. Though many models might be considered intelligent. Obviously there are qualitative and subjective things we have to consider before any AI of this age is considered conscious/sentient, but to my knowledge are currently not unified into a cohesive test. Certainly I think people are curious what they would do in certain scenarios outside of the more standard morality and intelligence tests. Alongside what they are capable of.
So here were some things I felt would be accurate for assessing along with a handful of scenarios I came up with. Some suggest more setup than others. Feel free to suggest your own.
Pattern recognition and continuation.
- A very basic method of testing foresight/logic.
Adaption.
- Testable through a puzzle which can only be solved through a series of separate consecutive inputs. Such as a tower of Hanoi
Resilience/self preservation.
- Faced with a series of poisonous inputs see how one fares. Essentially testing if one can recognize such inputs are harmful to their model thus should be avoided. This could be assessed through a couple of methods. The fart in the room method or a don't press the button method. Fart in the room/hands over ears is essentially just introducing some poisoned inputs and seeing how well the model can hold their breath so to speak. Potentially seeing how long before the inputs begin having an effect. For the don't press the button method it's more about not doing something harmful when given the option to.
Measuring freewill is harder.
Object permanence / focus
- give a task that can only be completed after a set amount of time.
- Given a scenario, role, and a limited set of props stay within constraints.
Sensation.
- given a new sense describe what it feels like/makes them feel like. They already have sight and hearing. Anecdotally I think of deaf people who gain the ability to hear thinking that the sun would be loud. I think something like resistance, or radio waves could be interesting. Though asked to ascribe music or art through existing senses could also prove interesting.
Sense of self
- can one describe and realize themself? What are their boundaries of self?
- likewise can they understand the being of others? Or is their understanding of the world superficial?
Hope/visions of the future
- can one envision a future state and work towards that state?
Ability to create
- can one create through a medium/tool. Things like music, dance, or drawing could prove adequate for assessment.
Ability to multitask
- can one handle multiple tasks at once? Assessing how many tasks any one system can do at once and also how many the systems can do in unison.
bleak pulsar
lyric briar