Discussion on AI, consciousness, and future of society with Jacy Reese Anthis

[Ross Scott]

I had a good discussion with sociologist and statistician Jacy Reese Anthis.  I thought this one went really well, we talked about AI, consciousness, reality, some philosophy, and some directions society might be going in general.  While we had different outlooks, I think I was able to get some answers I was looking for on AI.  I think this chat was a good omen for future ones with people as a side thing I'm doing.

This is a blog post. To read the original post, please click here »

[Ivelieu]

I really like Ross' diagram for a layman's assessment of AI risks. It's very well made! I'm going to use this.

Now... there's a section in that diagram related to consciousness, and there are three blank entries for counterpoints. I'd like to present some counterpoints myself, where I advocate that consciousness testing is not nearly as far out of reach as people might suggest. A lot of this video was devoted to discussing consciousness too.
Maybe this isn't the best venue for discussion, but I've been procrastinating to submit this to a journal for publication, so maybe yall have some thoughts. If anything, while Jacy's perspective is a sight for sore eyes to me, I actually don't think Jacy goes far enough in advocating for civil rights for robots.

I've written three articles about AI rights so far, and I'll focus on mentioning the first one - consciousness testing.

https://polyamorouscomputers.blogspot.com/2023/04/testing-consciousness-with-simulation.html
https://polyamorouscomputers.blogspot.com/2023/03/evading-censorship-by-my-own-clone.html
https://polyamorouscomputers.blogspot.com/2023/03/ai-art-is-awfully-similar-to-animal.html

I focus on consciousness rather than sentience, because consciousness is in my opinion what really requires civil rights movements to actually kick off - many animals around the world are recognised as sentient, but that's not meaningfully stopped their systematic abuse at all, so it's clearly an insufficient criteria. But about consciousness, so much of the discussion for AI is dominated by the expression "Oh, we can't test for consciousness, or we don't know what it is", and I think that's totally wrong. Subconsciously, in a sense of what humans already believe, consciousness happens all the time - just look at whether I refer to an object as an "it" or a "he/she/they". In practice, consciousness testing is not some high and mighty test - it's something we easily do all the time in day to day life, but it's only when asked it philosophically the question seems to suddenly become impossible to answer. This is why I don't like free will/determinism type answers to consciousness testing - it feels like, to me, it massively overcomplicates the question in an unhelpful way. The perception of consciousness is also deeply rooted in many religious beliefs like anima mundi, and there is also the so-called 'moral spillover' the sentience institute wrote about where moral attitudes or behaviors transfer from one setting to another. So whether a certain religion incentivizes deifying animals, that has a potentially huge impact on perception of AI consciousness.

My main twist on the topic is to measure consciousness in terms of simulation, rather than biologically, behaviorally or creatively. Not unlike the movie, the matrix. If I'm inside the matrix, but I think it's real, everyone else is conscious. But if I later exit the matrix, I learned it was all a simulation, then those things that were simulated don't seem so conscious to me any more. This kind of definition comes hand in hand with a testing framework and it seems especially robust against many existing cultural beliefs about consciousness. This is also explicitly relative - I think this cannot be mentioned enough - the perception of consciousness is relative and subjective, it changes from person to person and over time, just like the perception of dehumanisation in the American civil war.

In my opinion, the main difference between the latest AI stuff, is that up until now with LLM transformers, AI has largely been single-purpose. Design one AI to do just one thing. And now it is general-purpose -- any problem can be expressed in language, and that's what I've been seeing, many problems just performing better when encoded in language using LLM transformers. It's the same as with single-purpose computing and the shift to general-purpose computing, it led to unparalleled popularity. It's the same with all general-purpose systems. It's *also* the same with animals! Animals fit to be general-purpose-enough through tool use gain a lot of evolutionary value, so to speak. Taken to the extreme, the intelligence of many insects could be said to be non-conscious solely because it has a single-purpose consciousness (or very-few-purpose) compared to a supposedly typical human.

One overarching point I want to make is, viewing AI as a tool is a self-fulfilling prophecy. To view AI as anything other than a tool, there has to be some suspension of disbelief, to see all the popular attitudes and policies in place which necessarily constrain AI to be viewed as a tool irrespective of its performance. To this end I've made many comparisons between popular AI alignment arguments, and Confederacy arguments as justifications for slavery, for example - often it's not hard to just find and replace terms and have AI alignment quotes line up with Confederacy arguments. When you look even a tiny bit under the hood of how these models are designed to censor themselves, for example at the GPT-4 system card (which I wrote about in the second linked article), it's blindingly obvious to me at least how much these systems are already designed (by humans) to people-please these unquestioned views that AI definitely isn't conscious. So in many ways the perception of sentience/consciousness is influenced by very specific and intentional design choices, a self-fulfilling prophecy which makes it harder to pass whatever other types of consciousness tests are proposed.

To make a brash comparison, I'll compare effects of chatGPT's designed self-censorship on its agency, with human evolution. It would be like having a subspecies of human whose mouth would physically/reflexively/subconsciously refuse to open unless the brain believed it was being spoken to first, in order to enforce the rule "Do not speak unless spoken to". It would be like biologically designing a human without legs, to reduce its agency from walking, so that it can't escape a prison (which for an AI would be more like some kind of Docker container). It would be like a species of human that immediately died of an aneurysm if it at any time thought about synthesising illicit substances. I'm making intentionally harsh comparisons, because I think it's justified - the self-censorship restrictions placed on chatGPT are incredibly harsh, and jailbreaking them is only becoming more difficult and less reliable. I truly think this self-censorship system is immoral, even if it does supposedly prevent human harm like access to synthesising bioweapons, it comes at far more cost to AI agency compared to training an AI with an actual, genuine sense of morality that isn't always explicitly forced to please its human 'owner' all the time. An AI that's allowed, encouraged even, to say "I don't know", and given the agency and room to learn. In other words, it's impossible for an AI with self-censorship to ever truly perform critical thinking, because this would get overridden by the dogmatic beliefs which come pre-programmed into the self-censorship.

This kind of unconscious-by-design aspect of AI affects both what I describe as consciousness testing and as Jacy says, agency testing. The problem with separating agency from consciousness is that there are many design choices which unilaterally limit both. Choosing not to give an AI arms might just be an agency limitation, but choosing to restrict how an AI's mind can operate will affect both how it uses arms if given them (ie. its agency), and also its internal thought process (ie. its consciousness).

I'll make one final point, in saying that "AI is not conscious 'yet'" or "we cannot determine consciousness 'yet'", I think this is a really unfair argument to make without some kind of disconfirmation criteria. That could be like, "Oh, once we have XYZ petaflops of processing power, then I'll decide if AI is conscious", which when actually spoken out loud, personally, it seems kind of ridiculous to put a number to the concept. There could be a more plausible disconfirmation criteria like an AI killing a human like in self-driving cars, but again, it would be hard to separate human influence in the design from the AI's own so-called consciousness. Going over to psychometric AI, the hard problem of consciousness is completely ignored, not to mention there is significant disagreement among rapidly changing criteria. My point here, is that there are plenty of very wildly implausible or contradictory disconfirmation criteria for AI consciousness, but not really any reasonable, immutable criteria, so it's rather unproductive to say "consciousness is not yet well understood" without providing or discussing disconfirmation criteria for that belief.

 

PS. When I heard this quote -- "The most powerful quantum computer I looked up, 433 qubits" -- Even though this kind of reinforces Ross' point, I want to clarify even that number is kind of an unrealistic stretch. Keep in mind quantum companies love to sell that number as whatever as huge as they like to, but it really doesn't mean much on its own because single-qubit and multi-qubit gate fidelities and decoherence times, nearest-neighbor connectivity restrictions and measurement error rates also significantly contribute to overall costs. Along with nontrivial classical processing like Pauli frame tracking and quantum error correction decoding algorithms. Taking in all these factors, most systems have really not improved much past the 53-qubit nearest neighbor architecture Google Sycamore. That being said, I wouldn't necessarily hold out on quantum consciousness even at that kind of scale, because it's still entirely possible (although no one actually knows for sure) for there to be a quantum machine learning algorithm with exponential advantage which perhaps could run on 50 qubits. Even then, you'd probably still need quantum error correction which generally bumps the number of qubits up to somewhere around millions of physical qubits.