On 5/30/2023 at 3:59 PM, Ross Scott said:I've heard several people bring up the paperclip maximizer scenario, but I feel like that would inevitably get cut off at some point, the longer it went on. Once the AI stopped doing what the manufacturer wanted, you would see more and more effort to shut it down. It would still be limited to whatever facilities it was given. At some point the metal to make more paperclips run out, or the company runs out of funds to order more metal, or the power gets cut off remotely, or a counter-AI gets deployed against it, etc.
I could see a Machiavellian angle of convincing more people to assist it, but even that could only go so far before the authorities got involved and just treat it like a terrorist network. I'm having difficulty seeing how the AI would KEEP making paperclips unless it developed god-like powers also. This isn't to say it couldn't do damage, but its level of interaction with the real world v. the digital one is finite.
The assumptions are- that it will learn to hack, divert funds for itself covertly and illegally, order online any necessary machinery (including power generators), construct if necessary factories in other countries with less regulation, and expunge any mentions of itself from any law enforcement, government branch or news agency until it will reach enough momentum it will be nigh unstoppable.
edit:Oh- and will also keep backups of itself all over the internet which will continue its work if its thwarted at earlier iterations.