Posts
Comments
The only thing standing in the way of artificial intelligence is our inability to define natural intelligence to compare it to.
The term "friendly AI" is meaningless until we determine whether a friend is one who maximizes freedom or security for us.
The frustrating thing about your experiment is not that I don't know how you convinced someone to release you, as anyone can be convinced of anything given the correct leverage. It's that I don't know the terms of the exchange, given that some structure had to be made to properly simulate your respective position as an AI in a way that negated the metagaming goal of winning 10 bucks. Was this derived organically over the course of just playing it out for the first half hour or whatever as you felt each other out? Was it established before the simulation conversation was enacted, so that the conversation was a matter of "under conditions x/y/z, you would be convinced to release me and thus I win simply for having established this"?
Until at least that much is known, it's hard to take anything of value away from the mere knowledge that this experiment happened and resulted in what it did, except for those readers who were arrogant enough to think that anyone could be so resolute as to actually consider this experiment an impossibility. It's not doing the impossible as long as people are involved.