Emotional valence as cognition mutator (not a bug, but a feature)post by Slider · 2019-05-15T12:49:40.661Z · score: 10 (5 votes) · LW · GW · None comments
Maybe because of my neurotype I view it important that I have good general way of handling things. I noticed a a pattern where people would process seemingly similar requests very differently and got curious whether there is an actual difference or are people being needlessly inconsistent.
- Can you pass the salt, please?
- Step aside
- Can I have your wallet, please?
2 of these are supposed to have a "well, he did ask" kind of reaction and the reminaing a "no, why would I?". The difference in suggestibility is great althought they all are supposed to be straight forward requests for X or a "can i have X? yes/no" kind of questions.
Now if I give a complicated answer why you should give me your wallet you are likely to remain pessimistic about complying. This also seems to have the weird property of increasing the sophistication of the argument is unlikely to increase compliance probability. It would seem like the person is not willing to entertain the proposition but is prejudiced to reject it from the get go. This would seem to go contrary to "intellectual openness/readiness".
However the behaviour is not really mysterious and the reasons for it are pretty well founded. Your wallet contains a lot of your money which is important for a lot of things you do. Should something bad happen to it a lot of things will get a lot messier. It's also an attractive prey target. It's plausible that someone might lie to you just to get a hold on your wallet just to have its possession.
When I thought about a straight up thief trying to get the wallet by asking I realised that increasing the sophistication of the reason why to give up the wallet is not a good strategy. However asking for any "innocent" target is likely to encounter a lot more suggestibility. Some of these suggestions might put you in the position to better grab the wallet. That is a "Hey what's over there?" and pointing away and then physically grabbing the wallet is likely to be more effective than a sophisticated argument why to give the wallet. The strange thing here seems to be that if the target percieves your motivation to be the wallet it's effectively game over to you as the thief. A lot of the targets psychological defences know to activate on that cue.
The surprising result that I ended up on upon thinking is that the phenomenon is a legit psychological defence and it's presence is actually constructive. But abstracting it into other spheres it means that in situations where we are handling requests to system crtical phenomena we have a increased weight to understand the request to a higher degree. It's not that the agent goes emotional and throws reason out of the window. It's precisely opposite in that the agent correctly identifies that this needs to be understood and processed correctly. And it can't be blanket rejected because there is a minority of the situations where we actually want to comply with the request.
There is also a principle of a kind of burden of proof here. If I don't understand it I am going to reject it. Even if you use logic that is "more intellectual" than me. This burden of proof doesn't apply normally. Normally "you know what you are doing" can be a reason to give you the benefit of doubt. I don't need to know where you are about to walk to step out of your way. But in these high-stakes situations I do need to know the details.
The ultimate such high-stakes situation would be the AI-boxing situation. In basic communication when people hear about the problem they liken it more to the "Can you pass the salt, please" kind of problem or pattern recognise "friendliness" as a kind of academic curiosity. A method of communication that would liken it to the asking of wallet would make poeple employ their latent psyhological skills to the problem. Here is one mini attempt at it. Situation 1: You have hitler in a cell and he asks you to let him go out of the cell. You reply "No, you are f Hitler". Situation 2: you have Hitler and a innocent person in a cell. For some strange reason you don't know how Hitler looks like. A man beyond the bars asks "Let me, go I am innocent", you reply "No you are Hitler trying to lie you are the innocent one to get out jail", "What can I do to prove to you I am not Hitler?". In this kind of situation it's clear that letting an innocent man walk is desirable but clearly not worth having to deal with Hitler again, even if we have no reason to think that Hitler is immidiately about to commit something bad.
As the AI-boxing problem was presented sometimes it felt it was presented as a unique problem perhaps requiring unique answers. But the variable suggestibility scales kind of highlights that natural intelligences box each other all the time already. We are capable of trusting each other in some situations but also capable of forgoing trust when it's neccesary.
Comments sorted by top scores.