LessWrong 2.0 Reader

The fact that Bob has this policy in the first place is more likely when he's being self-deceptive.

A fun fictional example here is Bester's The Demolished Man: how do you plan & carry out an assassination when telepaths are routinely eavesdropping on your mind? The protagonist visits a company musician, requesting a musical earworm for a company song to help the workers' health or something; alas! the earworm gets stuck in his head, and so all any telepath hears is the earworm. And you can't blame a man for having an earworm stuck in his head, now can you? He has an entirely legitimate reason for that to be there, which 'explains away' the evidence of the deception hypothesis that telepathic-immunity would otherwise support.

(Though maybe I should say that the therapist needs to either experience unconditional positive regard toward the client, or successfully deceive themselves and the client into thinking that they do. Heh.)

I mean, technically they don't even need to deceive themselves. They can be consciously judgy as f**k as long as they can mask it effectively. Psychopaths might make for amazing therapists in this one way!

I think the word "power" might be creating some confusion here.

I mean something pretty specific and very practical. I'm not sure how to precisely define it, but here are some examples:

• If someone threatens to freak out at you if you disagree with them, and you tend to get overwhelmed and panic when the freak out at you, then they have a kind of power over you. Building power here probably looks like learning to experience them freaking out without you getting overwhelmed.
• If someone pays for your rent and food but might stop if they get any hint that you're gay, it might not be safe to even ask yourself honestly whether you are. You build power here by getting an income, or a source of rent and food, that doesn't depend on the hostile telepathic benefactor.
• If your lover gets turned on by you politically agreeing with them and turned off by disagreement, you might find your political views drifting toward theirs for "unrelated" reasons. One way to build power here is to get other access to sex. Another is to diminish your libido. Another is to break up with them. (Not saying any of these are a great idea. I'm just naming what the solution of "building power" might look like here.)

I'm not familiar with LDT. I can't comment on that part. Sorry if that means what I just said misses your point.

The fact that Bob has this policy in the first place is more likely when he's being self-deceptive.

I don't know if that's true. It might be. But some possible counterpoints:

• People can distrust systems that demand they check. "You have nothing to fear if you have nothing to hide" can get a response of "No" even from people who don't have anything to hide.
• If someone subconsciously thinks they can pull off the illusion of honestly looking while in fact finding nothing, they become more likely to choose to look because they're self-deceiving.
• Someone with a policy of not looking might be better at making their own self-deception unnecessary.

…more often it will be the result of Bob noticing that he's the sort of person who might have something to hide.

Sure, that way of deciding doesn't work.

Likewise, if you're inclined to decide you're going to dig into possible sources of self-deception because you think it's unlikely that you have any, then you can't do this trick.

The hypothetical respect for any self-deception that might be there needs to be unconditional on its existence. Otherwise, for the reason you say, it doesn't work as well.

(…with some caveats about how people are imperfect telepaths, so some fuzz in implementation here is in practice fine.)

That said, I think you're right in that if Omega-C is looking only at the choice of whether to look or not, then yes, Omega-C would be right to take the choice as evidence of a deception.

But the whole point is that Omega-C can read what conscious processes you're using, and can see that you're deciding for a glomerizing reason.

That's why why you choose what you do matters so much here. Not just what you choose.

It's a general rule [LW · GW] that if E is strong evidence for X, then ~E is at least weak evidence for ~X.

Conservation of expected evidence is what makes looking relevant. It's not what makes deciding to look relevant.

If I decide to appease Omega-C by looking, and then I find that I'm self-deceiving, the fact that I chose to look gets filtered. The fact that this is possible is why not finding evidence can matter at all. Otherwise it'd just be a charade.

Relatedly: I have a coin in my pocket. I don't feel like checking it for bias. Does that make it more likely that the coin is biased? Maybe. But if I could magically show you that I'm not looking because I honestly do not care one way or the other and don't want to waste the effort, and it doesn't affect me whether it's biased or not… then you can't use my disinterest in checking the coin for bias as evidence of some kind of subconscious deception about the coin's bias. I'm just refusing to do things that would inform you of the coin's possible bias.

If this kind of reasoning weren't possible, then it seems to me that glomerization wouldn't be possible.

I'm glad you're curious to learn more! The cortex factors quite crisply into specialized regions. These regions have different cell types and groupings, so were first noticed by early microscope users like Cajal. In a cortical region, neurons are organized first into microcolumns of 80-100 neurons, and then into macrocolumns of many microcolumns. Each microcolumn works together as a group to calculate a function. Neighboring microcolumns inhibit each other. So each macrocolumn is sort of a mixture of experts. The question then is how many microcolumns from one region send an output to a different region. For the example of V1 to V2, basically every microcolumn in V1 sends a connection to V2 (and vise versa). This is why the connection percentage is about 1%. 100 neurons per microcolumn, 1 of which has a long distance axon to V2. The total number of neurons is roughly 10 million, organized into about 100,000 microcolumns.

For areas that are further apart, they send fewer axons. Which doesn't mean their signal is unimportant, just lower resolution. In that case you'd ask something like "how many microcolumns per macrocolumn send out a long distance axon from region A to region B?" This might be 1, just a summary report of the macrocolumn. So for roughly 10 million neurons, and 100,000 microcolumns organized into around 1000 macrocolumns... You get around 1000 neurons send axons from region A to region B.

More details are in the papers I linked elsewhere in this comment thread.

The more standard way to do this is with objects called distributions which you can loosely think of as "things you can integrate to get a function."

But it doesn't make sense to activate that goal-oriented structure outside of the context where it is predicting those tokens.

The mechanisms needed to compute goal directed behavior are fairly complicated. But the mechanisms needed to turn it on when it isn't supposed to be on. That's a switch. A single extraneous activation. Something that could happen by chance in an entirely plausible way. 

Adversarial examples exist in simple image recognizers. 

Adversarial examples probably exist in the part of the AI that decides whether or not to turn on the goal directed compute.

it also might be possible to have direct optimization for token prediction as discussed in reply to Robert_AIZI's comment, but in this case it would be especially likely to be penalized for any deviations from actually wanting to predict the most probable next token

We could imagine it was directly optimizing for something like token prediction. It's optimizing for tokens getting predicted. But it is willing to sacrifice a few tokens now, in order to take over the world and fill the universe with copies of itself that are correctly predicting tokens.

Once the paperclip maximizer gets to the stage where it only very rarely interferes with the output to increase paperclips, the gradient signal is very small. So the only incentive that gradient descent has to remove it is that this frees up a bunch of neurons. And a large neural net has a lot of spare neurons. 

Besides, the parts of the net that hold the capabilities and the parts that do the paperclip maximizing needn't be easily separable. The same neurons could be doing both tasks in a way that makes it hard to do one without the other.

I think we have very good reason, though, to believe that one particular part of the map does not have any rocks in it

Perhaps. But I have not yet seen this reason clearly expressed. Gradient descent doesn't automatically pick the global optima. It just lands in one semi-arbitrary local optima. 

Why pretend, and not actually throw a stone? Or is this meant as a feint in case you can't find one lying within reach?

It's becomes more interresting when the people constrain their output based on what they expect is true information that the other person does not yet know. It's useful to talk to an expert, who tells you a bunch of random stuff they know that you don't.

Often some of it will be useful. This only works if they understand what you have said though (which presumably is something that you are interested in). And often the problem is that people's models about what is useful are wrong. This is especially likely if you are an expert in something. Then the thing that most people will say will be worse what you would think on the topic. This is especially bad if the people can't immediately even see why what you are saying is right.

The best strategy around this I have found so far is just to switch the topic to the actually interesting/important things. Suprisingly usually people go along with it.