LessWrong 2.0 Reader

We have been shown that this search algorithm works, and we not yet have been shown that the other approaches don't work.

Remember, technological development is disjunctive, and just because you've shown that 1 approach works, doesn't mean that we have been shown that only that approach works.

Of course, people will absolutely try to scale this one up now that they found success, and I think that timelines have definitely been shortened, but remember that AI progress is closer to a disjunctive scenario than conjunctive scenario:

I agree with this quote below, but I wanted to point out the disjunctiveness of AI progress:

As we spoke earlier - it was predictable that this was going to be the next step. It was likely it was going to work, but there was a hopeful world in which doing the obvious thing turned out to be harder. That hope has been dashed - it suggests longer horizons might be easy too. This means superintelligence within two years is not out of the question.

https://gwern.net/forking-path

Suppose you have a randomly activated (not dependent on weather) sprinkler system, and also it rains sometimes. These are two independent causes for the sidewalk being wet, each of which are capable of getting the job done all on their own. Suppose you notice that the sidewalk is wet, so it definitely either rained, sprinkled, or both. If I told you it had rained last night, your probability that the sprinklers went on (given that it is wet) should go down, since they already explain the wet sidewalk. If I told you instead that the sprinklers went on last night, then your probability of it having rained (given that it is wet) goes down for a similar reason. This is what "explaining away" is in causal inference. The probability of a cause given its effect goes down when an alternative cause is present.

In the post, the supposedly independent causes are "hardcoded ant-in-mouth aversion" and "value of eating escamoles", and the effect is negative reward. Realizing that you have a hardcoded ant-in-mouth aversion is like learning that the sprinklers were on last night. The sprinklers being on (incompletely) "explain away" the rain as a cause for the sidewalk being wet. The hardcoded ant-in-mouth aversion explains away the-amount-you-value-escamoles as a cause for the low reward.

I'm not totally sure if that answers your question, maybe you were asking "why model my values as a cause of the negative reward, separate from the hardcoded response itself"? And if so, I think I'd rephrase the heart of the question as, "what do the values in this reward model actually correspond to out in the world, if anything? What are the 'real values' which reward is treated as evidence of?" (We've done some thinking about that and might put out a post on that soon.)

I'm only vaguely connected to EA in the sense of donating more-than-usual amounts of money in effective ways (❤️ GiveDirectly), but this feels like a strawman. I don't think the average EA would recommend charities that hurt other people as side effects, work actively-harmful jobs to make money^[1], or generally Utilitarian-maxxing.

The EA trolley problem is that there are thousands (or millions) of trolleys that have varying difficult of stopping, barreling toward varying groups of people. The problem isn't that stopping them hurts other people (it doesn't), it's just that you can't stop them all. You don't need to be a utilitarian to think that if it's raining planes, Superman should start by catching the 747's.

1. ^{^}

   For example, high-paying finance jobs are high-stress and many people don't like working them, but they're not actually bad for the world.

Emotions represent swift, comprehensive assessments that enable an organism to promptly evaluate the importance of stimuli in relation to its survival and overall well-being.

It seems to me that the ability of emotions to evaluate/categorize stimuli is descriptively correct, but there is a large gap between that and a prescription. Emotions surely do act as shortcut heuristics to detail with the excess of detail, however this can lead to as many mistakes as there are successes. Moreover, AI already is capable of coming up with statements or decisions despite the immense quantity of data at its disposal.

Said another way, if "emotion" can be reduced to "decision making before sufficient detail is understood", well it seems like we're already there.

Perhaps the broader point is that there need not be any "actual" emotion for us to believe there is--there doesn't have to be a "ghost in the machine".

An AI also does not need some emotional capacity for a successful operation of ethics either as that can be (arbitrarily) programmed in. Or there could theoretically be a 50/50 dice roll between utilitarianism and deontology to give the impression of "compassionate moral reasoning"--all the AI would need to do is give an ex post facto explanation for their decision, along with adding something like "I regret the losses that were had along the way".

All of this to say, I don't believe emotions are necessary to have a fully functioning artificial intelligence--such "emotional" capacities would merely affect our interactions/feelings towards it.

Lastly, your example of the prompt you gave to 4o is illustrative but imperfect. Within the prompt, it is not clear that what is requested is a more emotionally intelligent result--a more specific prompt, or a more expansive memory could tease out the type of answer you want. That being said, the requirement to make a more specific prompt at all is indeed a difficulty as it feels like, in day to day conversation with others, there is less need to be as specific. That being said, that difference may be an illusion. After all, a day to day conversation has much more data given it is face to face, with someone you know, etc. As such, the importance of specificity with a prompt remediates the lack of all that data.

An interesting thought experiment: imagine if we had no facial expressions and our tones of voice didn't change. If this were the case, our language would have to do most of the work, and therefore we would have a much more complex language referring to attitudes, emotions, expectations, etc.--all things that our normal gesticulations already take care of.

The main observation from the quoted block is "man, this sure sounds like explaining away, if I'm treating my hardcoded reactions as a signal which is sometimes influenced by things besides values". But exactly when do I treat my hardcoded reactions as though they're being influenced by non-value stuff? I don't know yet; I don't yet understand that part.

I’m curious to understand that a bit better, if you don’t mind (and happen to be familiar enough with the German legal system to answer). Which of the following would a German judge commonly do in the course of an ordinary proceeding?

(i) Ask a witness to clarify statements made;

(ii) ask a witness new questions that, while relevant to the case, do not constitute clarifications of previous statements made;

(iii) summon new witnesses (including but not limited to expert witnesses) without application from either party;

(iv) compel a party to produce documents not in discovery, without application from the other party;

(v) compel third parties to produce documents neither party has requested be produced.

All the above used to be pretty standard in most jurisdictions AFAIK. But what tends to happen nowadays is that either some of those are expressly disallowed, or else, while judges may well retain legal authority to perform all those kinds of digging, in practice that authority is used very sparingly.

May I ask whether there is anything planned on fixing this rendering/loading bug [LW(p) · GW(p)] which occurs with Firefox? It affects unread/uncached posts opened in a background tab.

Of course I do. 

It seems very plausible to me that alignment targets in practice will evolve out of things like the OpenAI Model Spec. If anyone has suggestions for how to improve that, please DM me.

In theories of Bayesianism, the axioms of probability theory are conventionally assumed to say that all logical truths have probability one, and that the probability of a disjunction of logically inconsistent statements is the sum of their probabilities. Corresponding to the second and third Kolmogorov axiom.

If one then e.g. regards the Peano axioms as certain, then all theorems of Peano arithmetic must also be certain, because those are just logical consequences. And all statements which can be disproved in Peano arithmetic then must have probability zero. So the above version of the Kolmogorov axioms is assuming we are logically omniscient. So this form of Bayesianism doesn't allow us to assign anything like 0.5 probability to the googolth digit of pi being odd: We must assign 1 if it's odd, or 0 if it's even.

I think the simple solution is to not talk about logical tautologies and contradictions when expressing the Kolmogorov axioms for a theory of subjective Bayesianism. Instead talk about what we actually know a priori, not about tautologies which we merely could know a priori (if we were logically omniscient). Then the second Kolmogorov axiom says that statements we actually know a priori have to be assigned probability 1, and disjunctions of statements actually known a priori to be inconsistent have to be assigned the sum of their probabilities.

Then we are allowed to assign probabilities less than 1 to statements where we don't actually know that they are tautologies, e.g. 0.5 to "the googolth digit of pi is odd" even if this happens to be, unbeknownst to us, a theorem of Peano arithmetic.