LessWrong 2.0 Reader

Sure. At this point I agree that some people will be so foolish and stubborn that no demo will concern them. Indeed, some people fail to update even on actual events.

So now we are, as Zvi likes to say, 'talking price'. What proportion of key government decision-makers would be influenced by how persuasive (and costly) of demos.

We both agree that the correct amount of effort to put towards demos is somewhere between nearly all of our AI safety effort-resources, and nearly none. I think it's a good point that we should try to estimate how effective a demo is likely to be on some particular individual or group, and aim to neither over nor under invest in it.

I think we can use the same method Eliezer applied to the regular epiphenomenalist Zombie argument to deal with this, weaker one.

Whether your mind interprets certain colour in a certain way actually has causal effects on the world. Namely, things that appear beautiful to you in our world may not appear beautiful to your qualia inversed counterpart. Which naturally affects your behaviour: whether you look at a certain object more, whether you buy a certain object and so on.

This is even more obvious for people with selective colour blindness. Suppose your mind is unable to distinguish between qualia of blueness and redness. And suppose there are three objects: A is red, B is blue and C is green. In our world you can't distinguish between objects A and B. But in the qualia inversed world you wouldn't be able to distinguish between objects B and C.

And if you try to switch to substance dualist version - all the reasoning from this post still stands.

Didn't like the post then, still don't like it in 2024. I think there are defensible points interwoven with assumptions and stereotypes. 

First: generalizes from personal experiences that are not universal. I think a lot of people don't have this or don't struggle with this or find it worth it, and the piece assumes everyone feels the way the author feels.  

Second: the thing it describes is a bias, and I don't think the essay realizes this.

Okay, part of the thing is that this doesn't make a case or acknowledge this romantic factor as being different from, like, friendship. Like, in the people-at-work case, you might also do someone a favor at work because you like them as a buddy, which is not necessarily the same as whether they're a good worker or it's a strategic thing for you to do, or whatever - you're inclined to give your friends special treatment. Even in straight same-gender groups, people will end up being friends and having outgroups.

Anyway, you have to be careful reasoning out of "what your in-built stereotypes say". This is sometimes relevant information, totally. But A) your in-built stereotypes are not everyone else's in-built stereotypes, even within your culture, and B) this is reasoning from the territory, not the map. Are they true? In some of the cases given in this piece, it matters if they're true.

Like, the thing being described here is a bias, a flaw in the lens. "Having to navigate around possible sexual dynamics with other people makes it harder to do regular communication with them" is a thing that'll make you less able to reason about them and less effective. (Especially if it still fires strongly in cases like "this woman is at this event about an unrelated topic, with a partner, and so is probably not available for dating.") I don't begrudge the author for having it. I think it's really common. God knows my own best judgment has failed me before in the face of very pretty people.

But I like this community for usually not giving up on matters of self-improvement and epistemics. Even if you don't prioritize it, you're at least recognizing it and not throwing it out. It's very disconcerting to read "I notice my brain does extra work when I talk with women... wouldn't it be easier if society were radically altered so that I didn't have to talk with women?" Like, what? And there's no way you or anyone else can become more rational about this? This barrier to ideal communication with 50% of people is insurmountable? It's worth giving up on this one? Hello?

I get that the author views this as sort of a series of tenuous hypotheticals and doesn't necessarily stand by these stances and was just putting it out there, which is respectable. I think it's wrong and so tenuous as to be unhelpful.

Overall: bad takes, did have a solid 20 seconds of mixed fun and horror imagining this totally-unsexist society where straight men and women are kept in polite segregated groups, and 10% of people are in fringe situations - stable lesbian gay-male duos who must rely on each other, the bisexuals and the nonbinary people wandering the earth alone, the asexuals reigning supreme; incorruptible, masters of all domains.

I was pointing to this quote:

It sounds as though you're imagining that we can proliferate the one case in which we caught the AI into many cases which can be well understood as independent (rather than basically just being small variations).

and this comment, which talks about proliferating cases where 1 AI schemes into multiple instances to get more evidence:

https://www.lesswrong.com/posts/YTZAmJKydD5hdRSeG/#BkdBD5psSFyMaeesS [LW · GW]

The question is how we should extrapolate, and in particular if we should extrapolate faster than experts currently predict. You would need to show that Willow represents unusually fast progress relative to expert predictions. It's not enough to say that it seems very impressive.

There are all sorts of "strategies" (turn it off, raise it like a kid, disincentivize changing the environment, use a weaker AI to align it) that people come up with when they're new to the field of AI safety, but that are ineffective. And their ineffectiveness is only obvious and explainable by people who specifically know how AI behaves.

yep but the first three all fail for the shared reason of "programs will do what they say to do, including in response to your efforts". (the fourth one, 'use a weaker AI to align it', is at least obviously not itself a solution. the weakest form of it, using an LLM to assist an alignment researcher, is possible, and some less weak forms likely are too.)

when i think of other 'newly heard of alignment' proposals, like boxing, most of them seem to fail because the proposer doesn't actually have a model of how this is supposed to work or help in the first place. (the strong version of 'use ai to align it' probably fits better here)

(there are some issues which a programmatic model doesn't automatically make obvious to a human: they must follow from it, but one could fail to see them without making that basic mistake. probable environment hacking  and decision theory issues come to mind. i agree that on general priors this is some evidence that there are deeper subjects that would not be noticed even conditional on those researchers approving a solution.)

i guess my next response then would be that some subjects are bounded, and we might notice (if not 'be able to prove') such bounds telling us 'theres not more things beyond what you have already written down', which would be negative evidence (strength depending on how strongly we've identified a bound). (this is more of an intuition, i don't know how to elaborate this)

(also on what johnswentworth wrote: a similar point i was considering making is that the question is set up in a way that forces you into playing a game of "show how you'd outperform magnus carlsen {those researchers} in chess alignment theory" - for any consideration you can think of, one can respond that those researchers will probably also think of it, which might preclude them from actually approving, which makes the conditional 'they approve but its wrong'^[1] harder to be true and basically dependent on them instead of object-level properties of alignment.)

i am interested in reading more arguments about the object-level question if you or anyone else has them.

If the solution to alignment were simple, we would have found it by now [...] That there is one simple thing from which comes all of our values, or a simple way to derive such a thing, just seems unlikely.

the pointer to values [LW · GW] does not need to be complex (even if the values themselves are)

If the solution to alignment were simple, we would have found it by now

generally: simple things don't have to be easy to find. the hard part can be locating them within some huge space of possible things. (math (including is use in laws of physics) come to mind?). (and specifically to alignment: i also strongly expect an alignment solution to ... {have some set of simple principles from which it can be easily derived (i.e. whether the program itself ends up long)}, but idk if i can legibly explain why. real complexity usually results from stochastic interactions in a process, but "aligned superintelligent agent" is a simply-defined, abstract thing?)

1. ^{^}

   ig you actually wrote 'they dont notice flaws', which is ambiguously between 'they approve' and 'they don't find affirmative failure cases'. and maybe the latter was your intent all along.

   it's understandable because we do have to refer to humans to call something unintuitive.

People who agree with the assassination want the assassin to agree with their politics.
People who disagree with the assassination want the assassin to agree with their political adversaries' politics.

He'll be used for partisan mud-slinging, as practically everything that's on the news in America is.

Watching that Cruise video for the first time, I'm struck by how patient and conservative the system is when encountering obstacles or waiting for pedestrians to cross, it doesn't try to sneak or squeeze past double parked vans in a hurry for example.

I wonder how much safer roads could be if human drivers were all a little more patient? Although, as I understand it the chief benefit of FSD is that it is highly predictable and doesn't make impulsive choices.

This was a fun little exercise. We get many "theory of rationality" posts on this site, so it's very good to also have some chances to practice figuring out confusing things also mixed in. The various coins each teach good lessons about ways the world can surprise you.

Anyway, I think this was an underrated post, and we need more posts in this general category.