LessWrong 2.0 Reader

There are all sorts of "strategies" (turn it off, raise it like a kid, disincentivize changing the environment, use a weaker AI to align it) that people come up with when they're new to the field of AI safety, but that are ineffective. And their ineffectiveness is only obvious and explainable by people who specifically know how AI behaves.

yep but the first three all fail for the same reason: not believing (enough) that programs will do what they actually say to do, including in response to your actions. (the fourth one, 'use a weaker AI to align it', is at least that obviously not itself a solution. the weakest form of it, using an LLM to assist an alignment researcher, is possible, and some less weak forms likely are too.)

(there are some issues which a programmatic model doesn't automatically make obvious to a human: they must follow from it, but one could fail to see them without making that basic mistake. probable environment hacking  and decision theory issues come to mind. i agree that on general priors this is a bit of evidence that there are deeper subjects that would not be noticed even conditional on those researchers approving a solution.)

i guess my next response then would be that some subjects are bounded, and we might notice (if not 'be able to prove') such bounds telling us 'theres not more things beyond what you have already written down', which would be negative evidence. (this is more of an intuition, i don't know how to elaborate this)

(also on what johnswentworth wrote. a similar point i was considering making is that you have set up the question such that you're forced into playing a game similar to "show how you'd outsmart magnus carlsen in chess" - for any consideration you can think of, one can respond that eliezer and others will probably also think of it, which might preclude them from actually approving, which makes your conditional 'they approve but its wrong' harder to be true and basically dependent on them instead of object-level properties of alignment. i think it would be better to abstract the question of what they'd think away, and also abstract away whether they might approve out of having no better option, and just ask: are there likely to be 'unintuitive' classes of issues that haven't yet been noticed, conditional on agent foundations researchers creating something that looks like an alignment solution and which they do not notice flaws in after lots of focus?)

i am interested in reading more arguments about that object-level question if you or anyone else has them. it's still about human minds, but only to the extent it has to be for a problem to be called unintuitive.

If the solution to alignment were simple, we would have found it by now [...] That there is one simple thing from which comes all of our values, or a simple way to derive such a thing, just seems unlikely.

the pointer to values does not need to be complex [LW · GW] (even if the values themselves are)

People who agree with the assassination want the assassin to agree with their politics.
People who disagree with the assassination want the assassin to agree with their political adversaries' politics.

He'll be used for partisan mud-slinging, as practically everything that's on the news in America is.

Watching that Cruise video for the first time, I'm struck by how patient and conservative the system is when encountering obstacles or waiting for pedestrians to cross, it doesn't try to sneak or squeeze past double parked vans in a hurry for example.

I wonder how much safer roads could be if human drivers were all a little more patient? Although, as I understand it the chief benefit of FSD is that it is highly predictable and doesn't make impulsive choices.

This was a fun little exercise. We get many "theory of rationality" posts on this site, so it's very good to also have some chances to practice figuring out confusing things also mixed in. The various coins each teach good lessons about ways the world can surprise you.

Anyway, I think this was an underrated post, and we need more posts in this general category.

Then recently we have the example where an 11-year-old (!) walked less than a mile into a 370-person town, and the mother was charged with reckless conduct and forced to sign a ‘safety plan’ on pain of jail time pledging to track him at all times via an app on his phone.

Though there was some pushback that the mother did not know where the kid was, this still seems confusing given rules around school commutes. Many schools do not provide bus service within half a mile up the school, expecting kids to walk or bicycle. In Alaska, it was 1.5 miles even though it got down to -40°! And there generally does not appear to be an age limit where parents are required to go with their kids, so it sounds like it's okay for a 5-year-old to do this.

Sorry, on my phone for a few days, but iirc in ch. 3 they consider the loss you get if you just predict according to the simplest hypothesis that matches the data (and show it's bounded).

Thank you for this insight!

I agree that certain demos might change the mind of certain people. (And if so, they’re worthwhile.) But I also think other people would be immune. For example, suppose someone has the (mistaken) idea: “Nobody would be so stupid as to actually press go on an AI that would then go on to kill lots of people! Or even if theoretically somebody might be stupid enough to do that, management / government / etc. would never let that happen.” Then that mistaken idea would not be disproven by any demo, except a “demo” that involved lots of actual real-life people getting killed. Right?

I don't think this makes much sense. In a regulated industry, you want to build up a positive reputation and working relationship with the regulators, where they know what to expect from you, are familiar with your work and approach, have a sense of where you're going, and generally like and trust you. Engaging with him early and then repeatedly over a long period seems like a way better strategy than waiting until you have something extremely ambitious to try to get them to approve.

I am more optimistic that we can get such empirical evidence for at least the most important parts of the AI risk case, like deceptive alignment, and here's one reason as comment on offer:

Can you elaborate on what you were pointing to in the linked example? The thread specifically I’ve seen a few people mention recently but I seem to be missing the conclusion they’re drawing from it.