LessWrong 2.0 Reader

To be clear, I think that embedding human values is part of the solution  - see my comment [LW(p) · GW(p)]  

Always happy to chat!

To be clear, I want models to care about humans! I think part of having "generally reasonable values" is models sharing the basic empathy and caring that humans have for each other. 

It is more that I want models to defer to humans, and go back to arguing based on principles such as "loving humanity" only when there is gap or ambiguity in the specification or in the intent behind it. This is similar to judges: If a law is very clear, there is no question of the misinterpreting the intent, or contradicting higher laws (i.e., constitutions) then they have no room for interpretation. They could sometimes argue based on "natural law" but only in extreme circumstances where the law is unspecified. 

One way to think about it is as follows: as humans, we sometimes engage in "civil disobedience", where we break the law based on our own understanding of higher moral values. I do not want to grant AI the same privilege. If it is given very clear instructions, then it should follow them.  If instructions are not very clear, they may be a conflict, or we are in a situation not forseen by the authors of the instructions, then AIs should use moral intuitions to guide them. In such cases there may not be one solution (e.g., a conservative and liberal judges may not agree) but there is a spectrum of solutions that are "reasonable" and the AI should pick one of them. But AI should not do "jury nullification".

To be sure, I think it is good that in our world people sometimes disobey commands or break the law based on higher power. For this reason, we may well stipulate that certain jobs must have humans in charge. Just like I don't think that professional philosophers or ethicists have necessarily better judgement than random people from the Boston phonebook, I don't see making moral decisions as the area where the superior intelligence of AI gives them a competitive advantage, and I think we can leave that to humans.

Since I am not a bio expert, it is very hard for me to argue about these types of hypothetical scenarios. I am even not at all sure that intelligence is the bottleneck here, whether on defense or the offense side.

I agree that killing 90% of people is not very reassuring, this was more a general point why I expect the effort to damage curve to be a sigmoid rather than a straight line.

Right now I think you can replace junior programmers with Claude 3.5 Sonnet or even better with one of the agents based on a looped chain of thoughts + access to tools.

On the other hand, it does not yet go in that direction for being a preferred way to work with models for more advanced devs. Not for me, and not for many others.

Models still have strange moments of "brain farts" or gaps in their cognition. It sometimes makes them do something wrong and cannot figure out how to do that correctly until told exactly how. They also often miss something.

When writing code if you make such an error and build on top of that mistake, you might end up having to re-write or at least analyze and modify a lot of code. This makes people like me prefer to work with models in smaller steps. Not as small as line by line or function by function, but often one file at a time and one functionality/responsibility at a time. For me, it is often a few smaller functions that realize more trivial things + one gathering them together into one realizing some non-trivial responsibility.

The only safety techniques that count are the ones that actually get deployed in time.

True, but note this doesn't necessarily imply trying to maximize your impact in the mean timelines world! Alignment plans vary hugely in potential usefulness, so I think it can pretty easily be the case that your highest EV bet would only pay off in a minority of possible futures.

But anyway, it sometimes seems to me that you often advocate a morality regarding AI relations that doesn't benefit anyone who currently exists, or, the coalition that you are a part of. This seems like a mistake. Or worse.

I dispute this, since I've argued for the practical benefits of giving AIs legal autonomy [EA · GW], which I think would likely benefit existing humans. Relatedly, I've also talked about how I think hastening the arrival AI could benefit people who currently exist. Indeed, that's one of the best arguments for accelerating AI. The argument is that, by ensuring AI arrives sooner, we can accelerate the pace of medical progress, among other useful technologies. This could ensure that currently-existing old people who would otherwise die without AI will be saved and live a longer and healthier life than the alternative.

(Of course, this must be weighed against concerns about AI safety. I am not claiming that there is no tradeoff between AI safety and acceleration. Rather, my point is that, despite the risks, accelerating AI could still be the preferable choice.)

However, I do think there is an important distinction here to make between the following groups:

1. The set of all existing humans
2. The human species itself, including all potential genetic descendants of existing humans

Insofar as I have loyalty towards a group, I have much more loyalty towards (1) than (2). It's possible you think that I should see myself as belonging to the coalition comprised of (2) rather than (1), but I don't see a strong argument for that position. 

To the extent it makes sense to think of morality as arising from game theoretic considerations, there doesn’t appear to be much advantage for me in identifying with the coalition of all potential human descendants (group 2) rather than with the coalition of currently existing humans plus potential future AIs (group 1 + AIs) . If we are willing to extend our coalition to include potential future beings, then I would seem to have even stronger practical reasons to align myself with a coalition that includes future AI systems. This is because future AIs will likely be far more powerful than any potential biological human descendants.

I want to clarify, however, that I don't tend to think of morality as arising from game theoretic considerations. Rather, I mostly think of morality as simply an expression of my personal preferences about the world.

Thought on short timelines. Opinionated.

I think that AGI timelines might be very short based on an argument taken from a different side of things.

We all can agree that humans have general intelligence. If we look at how our general intelligence evolved from simpler forms of specific intelligence typical for animals - it wasn't something that came from complex interactions and high evolutional pressure. Basically there were two aspects of that progress. The first one is the ability to pass on knowledge through generations (culture). Something that we share with some other animals including our cousins chimpanzee. The second one is intersexual selection - at some moment in the past, our species started to have sexual preferences based on the ability to gossip and talk. It is still there, even if we are not 100% aware of that - our courtship, known as dating, is based mostly on meeting together and talking. People who are not talkative and introverts, even if successful, have a hard time dating.
These two things seem to be major drivers for us to both develop more sophisticated language and better general intelligence.

It seems to me that this means that there are not many pieces missing from using current observations and some general heuristics like animals do, to have full-fledged general intelligence.

It also suggests that you need some set of functions or heuristics, possibly a small set, together with a form of external memory, to tackle any general problem by dividing it into smaller bits and rejoining sub-solutions into a general solution. Like a processor or Turing machine that has a small set of basic operations, but can in principle run any program.

Oh, my bad, yeah. When I was writing the comment, I flipped the direction of advantage for longer time controls (longer time controls are actually better for humans in odds matches of course), but this way I agree it's unclear a priori whether 200 elo drop would be enough to account for longer time controls.

I think this comment is either incredibly stupid or extremely insightful and borderline genius. I honestly can't tell (although I'm leaning towards the former, and the karma seems to agree with me), and that scares me just a little.