post by [deleted] · · ? · GW · 0 comments

This is a link post for

0 comments

Comments sorted by top scores.

comment by Gunnar_Zarncke · 2023-11-26T09:01:17.369Z · LW(p) · GW(p)

I don't think biological superintelligence completely avoids the safety problems. You can see it already with very powerful humans. Imagine a very powerful human dictator would live forever. 

Replies from: carado-1, Yarrow Bouchard
comment by Tamsin Leake (carado-1) · 2023-11-26T15:13:13.298Z · LW(p) · GW(p)

what you describe wouldn't be super-great, but it would address the challenge of alignment [LW · GW]:

all I am looking at is prospective results, all I want is that we have justifiable cause to believe of a pivotally useful AGI 'this will not kill literally everyone'. Anybody telling you I'm asking for stricter 'alignment' than this has failed at reading comprehension. The big ask from AGI alignment, the basic challenge I am saying is too difficult, is to obtain by any strategy whatsoever a significant chance of there being any survivors.

comment by [deactivated] (Yarrow Bouchard) · 2023-11-26T12:59:25.171Z · LW(p) · GW(p)

I guess we could say governance remains a problem with biological superintelligence? As it does with normal humans, just more so.

Replies from: mishka
comment by mishka · 2023-11-26T14:49:48.163Z · LW(p) · GW(p)

Yes, nevertheless the S-risk and X-risk problems don't go away. There are humans who like causing suffering. There are human advocating for human extinction (and some of them might act on that given the capabilities). There are humans who are ready to fight wars with weapons which might cause extinction, or would be ready to undertake projects which might cause extinction or widespread suffering.

Stepping back, we know that Eliezer was very much against anthropomorphic superintelligences in 2011. He thought we needed much higher levels of safety ("provably friendly AI", which would not be possible with something as messy as human-like systems). Since then he strongly updated towards pessimism regarding our chances to create beneficial artificial superintelligence, and he arrived at the conclusion that our chances with biological superintelligence might be higher.

But it would be good to try to articulate what are the reasons for our chances with biological superintelligence to be higher.

One aspect is that we do have an intuition that biology-based systems are likely to self-improve slower, and thus would have more time to ponder solutions to various issues as they get smarter. So they might be not superintelligent, but just very smart for quite a while, and during that period they would decide what to do next. Another aspect is that biology-based systems are more likely to be automatically sentient, and their sentience is more likely to be at least somewhat similar to ours, and so even if things go badly initially, the chances for having a lot of value in the future lightcone are higher, because it is more likely that there would be first-person experiencers.

But it would be good to pause and think whether we are sure. Also speaking of

Brain implants/brain-computer interfaces: Devices under development by companies such as Neuralink, Kernel, Openwater, and Meta’s Reality Labs. Could hypothetically enhance human intelligence.

these devices can also lead to the hybrid human-AI systems, and that might be a more technologically likely route. The hybrid system becomes smarter, both because of its biological part working better, but also because of a tight coupling with an AI thinking part. In some cases of BCI use, it might be difficult to distinguish between straight human intelligence enhancement and creation of a hybrid thinker. We might want to ponder whether this is a desirable route. (I personally find this route very attractive for a number of reasons, but safety issues along this route are quite acute as well).

Replies from: Yarrow Bouchard
comment by [deactivated] (Yarrow Bouchard) · 2023-11-27T06:33:11.192Z · LW(p) · GW(p)

Potential solutions to foreseeable problems with biological superintelligence include: a) only upgrading particularly moral and trustworthy humans or b) ensuring that upgrading is widely accessible, so that lots of people can do it.

Replies from: mishka
comment by mishka · 2023-11-27T07:01:13.959Z · LW(p) · GW(p)

b) does not solve it without a lot of successful work on multipolar safety (it's almost an equivalent of giving nuclear weapons to lots of people, making them widely accessible; and yes, giving gain-of-function labs equipment too)

a) is indeed very reasonable, but we should keep in mind that upgrade is a potentially stronger impact than any psychoactive drugs, a potentially stronger impact than any most radical psychedelic experiences. Here the usual "AI alignment problem" one is normally dealing with is replaced by the problem of conservation of one's values and character.

In fact these problems are closely related. The most intractable part of AI safety is what happens when AI ecosystems starts to rapidly recursively self-improve, perhaps with significant acceleration. We might have current members of AI ecosystem behave in a reasonably safe and beneficial way, but would future members (or same members after they self-improve) behave safely, or would "a sharp left turn" happen?

Here it is the same problem for a rapidly improving and changing "enhanced human", would that person continue to maintain the original character and values while undergoing radical changes and enhancements, or would drastic new realizations (potentially more radical than any psychedelic revelations) lead to unpredictable revisions of that original character and values?

It might be the case that it's easier to smooth these changes for a human (compared to AI), but the success is not automatic by any means.

comment by cousin_it · 2023-11-27T13:20:25.629Z · LW(p) · GW(p)

I think these are all still pretty bad. For example if there are human uploads but no stronger AI, that will lead to horrors ("data can't defend itself" - Sam Hughes). If there are biological superbrains, same. Look at what we humans did with our intelligence, we've introduced a horror (factory farms and fish farms) that surpasses anything in nature. Going forward we must take two steps in morality for every step in capability, otherwise horrors will increase proportionally.

The physical world with all its problems is kind of a marvel, in that it allows a degree of individualism, locality, a speed limit. One can imagine that it was engineered by creatures from a more nasty world, who one day said: you know what, let's build a system where creatures fundamentally cannot be hurt and erased at a distance, or at least it's much harder to do so. That's just a metaphor, but to me the holy grail of AI safety would be building the next world in this metaphorical chain. Where we'd still have to live by the sweat of our brow, but where the guardrails would be a little bit stronger than in our world.

For example, one could imagine a world where game theory is less of a cruel master than it is in ours, where "no need for game theory" is written into the base bricks of existence. A world that is a challenge but not a battleground. Designing and building such a world would be a monumentally difficult task, but that's what superintelligence is for.

comment by Mitchell_Porter · 2023-11-26T21:42:45.523Z · LW(p) · GW(p)

I use the term “biological superintelligence” to refer to superhuman intelligences that have a functional architecture that closely resembles that of the natural human brain. A biological superintelligence does not necessarily have an organic substrate. 

I would use a slightly refined definition, according to which biological superintelligence per se does necessarily have an organic substrate. A superintelligent mind upload could be called a biologically descended superintelligence, and a neuromorphic AI could be called a biologically inspired superintelligence. 

("Neuromorphic" itself is a broad fuzzy term. To some extent, all artificial neural networks are already brain-inspired.) 

Biological superintelligence is the holy grail of AI safety because it solves the alignment problem and the control problem by avoiding them entirely. 

This isn't true, for reasons which illuminate the nature of problems pertaining to superintelligence. 

Natural human brains do not contain superintelligence. Therefore, to make them superintelligent, you either have to add something to them, or change something in them. If you change something in them, you're potentially changing the parts of the individual or the parts of human nature that matter for alignment. If you add something to them, it's the same situation as a human with an external AI. 

Biohacking, neurohacking, and uploading contain their own risks. The obvious risk is that you kill or injure yourself. A more subtle risk is that you change yourself in a way that you wouldn't actually have wanted. This second kind of "risk" encompasses a continuum from clearly undesirable outcomes, e.g. after the change you end up believing things that you would never have wanted to believe, through to very subtle things which are on a par with the ambiguous decisions of existing everyday life. 

But to consider these topics is far from a waste of time. Neuroscience matters for at least two reasons. One is that knowledge of the human brain has often been considered essential to the creation of deeply aligned AI; all the way from the days of CEV, through to more recent proposals like Metaethical AI. The other is that the theory and practice of safely aligning brainlike AI [? · GW] is a natural source of ideas for the presently unknown theory and practice of safely modifying and enhancing biological human brains.