Observing Optimization

Should you expect more speciation after the invention of sex, or less

What meaning of species are you using? Under some definitions speciation doesn't make sense without sex... as a species is defined as a group of organisms capable of interbreeding.

I kinda like this whole back and forth between Robin and Eliezer. At the same time... The blog post v. blog post style almost feels like being a child stuck between two divorced parents who are ignoring each other but keep saying things very loudly so the other one can hear ;-)

So, like, every bacterium is its own species?

Bacteria have a form of sex. It is horizontal rather than vertical biological information transfer though. I don't know what the limits of the transfer are, but it might make sense to talk about groups of bacteria that can transfer genes between them as a species. Evolution is not a simple thing.

Eric, it's more amusing that both often cite a theorem that agreeing to disagree is impossible. And even more amusing that in "Nature of Logic" Eliezer practically explained agreeing to disagree: our mind is more cognition than logic. Eliezer and Robin generalize from facts to concepts differently which leads them to different predictions. When they try using logic to reconcile, logic kinda bottoms out at the concepts and there doesn't seem to be any way out except to test both theories. The argument goes on because both are polite and respectful, but it doesn't seem to shed any light.

(I apologize to the hosts for harping on the same topic repeatedly.)

If you can't usefully connect your abstractions to the historical record, I sure hope you have some data you can connect it to. Otherwise I can't imagine how you could have much confidence in them.

Eli: "I'd be surprised to learn that sex had no effect on the velocity of evolution. It looks like it should increase the speed and number of substituted adaptations, and also increase the complexity bound on the total genetic information that can be maintained against mutation."

Without crossover, the average rate of fitness gain with optimal mutation rates is 1/2 bit per genome per generation, and the maximum tolerable error rate is one error per genome per generation. For a fixed error probability m of each bit being flipped in reproduction, the largest possible genome size is of order 1/m.

With crossover, both the average rate of fitness gain and the tolerable errors per generation are equal to the square root of the genome size in bits; the largest possible genome size is of order 1/(m^2).

So yes, sex has an effect, and it's enormous. Rather than work out the math here and probably get it wrong, I'll point to an excellent (and downloadable) textbook that discusses the issue: MacKay's "Information Theory, Inference, and Learning Algorithms" at http://www.inference.phy.cam.ac.uk/mackay/itila/ . Chapter 19 contains the discussion of sexual vs. asexual evolution.

--Jeff

It might be difficult to measure evolutionary progress, but that is no excuse for ditching the whole concept:

Evolution is an optimisation process - similar to a genetic algorithm. We even know what the fitness function is - and so can see what is being optimised. Optimisation processes are - by their very nature - powerfully directional processes. There is a way in which such a process could fail to be directional - if it had too large a mutation rate. That might happen if our planet was extremely radioactive, or if it was frequently bombed by large meteorite strikes. However, in practice, neither of these things applies.

• http://alife.co.uk/essays/technology_explosion/

A brief examination of the history of life shows that it is characterised by an accumulation of "survival technology" - i.e. either adaptations or technology. In particular, the biosphere is accumulating "natural technology" that helps it more rapidly identify sources of potential energy - and degrade them in the process of constructing offspring. The accumulation is progressive, cumulative - and inexorable in character - and the resulting ratchet mechanism provides evolution with a powerful progressive, directional character.

• http://originoflife.net/direction/

If you can't usefully connect your abstractions to the historical record, I sure hope you have some data you can connect it to. Otherwise I can't imagine how you could have much confidence in them.

Depends on how much stress I want to put on them, doesn't it? If I want to predict that the next growth curve will be an exponential and put bounds around its doubling time, I need a much finer fit to the data than if I only want to ask obvious questions like "Should I find rabbit fossils in the pre-Cambrian?" or "Do the optimization curves fall into the narrow range that would permit a smooth soft takeoff?"

Eric, it's more amusing that both often cite a theorem that agreeing to disagree is impossible.

It's only impossible for rational Bayesians, which neither Hanson nor Yudkowsky are. Or any other human beings, for that matter.

Eliezer, it seems to me that we can't really debate much more until you actually directly make your key argument. If, at it seems to me, you are still in the process of laying out your views tutorial-style, then let's pause until you feel ready.

I think we ran into this same clash of styles last time (i.e. back at Oxford). I try to go through things systematically, locate any possible points of disagreement, resolve them, and continue. You seem to want to jump directly to the disagreement and then work backward to find the differing premises. I worry that this puts things in a more disagreeable state of mind, as it were - conducive to feed-backward reasoning (rationalization) instead of feed-forward reasoning.

It's probably also worth bearing in mind that these kinds of meta-discussions are important, since this is something of a trailblazing case here. And that if we really want to set up conditions where we can't agree to disagree, that might imply setting up things in a different fashion than the usual Internet debates.

When I attend a talk, I don't immediately jump on anything a speaker says that sounds questionable. I wait until they actually make a main point of their talk, and then I only jump on points that seem to matter for that main point. Since most things people say actually don't matter for their main point, I find this to be a very useful strategy. I will be very surprised indeed if everything you've said mattered regarding our main point of disagreement.

You guys should keep in mind the nature of the disagreement results. You don't have to go through all these details to reach agreement. The mere fact that each of you holds to his view even in light of the full knowledge that the other not only disagrees, but disagrees knowing that your own position is very different, should send you a message that you are very possibly - one might even say very probably - wrong. You can reach agreement very quickly once you acknowledge this fact. I hope you are both willing at least to say that the other's contrary stance tells you that there is a good likelihood that you are wrong.

Now, the procedure you are following, of laying out your arguments and premises and thinking processes, is likely to lead to a higher quality agreement than merely agreeing to agree. But there's no reason you can't agree first, and then mutually explore the issues second. Agreement does not have to be the end point of the discussion - I think that is something which Robin has emphasized many times.

The only explanation I can see for your having failed to agree is that you each suspect that the other is being irrational. It might be impolite, but if that is your reason, perhaps you could be open about those judgements and make progress from there. Or if there is some other reason why the other person's stubbornness is not as convincing as it should be, it might help to explain why. But I don't see much point in continuing to go over the reasons for your beliefs; those are irrelevant. The mere fact of contrary belief held by a rational individual should be enough.

Hal, I must consider the opinions of everyone on this subject, weighed by their expertize. I'd love to see more concrete data, but it seems to me that Eliezer is a bit of an outlier on this, and robustly so for a variety of ways to weigh expertize.

If I want to predict that the next growth curve will be an exponential and put bounds around its doubling time, I need a much finer fit to the data than if I only want to ask obvious questions like..."Do the optimization curves fall into the narrow range that would permit a smooth soft takeoff?"

Presenting that analysis would be an excellent place to start.

I hope you are both willing at least to say that the other's contrary stance tells you that there is a good likelihood that you are wrong.

If Robin knows that Eliezer believes there is a good likelihood that Eliezer's position is wrong, why would Robin then conclude that his own position is likely to be wrong? And vice versa?

The fact that Eliezer and Robin disagree indicates one of two things: either one possesses crucial information that the other does not, or at least one of the two have made a fatal error.

The disagreement stems from the fact that each believes the other to have made the fatal error, and that their own position is fundamentally sound.

Robin, I agree that everyone's opinion on this matter is certainly relevant information which you should incorporate. And arguably Eliezer's position is an outlier. But you are not actively interacting this week with the AI expert community (I assume). You are interacting with Eliezer. He is the one who knows your opinion and knows you have not changed it despite knowing his opinion, and so on ad infinitum. It is the two of you who are interacting and the two of you who must agree.

Everyone else's opinions are relevant data which will influence both of your initial estimates, to be sure. But that is the extent of that influence. The fact that everyone else holds a different position than one of you does not prevent you from coming to agreement. Assuming that Eliezer is aware of his differences with other experts, the fact that he still holds to his view ought to be even more persuasive to you that he has good grounds for his position. And if somehow you thought that Eliezer was not aware of his outlying position, your own stubbornness on these grounds would have communicated to Eliezer that you have persuasive private information that he is wrong. And again in that case, the fact that he has not changed his mind despite your persistence must again imply that his own reasons for his belief are robust and powerful.

I can understand why you might be puzzled as to how Eliezer can reach such a different view than the AI expert community, and with such confidence, but fundamentally that is irrelevant. The fact remains that regardless of how reliable you think Eliezer's conclusions are a priori, once you interact with him and he remains set on his position, as long as you see each other as rational (and meta-rational), you must come to agree.

Hal, I don't think my belief in simply the intelligence explosion is an outlier in the AI community. My detailed explanation is my own; my beliefs about theory-driven AGI are if anything more unusual than my belief in simply the intelligence explosion; my concern for Friendliness is not so much outlier as something that others haven't thought much about.

Eliezer, that's actually good news, as it sounds like whether you are an outlier or not is something else that you and Robin disagree on. Now it's not 100% clear which exact position it is that you are potentially an outlier on, but I gather that it would include whether the chance that initially hand-coded AI may reach a dangerous potentiality within this century is closer to 70% or 1%. The good part is that your outlier-hood is probably something that could be empirically checked pretty easily, by asking a bunch of AI researchers.

Now, from the point of view of exploring how to overcome disagreement this is a bit of a cheat, like looking up the answer in the back of the book. It shouldn't be necessary to be bothered by anything as mundane as the facts, in order to reach agreement on this issue or any other. But if it happens that facts are available, maybe you should use them. And since one of you would be surprised by the result, it might lead to progress on your larger and more important issue.

Hal, you do not know that I have not changed my opinion since learning of Eliezer's opinion, and I do not assume that he has not changed his opinion. A large fraction of all those other people with opinions are not changing them even though they are aware of others' opinions. I do give Eliezer's opinion more weight but even so he seems, according to my informal survey, to be an outlier on the need for friendliness.

We can practice disagreeing and understanding its rationality on most any topic. But since this topic is important, my focus here is on gaining a better understanding of it.

That's true, Robin, I did not consider that you guys might be updating your opinions as you go along.

I can understand your interest in pursuing the truth of the matter rather than just rushing to a declaration of agreement. Still I think there is a problem when you, a prominent advocate of the no-disagreement results, appear to have a persistent disagreement with a seemingly intelligent and rational person like Eliezer. It tends to discredit the disagreement theorems. If the truth is that your disagreement is more apparent than actual, and is in the process of being resolved, I think it would be beneficial to make note of that.

It is bad luck that this one issue where you and Eliezer have at least the appearance of persistent disagreement is also one which you both view as important, hence a matter where you prefer to devote your energy to airing the issues rather than working through the mechanics of agreement. It means that the dialog ends up being not much different from that of two people who are unfamiliar with the disagreement results. I was hoping, given the history of this forum, that you two might choose to model how agreement can be reached quickly, while maintaining vigorous discussion of the issues.

The only explanation I can see for your having failed to agree is that you each suspect that the other is being irrational.

IMO, a more obvious explanation is that one or both parties does not have believing the truth (and helping others to believe it) as their primary aim.

Robin seems to have said his aim is to believe the truth. The last I heard, Eleizer's aim was to reach something called "the singularity" as fast as possible.

So, Robin claims to be a pure truth seeker - though AFAIK, he hasn't explicitly stated that he in interested in helping others sort out their views. Eleizer doesn't claim to be a pure truth seeker in the first place.

I suspect that Robin does not actually value the truth that much. Four billion years of his ancestors valuing other things probably bears on him substantially. For example, Robin has stated: "I think about sex an awful lot." I don't see what that has got to do with believing the truth. I suspect that believing the truth is something Robin consciously aspires to do - and isn't what an unbiased economic analysis of him would uncover as his actual aim.

So: the premises of the disagreement theorems seem unlikely to be satisfied.

Tim, I've asked Eliezer about this now and then in various comments, and he has always maintained that he is being sincere and honest in his writings on the singularity. Recently he had a series of postings on the importance of honesty, such as Protected from Myself. The mere fact that he has put so much time and energy into working on this issue over many years is strong evidence that he sincerely believes that it is a real possibility, for good or evil. So I don't see any particular reason to discount his recent estimate of at least a 70% chance that AI could become super-intelligent within 100 years. Likewise with Robin's initial estimate of less than 1% for this event; I don't see any reason why he wouldn't be reporting that honestly.

For the disagreement results to hold, the participants don't have to be perfect truth-tellers, they just need to honestly give their opinions on the issue in question. Now Robin hints above that he may be revising his estimate, and says he considers it possible that Eliezer's position is also shifting. Maybe 1% vs 70% is no longer the state of play. But if they do choose to offer their revised estimates at some time in the future, and if they were to continue to do so, the disagreement theorems would pretty well force them to agree within a few rounds, I think. If that didn't happen then yes, maybe they would be lying; more likely IMO is that they suspect that the other is lying; and most likely I still think is that they suspect the other is simply being unreasonable.

The mere fact that he has put so much time and energy into working on this issue over many years is strong evidence that he sincerely believes that it is a real possibility

Only if there are no other consequences of his actions that he desires. People working to forward an ideology don't necessary believe the ideology they're selling - they only need to value some of the consequences of spreading it.

Eliezer Yudkowsky gave > 70% to "a key event occurring sometime in the next century".

Robin Hanson gave < 1% to "a hand-coded version will suddenly and without warning change from nearly powerless to overwhelmingly powerful" and "a friendliness-needing event by 2108".

So far, ISTM, that the participants haven't publicly established that they are talking about the same thing. Also, what they do seem to be talking about is rather vague. How key is "key"? What does "friendliness-needing" actually mean? How suddenly is "suddenly"? - and so on.

Robin seems to visualise uploaded humans running over the planet and taking everyone's jobs in the none-too-distant. Does he think that is "friendly"? Does Eliezer agree about that?

As far as disagreement goes, what is the evidence that the participants are seeking the truth on this topic - or any other? Are we supposed to ignore the council of sociobiologists that tell us that such debates frequently revolve around social factors such as status, reputation, money, signalling - and so on?