Conjuring An Evolution To Serve You

There are lots of examples of unexpected selective outcomes.

A story -- a long time agon a swedish researcher tried to increase wheat yields by picking the biggest wheat kernels to plant. In only 5 generations he had a strain of wheat that produced 6 giant wheat kernels per stalk.

When scale insects were damaging citrus fruits, farmers tried to poison them with cyanide. They'd put a giant tent over the whole tree and pump in the cyanide and kill the scale insects. Plants can be immune to cyanide but no animal that depends on respiration can be. And yet in only 5 years or so they got resistant scale insects. The resistant insects would -- when anything startling happen -- sit very still and hold their breath for half an hour or so.

If you want to do directed evolution, you do better to do it in controlled conditions. Take your results and test them carefully and make sure they're what you want before you release them. Microbiologists who want mutants for research commonly take 20 or 100 mutants who survive the conditions they're selected to survive, and test until they get a few that appear to be just what they want. Eliminate the rest.

So, for example, to find a mutant that has a high mutation rate -- start with a strain of bacteria that has at least 4 selectable traits. Say, they don't survive without threonine, don't survive without isoleucine/valine, don't survive penicillin, and don't survive rifampicin. So you grow up a hundred billion or so of them and then you centrifuge them down and resuspend them in medium that doesn't have threonine. Most of them die. Wait for the survivors to grow, and then centrifuge them down and resuspend them in medium that doesn't have isoleucine/valine. Most of them die. Wait for the survivors to grow, and centrifuge them down and resuspend them in medium that has penicillin. Do it a fourth time with rifampicin. Plate them out on media that has lactose (when the originals couldn't use lactose). Some of the colonies will be large and some small, pick a colony that has lots of little warts of bigger growth, because it gets lactose-using mutants even while the colony is growing. A strain that has a hundred times the mutation rate can be easily selected this way. It started out at frequency around 10^-8. After the first selection cycle it was frequency around 10^-6. By the fourth round it was common. Sometimes you can get a mutation rate around 1000 times the normal rate. Much above that and it doesn't survive well.

Take one colony per try because you don't want to test multiple colonies and then find out they're the same mutation over again.

The examples seem to demonstrate the weaknesses of selective breeding rather than evolution. Human intent and imperfect knowledge appear to be poor substitutes for the blind, mindless processes of nature.

Hmmmm...

Eliezer_Yudkowsky: I'm not sure I see the relevance of evolutionary theory to Enron. According to the characterization you quoted, the problem was that the stakes were so high that people cheated. Why do evolution's insights help me see that? That mishap can be explained through poor incentive alignment: what was optimal behavior for a trader was not regarded by Enron as optimal behavior. The disutility to Enron of "false profits" was not reflected in an individual trader's utility curve.

So Skilling picked a bad incentive structure. Does everyone who picks a bad incentive structure fancy himself an evolution conjurer?

Skilling was selecting badly. The 10% he discarded each year might have included some he should have kept, and vice versa.

Similarly, God at one point said he was going to get rid of evil people and keep good people and so people would get better. I don't see much evidence that's worked well.

Evolution happens, but if you want to harness it for your own goals you have to be very careful. Try to arrange it so you can throw away your mistakes.

3.85 billion years of natural selection produced molecular nanotechnology (cells) and Artificial General Intelligence (brains), which even we humans aren't going to get for a few more decades.

Does that mean that the singularity is at least a few decades away?

Has anyone tried breeding the smartest nonhuman primates (chimps, bonobos?) for intelligence? If not, what could one expect to achieve by doing this for 10 generations? To what extent are the genes for intelligence additive? That is, if there are multiple distinct genes that increase intelligence via distinct mechanisms, does having all these genes give you the sum of the intelligence boosts of having these genes individually?

"You really don't want to conjure Azathoth at full power. You really, really don't. You'll get more than pretty butterflies."

How confident are you that there aren't any mad scientists reading OB, looking for the perfect tool to do something randomly destructive?

Tom_McCabe: If you're worried about mad-scientist OB readers obtaining the tools for random destruction from this site, you're too late. Robin_Hanson already gave them the perfect idea.

Picture

On the Enron point:

An article in today's NY Times claims that a major danger to investment banks is the empowerment and growth of whichever division happens to be benefiting from transitory financial cycles. Since members of a particular department have specialized skills that are less valuable in other areas, they tend to be biased in favor of excessive investment of resources in their areas. If the mortgage market booms for too long you will wind up with a high frequency of mortgage people in the executive corps and reduced ability to cut loose if risks appear dangerously high for the firm.

Supposedly, part of Goldman Sachs' chart-topping success during the recent credit crunch (although it has been very successful for more or less its entire history) comes from the creation of a powerful independent institution with veto powers and less bias towards particular investment classes.

http://www.nytimes.com/2007/11/19/business/19goldman.html?pagewanted=2&ei=5087&em&en=a3db4f1df6a297ef&ex=1195621200 "At Goldman, the controller's office--the group responsible for valuing the firm's huge positions--has 1,100 people including 20 PhDs. If there is a dispute, the controller is always deemed right unless the trading desk can make a convincing case for an alternate valuation. The bank says risk managers swap jobs with traders and bankers over a career and can be paid the same multimillion-dollar salaries as investment bankers."

Let me answer a slightly different question: how confident are you that the benefits of publicizing the destructive potential of genetic algoritms outweighs the risks?

I am pretty confident that people setting out intentionally to do destruction on the scale addressed here are rare compared to people who do large-scale destruction as an unintentional side effect of trying to do good or at least ethically neutral things. Most evil is done by people who believe themselves to be good and who believe their net-evil deeds are net-good or net-neutral.

People of course differ in their definition of the good, but almost everyone capable of affecting them agree that certain outcomes (e.g. toasting the planet) are evil.

Put more simply, in artificial evolution you get exactly what the fitness function you've written asks for, even when you don't know what it's actually asking for.

Put more simply, in artificial evolution you get exactly what the fitness function you've written asks for

You don't even necessarily get that. The animal breeders thought they were asking for more eggs. They did get some eggs, but with side effects, and not nearly as many eggs as they could have gotten, if they'd used a different breeding format with the same fitness function: fitness=eggs, vehicle=group instead of fitness=eggs, vehicle=individual.

They could well have gotten fewer eggs by breeding for eggs, as Enron did, if the chickens had discovered enough negative-sum tricks, as Enron did.

Fantastic post!! This certainly applies to the HGP and GMO's. An excellent page about this: http://www.psrast.org/strohmnewgen.htm Eliezer. there are many things I'm sure we disagree about. But, we must not allow ourselves to become enslaved beyond all desire of escape. Is it OK if I love you for that sentiment?

"Let me answer a slightly different question: how confident are you that the benefits of publicizing the destructive potential of genetic algoritms outweighs the risks?"

I am quite confident of that. I wanted to know how seriously everyone else had considered the risk.

Genetic algorithms have potential, period. It's human beings that will because that potential to be used unwisely. Publicizing the power of the algorithm might -might- cause enough people to be wary of what they can do for balancing principles to come into play. Trying to limit the knowledge will eventually be more harmful than not.

Tom, I don't take the risk seriously. Richard Hollerith said well why. I don't think people who just want to do something randomly destructive are good at the kind of long-range planning and collaboration needed to be seriously threatening, and if they were, they wouldn't need us to give them extremely general ideas.

I was reminded of something Michael Vassar said on SL4 (emphasis mine):

For all of our arrogance, most Transhumanists grossly overestimate the abilities of ordinary humans. This is substantially a consequence of how folk psychology works, and fails to work for outliers, but also a consequence of typically limited and iscolated life experience. Unfortunately, it has serious consequences when predicting the future. Our estimates of the likely behavior of large scale groups, the effort that will be devoted to a particular research objective, or the time until some task is accomplished are all grossly distorted. For many transhumanists this means that boogie men such as "terrorists" are imagined as something that never was, disutility maximizers, and the resultant threats of bioterror and nanoterror are overestimated by many orders of magnitude. For almost all transhumanists this means an underestimation of inertia, leading Chris Phoenix's fears of pre-emptory arms races and Nick Bostrom's utopian dreams of world government and benign regulation of dangerous tech.

Most of the truly frightening possibilities are simply too unlikely. To produce them from a genetic algorithm, you'd have to expend massive amounts of resources creating a specific environment that would select for the traits you desired - no mean feat. Imagine what it would take to develop a hypervirulent and extremely lethal plague through an algorithm on purpose. The requirements are crushing.

Making people more aware of such algorithms, and their potential, might force a shift in the way food animals are dealt with so that they don't act as a optimalization procedure for plagues.

As an evolutionary biologist with an interest in practical applications to agriculture and to human longevity, I think your emphasis on the slow pace of evolution is misplaced. It took most of life's 3.85 billion year history to evolve multicellularity, but that slowness seems to mainly reflect lack of selection for multicellularity over most of that period. With strong selection, primitive multicellularity can evolve quickly under lab conditions ( Boraas,M.E. 1998 "Phagotrophy by a flagellate selects for colonial prey: A possible origin of multicellularity" and current work in my lab).

Your point about individual vs. group selection is correct and important, though. Individual selection, like free-market competition, is an effective way of making certain kinds of improvements. But some form of group selection (the chicken example, or small-plot trials in plant breeding) is often key to improvements missed by individual-based natural selection. See my 2003 review article and forthcoming book on Darwinian Agriculture.

"You really don't want to conjure Azathoth at full power. You really, really don't. You'll get more than pretty butterflies."

One more example proving the Mythos rule that one should learn "Bind Azathoth" before casting "Summon Azathoth"

:)

Invocation of hypothetical (often ideologically linked) expectations of the future as if they were deterministic processes is rampant.

(This post could be read as a predecessor to the Immoral Mazes sequence [? · GW].)

It seems Enron did what corporations normally do, just faster. If I remember correctly the percentage of psychopaths between corporate managers is around six times the normal ratio.

Extremely cool evolution experiment where E. coli bacteria evolve to eat citrate along with many other interesting happenings.