UFAI cannot be the Great Filter

What if post-singularity civilizations expand at the speed of light? Then we should not expect to see anything:

It looks like we are going to have less than 200 years between interstellar detectability and singularity. So the chance of us being around at the same time (adjusted for distance) as another civilization to a resolution of a few hundred years seems quite low.

Life will only get to the point of asking questions like these on worlds that haven't been ground up for resources, so we can only be outside the "expansion cone" of any post-singularity civilization. If the expansion cone and the light cone are close (within a few hundred years), then, given that we are outside of the expansion cone, we are probably outside the light cone as well. So the AI-as filter doesn't get falsified by observing no AIs.

It doesn't even have to be a filter, though it probably is; 100% of civilizaions could successfully navigate the intelligence explosion and we would see nothing, because we can only exist in the last corner of the universe that hasn't been ground up by them.

This is all assuming lightspeed expansion. Here's a few ideas: a single nanoseed catapulted at 99.9 % the speed of light, followed by a laser encoding the instructions. Cross galaxy light-lag would be only 100 years. How to slow it down on arrival is unknown... Another possibility is actually creating spaceships out of light; some kind of super laser that would excite whatever it hit in just the right way to create a nanoseed. This one seems mush less plausible, but I wouldn't bet against the engineering skill of a superintelligence.

Hmm.

Okay, filters that would produce results consistent with observation.

1: Politics: Aka: "The Berzerker's Garden" The first enduring civilization in our galaxy rose many millions of years before the second, and happened to be both highly preservationist and highly expansionist. They have outposts buried in the deep crust of every planet in the galaxy, including earth. Whenever a civilization arises that is both inclined and able to turn the galaxy entire into fast food joints/smily faces/ect, arises said civilization very suddenly disappears. The berzerkers cannot be fought, and cannot be fooled, because they have been watching the entirety of history, and their contingency plans for us predate the discovery of fire. If we are really lucky, they will issue a warning before annihilating us.

2: Physics is booby trapped: One of the experiments every technological civilization inevitably conducts while exploring the laws of the universe has an unforeseeable, and planet-wrecking result. We are screwed.

3: Economics: The minimal mass of a technological "ecology" capable of sustaining itself outside of a compatible biosphere is just too large to fit into a star ship. The interlocking chains of expertise, material extraction and recycling, energy production and so on, and so forth, flat out cannot be compacted down enough to be moved. No such thing as a von-neuman probe or a colony ship can be built. Civilizations expand to the limit of how far spare parts and help can can be sent, and then halt.

4: Diversion: Advancing tech opens "frontiers" much, much more attractive than star flight before starflight becomes possible. Alternate timeline gates, uploads into the underlying computational substrate of the universe, ect, ect.

5: Anthropic engineering. Advanced civilizations have proof of the manyworlds and anthropic principles - And use them. Which from outside any given circle of coordination looks like collective suicide. So the universe is full of empty planets, and every civilization has all of it to themselves.

AFAIK the idea that "UFAI exacerbates, and certainly does not explain, the question of the Great Filter" is standard belief among SIAI rationalists (and always has been, i.e., none of us ever thought otherwise that I can recall).

And this disaster can’t be an unfriendly super-AI, because that should be visible

This is not necessarily true. If the goals of the AI do not involve a rapid acquisition of resources even outside its solar system, then we would not see evidence for it (E.g, wireheading that does not involve creating as many sentient organisms as possible).

However, because there would be many instances of this, AI being the filter is probably still not likely. If it's very likely for UAI to be screwed up in a self-contained way, we would not expect to see evidence of life. If UAI has a non-negligible chance to gobble up everything it sees for energy, then we would expect to se it.

For galactic civilisations I'd guess that there would be a strong first mover advantage. If one civilisation (perhaps controlled be an AI) started expanding 1000 years before another then any conflict between them would likely be won by the civilisation that started capturing resources first.

But what if none of them know which of them expanded first? There might be several forces colonising the galaxy, and all keeping extremely quiet so that they don't get noticed and destroyed by and older civilisation. Thus no need for a great filter, and even if UFAI were common we wouldn't observe it colonising the galaxy.

The relevant notion of intelligence for a singularity is optimization power, and it's not obvious that we aren't already witnessing the expansion of such an intelligence. You may have already had these thoughts, but you didn't mention them, and I think they're important to evaluating the strength of evidence we have against UFAI explosions:

What do agents with extreme optimization power look like? One way for them to look is a rapidly-expanding-space-and-resource-consuming process which at some point during our existence engulfs our region of space destroys us, which we haven't seen happen yet. Another way for them to look is such an optimization process that has already engulfed our region of space. And if that were the case, we would necessarily be a byproduct of or noise term in that process, and the laws of physics actually meet that description.

(I had this thought when I first asked myself to look for naturally-occurring examples of extremely powerful optimization processes, and the laws of physics were the best examples I could think of. E.g., an agent wishing to minimize the L^2 norm of the differential of energy with respect to time would produce an environment that conserves energy.)

It's difficult and conjunction-fallacious to come with an agent whose utility function would exactly be to have an environment that follows our particular laws of physics with our particular initial conditions, so one should not update heavily in favor of this idea based on observing the laws of physics to be true. What I'm saying is that we only have seen evidence against UFAIs that could have expanded from regions of nearby space and interrupted human progress, not UFAIs that could have expanded before and perhaps caused the existence of the observable universe.

I take issue with the assumption that the only two options are perpetual expansion of systems derived from an origin of life out into the universe, and destruction via some filter. The universe just might not be clement to expansion of life or life's products beyond small islands of habitability, like our world.

You cannot assume that the last few hundred years of our history is typical, or that you can expect similar exponentiation into the future. I would argue that it is a fantastic anomaly and regression to the mean is far more likely.

Good post, good explanation. I agree. I saw the recent comment on OB that probably sparked you making this topic, I was thinking of posting it fleetingly before akrasia kicked in. So, thanks.

A throwaway parenthesized remark from RH that nevertheless should be of major importance, because it lowers the credence we should assign to the argument that "UFAI is a good great filter candidate, and a great filter is a good explanation for the Fermi paradox, ergo we should raise our belief in the the verisimilitude of UFAI occurring."

Early or late great filter?

I'm currently leaning strongly towards late filter, because many of the proposed early filters seem to not be such big barriers. We've for example found a bunch of exoplanets in the last decade or so and several of those seem plausibly in the habitable zone. Life on Earth arose very early in its history so if life arising is the hard and rare step I would expect there to be many more hundreds of millions or even billions of years of conditions on Earth being seemingly ripe for it arising and it not doing so.

" Abiogenesis likely occurred between 3.9 and 3.5 billion years ago, in the Eoarchean era (the time after the Hadean era in which the Earth was essentially molten)."

Obviously maybe the conditions ripe for life being there at all is the tricky part. Another possible hard step is multicellular life.

Correction: Multicelular life seems easy eukaryotic life not so much.

The Cambrian explosion seems to have happened like a billion or more years after it "could" have happened.

Really, it seems like any kind of superintelligent AI, friendly or unfriendly, would result in expanding intelligence throughout the universe. So perhaps a good statement would be: "If you believe the Great Filter is ahead of us, that implies that most civilizations get wiped out before achieving any kind of superintelligent AI, meaning that either superintelligent AI is very hard, or wiping out generally comes relatively early." (It seems possible that we already got lucky with the Cold War... http://www.guardian.co.uk/commentisfree/2012/oct/27/vasili-arkhipov-stopped-nuclear-war)

Katja says:

The large minimum total filter strength contained in the Great Filter is evidence for larger filters in the past and in the future.

That's true - but anthropic evidence seems kind-of trumped by the direct observational evidence that we have already invented advanced technology and space travel, which took many billions of years. From here, expansion shouldn't be too difficult - unless, of course, we meet more-advanced aliens.

Other civilizations may possibly be expanding too by now - SETI is still too small and young to say much about that directly. Probably not within our galaxy, but only because us and them becoming civilised at the same time would be quite a coincidence.

Interesting. However, I'd like to propose an alternative: The real probability of another alien civilisation being inside our universe shard, that is the area of the universe that us humans can possibly explore below the speed of light, is very low. So there might be predatory super intelligence that has wiped out the civilisation that made it, but we're just not in its universe shard.

When you (and Robin) say "because [UFAI] should be visible," that seems to imply that there are a significant number of potential observer moments that occur where we can see evidence for a UFAI but the UFAI is not yet able to break us down into spare parts. I've always assumed that if a UFAI was created in our lightcone, we would be extinct in very short amount of time. Thus, the assertion "UFAI is not the great filter because we don't see any" is similar to saying "giant asteroids aren't the great filter because we don't see any smashing into earth and extinctifying us." That is, of course we don't observe these things because if they occurred, we wouldn't be around to observe them. Is the assumption that once a UFAI has advanced enough to be observable by us, it would be traveling to devour the rest of its observable universe at near-light-speed obviously silly for some reason I'm missing?

Alien UFAI could be dangerous for us if we find its radiosignal as a result of SETI seach. His messages could content a bite which could lure us to built a copy of alien AI based on schemas which he will send to us in this messages.

D. Carrigan wrote about it: http://home.fnal.gov/~carrigan/SETI/SETI_Hacker.htm

Some simple natural selection reason imply that UFAI radiosignals should dominate in all SETI signals if any exist. And the goal of such UFAI is to convert the Earth to another radio backon which will send his own code futher.

My article on the topic: Is SETI dangerous? http://ru.scribd.com/doc/7428586/Risks-of-SETI-Is-SETI-Dangerous

I have a hard time imagining a filter that could've wiped out all of a large number of civilizations that got to our current point or further. That's not to say that future x-risks aren't an issue--it just feels implausible that no civilization would've been able to successfully coordinate with regard to them or avoid developing them. (E.g. bonobos seem substantially more altruistic than humans and are one of the most intelligent non-human species.)

Also, I thought of an interesting response to the Great Filter, assuming that we're pretty sure it actually is ahead of us: Halt all technological development ASAP and stay on this planet chilling out. It's possible that other civilizations already have done this (having realized that the Great Filter was an issue and there was likely deadly tech in their future)--if they had, we wouldn't know about it.

Of course, that might just mean that 99.9% of all civilizations destroy themselves in the roughly 100 years between the invention of the nuclear bomb and the invention of AGI.

This should be "UFAI can't be the only great filter." Nothing says that once you get past a great filter, you are home free. Maybe we already passed a filter on life originating in the first place, or a technology-using species evolving, but UFAI is another filter that still has an overwhelming probability of killing us if nothing else does first.

The fact that UFAI can't be the only great filter certainly screens off the presence of a great filter as evidence of UFAI being a great filter, but there are good arguments directly from how UFAI would work that indicate that it is pretty big danger.