Posts

Comments

Comment by Brian Holtz (brian-holtz) on Phase transitions and AGI · 2022-04-11T21:01:20.945Z · LW · GW

Indeed! And what about the 7th phase shift? I must be missing something here. We're pointing to just two particular transitions in all of human history. I fear we're calling them "phase shifts" to convince ourselves that there is a simple underlying phenomenon which has law-ordained "phases". How can we assume that there will be a coherent series of N>2 such "phase shifts"? How can we assume that they will follow some simple mathematical function with just a handful of parameters? 

I guess this makes me even more of a singularity skeptic than Hanson. He marshals powerful economic arguments against naive singularitarianism. But then he forecasts a similarly epochal transition to whole-brain emulations, yet seems worried that it needs more support than its own inside analysis provides. So goes back to an outside extrapolation from past data points a la Kurzweil, but he extrapolates from only 2 points, and says the resulting curve can't really tell us anything beyond the 3rd point. By contrast, Kurzweil marshals multiple technological trends each with dozens of past data points. If we call that naive, then how confidently can we extrapolate along a single trend with only 2 prior data points?

That doesn't feel like a curve to me. It just feels like a prediction of a 3rd transition that will have similar significance to two previous transitions. But being similar in significance is not really evidence of being similar in explainability by some simple underlying lawlike mechanism. The word choice of "phase" seems like sleight of hand here. If instead we talk of "epochs" or "periods", we're less likely to bias ourselves towards a phantom unifying phenomenology of what could just be three not-very-causally-related transitions.

My critique here seems obvious and unoriginal, and the people it's aimed at are very well-informed and thoughtful. So I apologize if this has been addressed elsewhere already. What am I missing?