Say Not "Complexity"

Quote: "We think in words, "

No we don't. Apparently you do, though. No reason to believe otherwise. :)

Please keep up these postings! They are very enjoyable.

Going back to "explaining" something by naming it (from a couple of your earlier posts):

e.g. Q: Why does this block fall to the floor when I let go of it? ... A: Gravity!

I always thought that such explanations were common side-effects of thinking in words. Sort of like optical illusions are side-effects of how the visual system works. Perhaps not. One does not need to use words to think symbolically. There are, after all, other ways to do lossy compression than with symbols.

Anyway, I'll still assert that it's easier to fall for such an "explanation" if you think in words. ... An easy assertion, given how hard it is to count the times one does it!

Aren't we understating the role of labels in brevity here?

Where the labelled thing is understood well enough by the labeller and listener or of trivial importance to the problem domain, don't labels contribute to cognitive economy?

I'd have said when you need to get things done, fear of wasting time is desirable rather than deadly.

Actually, the "emergence" and "complexity" pseudo-causal explanation are much worse than Felix's "gravity" example: the answer "Gravity!" does explain the fact that the block falls to the floor by noting that it is a specific instance of a general phenomenon for which we have very precise information on how it works (attraction force is constant x m1 x m2 /d^2). We may not know why gravity exists, but that is a different (higher level?) problem.

In the case of "emergence" and "complexity", we just don't know.

P.S. I do think that "emergence" is a useful concept to describe situations where modelling is more conveniently done at a (more) aggregate level, but that's yet another story.

I don't think these parable posts convey information efficiently to the overcomingbias audience, but I like your point at the end. Specifically, I agree it's better to use placeholders that make lack of knowledge/understanding clear, rather than placeholders that seem to cover up such lack of knowledge/understanding.

"Then a miracle occurs..."

;p

I wonder if memetics would serve as a good candidate for the category of things that satisfy without explaining or predicting anything, along with phlogiston, emergence, and complexity. The analogy to biology seems interesting and fun, but is it more useful than as just a way to re-formulate our perspective?

In computer science there is a saying 'You don't understand something until you can program it.' This may be because programming is not forgiving to the kind of errors Eliezer is talking about. Interestingly, programmers often use the term 'magic' (or 'automagically') in precisely the same way Eliezer and his colleague did.

Step 1: Steal Underpants Step 2: ????? Step 3: Profits!!!!

Programming is not forgiving to the kind of errors Eliezer is talking about.

But it's a lot better to be unforgiving of yourself than to wait for reality to hit you over the head with it. It's better to notice in 10 seconds that you don't understand something, than to realize this only after 20 people spend 5 years and $10 million of venture capital and the "emergent behavior" you pinned your hope on fails to materialize. It's all too easy to program "chaos", "complexity", or "emergence", so long as you tell yourself that you need to program more of it before you reach Step 3 and Profit.

"That was when I thought to myself, "Maybe this one is teachable.""

How many people have asked you about becoming an AGI designer? It sounds like you have a good deal of experience with rejection, even after weeding out the obvious crackpots.

Well, this is partly a matter of what discipline one is dealing with. So, sure, for AI or computer science more generally, Kolmogorov or Chaitin or Rissanen measures are more useful and reasonably well defined. For other disciplines, other definitions may be more suitable. Thus for economics, I have (following Richard Day) defined complexity in a dynamic way based on erratic dynamics appearing endogenously out of the system (with "erratic" defined more specifically). I laid this out in a paper in 1999 in the Journal of Economic Perspectives, and have a more recent paper up on my website ("Computational and Dynamic Perspectives on Economic Complexity") comparing the two approaches, at http://cob.jmu.edu/rosserjb.

As a current student, I can confirm your suspicions about a seemingly complete paper being preferred over one that addresses all information about the topic. "I don't know" still is not an acceptable answer in many circles and I regard it as an unfortunate phenomenon.

Gray Area wrote, "You don't understand something until you can program it." As somewhat of an aside, Randy MacDonnell has written about APL and J (http://facilitatedsystems.com/weblog/2007/07/if-you-can-say-it-its-done.html), "If you can say it, it's done."

On a different note, I took a pair of summer high school mathematics courses sponsored by the NSF years ago at the University of Miami. One professor, a Dr. Hermann, I seem to recall, said he often imagined himself wearing "worry beads" and fingering them as he spoke. If he fingered the beads nearer his neck, he was speaking more precisely; if he fingered those nearer his waist, he was speaking less precisely. In reality, he wore no beads, but he did, on occasion, finger imaginary beads as he was explaining certain concepts.

Perhaps the same thing can be adapted here to indicate the level of magic in claims we make. If we finger imaginary beads near our necks, we claim we know what's going on; if we finger those nearer our waists, we admit there's magic here.

Forgive me for latching onto the example, but how would an AI discover how to solve a Rubik's cube? Does anyone have a good answer?

"We think in words, "

Correction: We think by magic!

Update: Rubik's cube solved http://news.cnet.com/8301-17852_3-20013666-71.html

I think I just thought of an insanely over-simplified analogy.

Say I'm not invited to my best friend's sleepover and I don't understand why. I call her, and the answer she gives me is: "It's complicated."

The situation might indeed be complicated, but the word complicated is just a fake explanation... :D Amiright, guys?

I've been working my way through the sequences in order for the last few weeks and trying to read all of the links. I love this blog and tell people about whenever I can.

Reading these entries has helped me realize some of the ways in which I tend to think incorrectly, and I hope I am taking it slow enough to reflect enough and make myself think better. :)

I suppose I should comment about at least one thing relevant to this article in particular. Posted at 4:22 am?! When do you sleep, Eliezer?

I think Many of the commanders have misread her question; it's not, how would you make a program to solve a rubiks cube (which is brute forceable as there are a finite number of states for a rubiks cube), but how do you program an ai so it can work out how to do so. The ai has to study the cube and determine the possible states of the cube (and how to write them down mathematically) and the operators available to change the state of the cube. This means it has to know what a state and an operator are (if not by name). It then has to work out how to combine the operators to change the state to a predetermined state, which we happen to call a solved cube. The ai has to be able to do this with you never having coded into the ai anything specific to do with a rubiks cube, or any methods to solve the rubiks cube. The next step would be to code an ai where it ha to work out for itself that it must combine operators to change the state of the cube. I don't think I could manage to code the simpler version; for it to have been coded well, it must be able to solve any problem, not just the rubiks cube.

we would say "magic" - as in, "X magically does Y"

That's a nice bit of semantic hygiene. I hope to remember it.

The solution (I posted it elsewhere also):

To solve Rubik's cube, you can just do hill climbing, with breadth-first-ish search for the higher hill point (i.e. you find higher point even if it is several moves away). This discovers the sequences. Cache the sequences.

It's a very general problem solving method, hill climbing with N move look-ahead. One does try maximizing various metrics, that are maximal in the final state, and finds one that works for you without getting you stuck in local maximum for too long. You also try various orders of iterating the moves (e.g. one could opt for repetitive sequences).

This works for chess as well, and for pretty much all puzzles. This is how I solve puzzles when I get a puzzle for first time, except of course I have terabytes worth of tricks that I can try, and 10^15 - ish operations per second; parallel, of course, but parallel works. Pre-generating sequences is not necessary. You arrive at them when hill climbing with breadth-first search, and cache them. You also tell them to other people whom you want to make into rubik-cube-solvers. The important thing that can't be stressed enough - try to figure out a good metric to climb. Some sides of hill are smoother than others.

One could hill climb some sort of complexity metric - evolution did that to arrive at humans, even though the bacteria is a better solution to 'reproduction'. You only need a comparator for climbing. Comparators are easy. You can make agents fight (or you can make agents cooperate). You don't need mapping to real number. You can do evolutionary hill climbing with n-move look ahead. edit: note that you do NOT need good ordering for hill climbing either. If sometimes a>b and b>c and c>a it is okay if you remember where you already been and avoid looping. That may still get you to the top of the hill.

'Mercury's gravitational pull has long since been destroyed by solar flares, which is why is has no atmosphere'. Something I read today- seems appropriate. Apparently they'd been watching a documentary, and I think they put the components of the explanation together incorrectly in their head.

I think when your friend was talking about "complexity" he didn't mean the word literally. He may have meant that you would have to create a complicated solution, as opposed to finding a nice and elegant solution. The difference is you try to hammer out every detail and special case, one at a time, and adding "complexity" as you go, as opposed to just thinking about a single solution which would handle every case.

This is what I think most people mean when they talk about "complexity" as a solution to their problem. They don't literally mean that adding more complexity will solve the problem. It is just a different approach to problem solving. And sometimes that approach is easier and gets things done, even if it is more messy. Sometimes it is not.

Different approaches to solving problems is an interesting subject in itself. I've seen it create huge divisions in both artificial intelligence and politics. I tend to prefer nice elegant solutions. But when the problem seems complicated, it's tempting to run to the complexity side of things. There is no guarantee you will ever find an elegant solution, but if you just handle special case after special case, you can make progress over time for sure.

I suspect that counts as a useful thinking-tool. Whenever I notice incomplete steps in my reasoning, I'll say "by magic!" and then I can worry less about fooling myself.

A thinking-tool needs a name in order to properly install it into memory. "Magic-markers" could work.

"Technical explanation of technical explanation" link is broken.

Here's a working one: http://www.yudkowsky.net/rational/technical/

This one goes down as one of the truly great essays on the sequences for me. Recognizing the gaps in my map is what has lead me to understand many things even though when I was not consciously noticing those kind of gaps. Now I'll do it consciously and I'm happy about that.

What's more, the sequences seem to be repetitive at surface-level, but they are not; they hammer-in the concepts. It was this specific essay that truly conveys to me the importance of not doing the "skip-overs", this was the one essay which leads me to think that I also might be teachable.

In my work I additionally find it useful to break "magic" itself down into categories:

magic:  It's part of the tool's intended problem domain, but you can't use it this way without a thorough understanding of precisely how it functions.

black magic:  This is not part of the tool's intended problem domain, but your thorough understanding of both the problem and the tool's internals lets you use it for this.  (Playing music on a floppy drive for example)

voodoo:  It's not what the tool was made for, you don't know why or how it works, you have no idea what range of inputs will produce acceptable outputs.  You just know that having the clock open in the top right corner of your screen keeps your word processor from crashing...

The fact that there is no "real" magic reminds people that there is a rational explanation, and the categories convey information about how deep the pond is likely to be to anyone considering diving for answers.