What Curiosity Looks Like

They would study logic, probability theory, argument, scientific method, and other core tools of truth-seeking. They would inquire into epistemology, the study of knowing. They would study artificial intelligence to learn the algorithms, the math, the laws of how an ideal agent would acquire true beliefs. They would study modern psychology and neuroscience to learn how their brain acquires beliefs, and how those processes depart from ideal truth-seeking processes. And they would study how to minimize their thinking errors.

Not necessarily. Hindsight bias is likely at work here. You know that studying these fields helped you to acquire better beliefs, and so you conclude that this consequence should be obvious. But unless a curious but untrained reasoner somehow finds out that studying these fields will help them, we shouldn't expect them to study them. Why on earth would someone try to read The Logic of Science if they didn't already know that it would improve their reasoning skills?

There are a lot more genuinely curious people out there than there are rationalists. But unless those curious people happen to meet a LWer, or stumble across a link to this site, their chances of learning the benefits of studying these subjects are not great. There are a few books that might get them started (Robyn Dawes is coming to mind), but how likely is it that they're going to stumble across one of those books, especially if they aren't explicitly interested in the field of human thought already (like EY?).

I would bet that there are a lot of genuinely curious people out there who have realized that thinking is a skill. But if you were to ask them for the best way they knew of to improve that skill, would say something along the lines of "sudoku puzzles". And that's pretty sad.

I'd handle shame-flavored incentives with tongs. It's plausible that I have an unusual degree of sensitivity on the subject, but I'm making progress on a very bad case of self-hatred and akrasia, and "is my curiosity good enough?" strikes me as a sort of self-alienation which takes focus away from paying attention to whatever you might be curious about.

"What might I be missing about this?", "How can I increase my enthusiasm for learning?", "How can I spend less time on errors while still taking on difficult projects?" seem much safer. "What am I doing to improve my life? Is it having the desired effect?" should probably be on the list.

Also, they would seek to personally become an immortal super-intelligence, since many truths simply can't be learned by an unenhanced human, and certainly not within a human lifetime.

(Which is why the Yudkowsky-Armstrong Fun-Theoretic Utopia leaves me cold. Would any curious person not choose to become superintelligent and "have direct philosophical conversations with the Machines" if the only alternative is to essentially play the post-Singularity equivalent of the World of Warcraft?)

I'm having difficulty knowing what level of rationalist this is aimed at. Are the people you talk to every week students of rationality, or 'normal' people?

This post applies to both, I imagine. But because you talk about "people" instead of explicitly talking about people like me, it's easy to see this post as not being aimed at me. (Maybe it's not).

What I mean is: It's easy to praise oneself and one's peers by talking about people of a lower class. When I was young, it was 'dumb people', when I was a bit more sophisticated it was 'theists', when I was an Objectivist, it was 'non-Objectivists', and now that I'm a rationalist the temptation is to criticize those who "know almost nothing of logic, probability theory, argument, scientific method, epistemology, artificial intelligence, human cognitive science, or debiasing techniques." So this post, because it isn't clearly directed at people who have worked hard to do better in the ways prescribed by the Sequences, causes my semiconscious mind to ask: "is this a beginning level post, or something I should actually pay attention to?" Are you telling me to do better, or criticizing outsiders in order to promote group bonding?

Of course my rider knows I should pay attention; I must always work harder to cultivate the virtues. And I don't actually expect that you're just trying to promote group bonding. But what I really want - and what this post may not have been designed for - is a honest criticism of people you see who are a lot like me in that they have access to all the same correct memes as I have, and they have exerted effort to improve in the appropriate areas.

.

You're doing that thing where you write like Yudkowsky again. It's kind of hot.

Dare I say it? Few people look like they really want true beliefs.

I think otherwise - most people want to have true beliefs. However, they have rather limited trust in the powers of their own logic, as the experience of school has taught them that they are often wrong. They don't have the numerical skills to embark on anything more numerically ambitious than what money requires. They expect to be wrong often, and rarely use formal reason as such. But they still want to have true beliefs, and rely mostly on intuition and experience to decide on that.

For most people, most beliefs are socially acquired - people acquire their beliefs from the people around them, and they tend to acquire large blocks of belief together. One shouldn't underestimate the sheer amount of work needed to do anything different.

Most people never create a new idea (in the sense you're talking about) in their entire lives - they have experiences, yes, and they change beliefs based on experience. But they do not regard themselves as having the basic equipment to generate ideas, or to be sophisticated in judging between them.

In the end I've come to the view that none of us can change this (well, not anytime soon at any rate). Human beings think in groups - and the best most of us can do to help others think better is to do it for them, and talk about it sometimes. Obviously there is a group of people who can do more than that, but they are a minority.

The other comment I have on your post is that all of the above is actually just one idea. Which is that the basis of all knowledge about the world is inductive reasoning, and, in principle, all such reasoning should be based on the sound use of statistics. There are many mathematical ways of messing up such use, and our intuitive reasoning also messes up these stats too. If you really need the right answer, you will need to learn enough to get your statistics right, and compensate for the shortcomings of your wetware. And that's the whole idea in a nutshell.

An example. Did you know that brakes are the most dangerous piece of equipment on your car? In a staggeringly large number of accidents, the driver was using the brake at the time of the collision. Surely we could make driving safer by removing the brakes, then? Of course the thesis is ludicrous, but how many of us are confident that we wouldn't make a similar statistical mistake in a different context? But once you embark on the journey of trying to fix such problems in your own thinking, the road leads all the way to the place your post describes.

Most people think the journey is beyond them, and leave it to people like you and Eliezer to make the journey for them, and report back your findings. And unfortunately I don't think they're wrong about that.

FWIW I linked to this though my Twitter feed and got a very negative reaction from my friends, though the reasons they give for disliking it are pretty varied; they said things like

• it's so arrogant!
• It's absolutist - not everyone will have time to study these things
• but what is "truth" really?
• what about my-favourite-discipline? (Physics in one case)

One person has promised to write up what they felt in a blog post, which I look forward to reading.

They would study artificial intelligence to learn the algorithms, the math, the laws of how an ideal agent would acquire true beliefs.

Really? The others make sense, but it's not clear this will be useful to a human trying to learn things themselves. If I want to notice patterns, "plug all of your information into a matrix and perform eigenvector decompositions" is probably not going to get me very far.

So are those who claim to seek truth, but really don't, merely akratic or are they hypocritical?

Do "curious" people want to learn the (already discovered) truth or to discover heretofore unknown truths? You seem to confound the two. Data on the statistical correlation between these distinct motives would be interesting, but I doubt most scientists are primarily concerned with personally accumulating true beliefs. Preparing to make contribution to human knowledge probably looks a lot different from preparing to absorb the greatest mass of truths. It probably also looks different from preparing to function rationally as far as go quotidian beliefs.

I think you are confusing between wanting to know, and being good at it.

Imagine someone in the stone age, would you say none was genuinely curious because they didn't know about all those fields which weren't invented yet ?

Then, what about someone living in our world, but not knowing about Bayesian reasoning, AI, ... ? How can he know that those fields are fundamental to learn, to satisfy their curiosity on another field, before at least learning the basis of them ? When you don't know about Bayes' theorem, but you are curious (you really want to know the truth) about, say, ancient Rome history or about if there ever was life on Mars, what would drive you to learn probability theory ? How can you know you must learn it to learn other thing, when you don't know much about that other thing ?

Sure, if you are curious, you'll want to learn in all fields. But since we have a limited amount of time, you can't except someone to learn Bayesian reasoning, even if he's really curious, unless there is some kind of trigger that makes him realize how useful it would be to be efficient in being curious.

Genuinely wanting something and being good at doing it are not directly linked. You can't say someone isn't really wanting something just because he pursues it in an efficient way.

I'm not sure there's an overarching "curiosity" that people have or don't have: I'm very curious about whether a specific kind of database will perform adequately in certain circumstances (long story) but I'm only mildly curious about how to identify which French painter during the 19th century painted which picture. Some art experts, I'm sure, have cultivated the skill to guess within seconds which painter it is for every picture. I wouldn't mind having that skill -- it sounds like a fun skill to have -- but it seems like it would be more resources than it's worth. OTOH, I really want my probability estimations re: the database to reflect reality. Do I need to use AI theory? Doubtful. Probably a little bit of statistics, and even that fairly mild, but I do have to think a lot about how to use my knowledge of databases to design experiments to find the truth out. I'm not sure if that would look "curious" to the lay person (and, of course, there's also a factor of "signaling curiosity" -- I want to make sure that everyone with a stake in the process sees that I've done the due diligence), but nonetheless, I'm truly curious about this (and yes, it could go both ways...I think this is the most important part of curiosity vs. fake curiosity).

When I was genuinely curious about the US immigration laws applied to me (and again, it could have gone both ways -- before running any experiments, I made sure to visualize both options, and realizing I can live with both) I just called an immigration lawyer (and, for a latter question, the paralegal who was involved with my visa). In that case I needed very little knowledge from LW -- I didn't apply my knowledge about Bayes, or about heuristics and biases, and just went and asked a professional (of course, in some cases, like wanting to know if a stock will go up, asking a professional is disasterous, but with immigration law, lawyers can estimate probabilities fairly accurately even if they lack formal rationality training).

Those were the two examples of real curiosity from my life that I could think of, that looked nothing like the description here of "real curiosity"....

Who can quarrel with this, except perhaps on emphasis? All that can really be asked is whether the list of subjects prime for pursuit by rationalists is complete. The one major omission: acquiring writing skill. This is vital not only for articulating your ideas, so you get worthwhile feedback. The quality of ideas themselves depends on how well they're expressed. That Darwin was a superb writer isn't an incidental fact. (See "Can bad writers be good thinkers?"; "Are good thinkers good writers?"; "Some writing skills undermine thought"; and "Plain-talk writing: The new literary obfuscation."

Revised citations using markdown format; added last two citations. - Jan. 15,2012, 1 p.m.

Truth seekers should deliberately impose costs on themselves for holding false beliefs. That is, they should increase the cost of being wrong. One way to do this is to bet on your beliefs. Another way is to bond your beliefs: post a bond that you will forfeit if your prediction is wrong. Yes, imposing such costs is bothersome, but for truth seekers the tradeoff is easily worth it.

I think the hardest step is deciding you just want to know what's true.

More or less accurate, though of course there's a ton of stuff that was left unsaid. "Study fields like these" is easy to say, "learn skills like these" is really difficult. There's no easy way to communicate skills, and sanity is a skill-set. You kinda just have to hope people have enough lucidity to make connections between fields and reliably see single step implications, and enough ambition to seek out and learn the skills from better thinkers than themselves. E.g. thanks to having brilliant friends I know a lot of cognitive tricks and verbal patterns that can't be learned from books or blog posts. Before I had these skills I wasn't a very good truth-seeker, and I'm sure a lot of people get stuck in that valley. Luckily a surplus of lucidity and reflectivity gets you a long way by itself.

…What would that look like?

The immediate answer which I cannot shake off is - Stanislaw Lem:) or, perhaps, the hero of his novel "Runny nose".

Something is missing here. Curiosity about what? Are we only supposed to care about having true beliefs or are we supposed to care about what those true beliefs tell us about the world? I'm betting on the latter. In fact, I would go further and say that it is better to have lots of literally false but approximately true beliefs about the world than to have only a few, completely true beliefs about the world or even to have the same number of completely true beliefs but not have them cover as much interesting territory.

I don't have a theory of what makes intellectual territory more or less interesting, but I am pretty sure that content matters. I could have infinite curiosity and complete commitment to having true beliefs and still never know anything really interesting if my curiosity is about the results of simple addition problems and I am careful in carrying out those additions.

In their search for "true beliefs" they would quickly discover that there is no such thing as "actually true" but that science deals in more and more viable models. So, they would abandon their search for "truth" and would go onward to the search for better and better fitting models. (See definitions of science ... and, for a philosophical point of view, radical constructivism).

Our senses don't perceive the "real" world. They build a highly refined and effective illusion of complete perception. (See for example the blind spot in the eye, the physiology of color perception, or our sense of hearing.

Likewise, our minds always use simplified models. No one would be able to catch a falling ball if he had to actually calculate the flight curve - yet, even children are able to do it. That's because if you keep your eyes fixed at the ball and have to lower your head in a constant way you are standing at the right spot to catch the ball. (If your rate of head-lowering is slower you have to move till it is.)

So, doing the actual calculations would be a waste of time, because there's a simpler way to catch a ball.

Even if one uses always the best and newest models science provides, you will never, ever be really at the frontier because so many papers are published every day. And even if one could, scientists are able to err. And do so frequently.

(See for example selection bias. There are LOTS of papers about it.)

So, someone on a quest to find "truth" is a romantic twerp who will accomplish nothing, because he will expect to find something static and final. Science and understanding are processes.

The only fields of human endeavor where you find "truth" are mathematics and religion.

(Most things I mention in this article are findable in the wikipedia. If you don't understand, look them up. Everyone should now about scientific models, "truth" and constructivsm. Oh, and the physiology of our senses.)