What Makes My Attempt Special?

I also note that in this case other people may have competing outright opposing interests (in the 'aggressively want to get' sense rather than just the 'curiosity' sense). In such cases it isn't a matter of not interesting enough. Some people will have an interest in outright suppressing that kind of discovery. I have a relative who has been battling to get research done on alternative fertaliser methods. Pivot really don't like people doing that and they have purchased a lot of power in most of the relevant academic institutions.

Prognosis is good for making discoveries, but abysmal for gaining status thereby.

Voted up for clear thinking and relevance.

And, as this is Less Wrong, I would add a seventh:

7) Others have thought of your question, found it challenging to let the question hang long enough to do research, failed to meet the challenge, let their biases choose their favorite answer to the question, and ran enough tests to produce a superficial veneer of data supporting their preferred answer. Your superior rationality might be enough to attempt an honest answer to the question, but you must also consider the possibility that the question is very hard to consider fairly. The prognosis: good, unless your Bayesian Jedi mind tricks are weak.

There are two equal and opposite faults here. One is assuming that you have some special kind of genius to see the answer to a problem which others have tried and failed to answer.

The opposite mistake is thinking that everything has been tried before - that there is nothing new under the sun. There are quite a lot of old and obvious questions still out there in plain sight which nobody has answered, and which aren't actually all that hard. There are also many easy questions out there where the given answer is wrong, and the right one can be given quite easily. The world of science is still very new, and lots of things have not yet been sorted out.

In my experience, the dumbest thing that's often done is answering a question that's already been answered before. It's tempting because often it isn't that hard to do, and a very large proportion of scientific research seems to do it. It's another good reason to research your field.

Part of the reason this happens is simple isolation. Another group is using different jargon, different techniques and so on, and because search relies on words, you never find the work that's been done before. It really isn't easy to know if nobody has asked your question before, unless there's an answer too.

1) Nobody else has ever thought of your question, even though all of the pieces of knowledge needed to formulate it have been known for years. If the field has many people involved, the probability of this is vanishingly small and you should systematically disabuse yourself of your fantasies if you think like this often.

I think this might underestimate the number of good questions/ideas in a field relative to the number of researchers. I think humans are already dealing with utterly huge problem spaces and carving out little areas of working technology, But there are many orders of magnitude times more of other possible technologies.

It could be that your idea is based on old information, but you only thought of it because it serves some weird specific goal, and no one in history has had that goal and knowledge of your field.

Or you're combining old information from several obscure fields.

I think there are other hints as to whether your ideas has been though of before, but I don't have a way to express them other than "obviousness".

8) If the question is in a field X, but the question was posed by people not in field X. This is a good sign.

Note that this sometimes happens in math where people will pose a question in one area of math and not realize that it is really equivalent to or related to some question in another branch of math.

Some questions/answers are hard to think of, not because we are missing knowledge or technology but because they are hard to think of.

I guess this is basically case (1), but I think "vanishingly small" is a bad description in most fields. For example, a significant fraction of advances in theoretical fields are of this form (the exception being for rapidly developing fields). I agree that if you think you have found a proof that P!=NP you should stop and reflect on how many people have thought for a very long time about it, but in general you should not be shocked that you have thought of a question or answer that eluded other researchers, potentially for a rather long time (since there are very many questions and not that many researchers equipped to ask or answer most of them).

I guess you are claiming that the situation is fundamentally different for experimentalists, but I would still be rather shocked if (1) was really negligible. At best there may be some fields where (1) is particularly uncommon (and for people who are still deciding what they want to do with their lives, it seems like this suggests a lot of interesting data which are really hard to extract from the normal discourse about possible fields of study).

Recently I've run into a case where my best guess for what's happened is the following:

10) There's an obvious stopping point, so the people who have previously looked at this question stopped there, without checking for easy ways to continue past. (Related to magfrump's comment, I think.)

I don't think that 3) is a bad sign if you can argue why other didn't find it interesting but it's actually interesting. A lot of scientific breakthrough came from answering questions that nobody thought to be interesting at the time.

A good candidate would be a question that explores unexplored areas but that isn't high status.

All of your bullet points involve the opinions or results of other people. That may be important, and useful as a fast heuristic, but it seems like it will promote groupthink.

9) Your question combines insides from many different fields and there aren't many people around who are interested in the same fields.

Related RSA Animate: "Where good ideas come from"

http://www.youtube.com/watch?v=NugRZGDbPFU

A good list, and I'm enjoying peoples' additions. An important point to keep in mind when formulating this sort of analysis is not to generalize more than the system in question merits. In this case, it's not emphasized that most research is institutionalized, and fueled by the social needs or the beliefs of non-researchers.

I've heard many people suggest a variation on #3 + #4:

11) People have thought of your question before, but research has always been smothered by non-researchers. Those who provide resources for research in that field haven't found the question to be worth their money/approval/etc. If you find independent funding from an iconoclast, and have the chutzpah to break from the primary research institutions, you have a unique opportunity.

Andy, it seems you might have experience in a particular field and I'd be interested to know which one if you're inclined share that info.