The Value of Theoretical Research

In the case of calculus, differential equations, statistics, functional analysis, linear algebra, group theory, and numerical methods, the important results for modern work were in fact developed after their usefulness could be appreciated by an intelligent observer.

This is simply untrue, unless you've rigged the definition of "intelligent observer" and added a dose of hindsight bias. It is unlikely that the true extent of the "practical" importance of calculus today could have been predicted by even the most imaginative of Newton's fellow Cambridge dons in the late 17th century. In that era, thinking about things like the orbits of celestial bodies was "idle speculation" par excellence. It's hard to appreciate this, because it seems so obvious in retrospect that we would have space rockets, doesn't it? Not to mention the use of differential equations in fields like economics, whose existence was a century away but were just so clearly on humanity's horizon, right?

Functional analysis is a particularly interesting choice of example. The fact that its application to quantum mechanics (which is what I presume you were thinking of) arose concurrently with the development of the subject itself was largely a fortuitous (if serendipitous) coincidence. The actual "physical" roots of the subject were more indirect, via differential/integral equations and the calculus of variations (18th-century physics, in other words), and it was basically the result of mathematicians' attempt to turn these somewhat ad-hoc disciplines into nice-looking abstract theories.

As for group theory, its origins lie in the attempt to solve the quintic by radicals -- about as "useless" an undertaking as could be imagined. (The cubic and quartic formulas already being much too complicated for practical use.)

Realistically, an argument like yours, made back in the day, would have shown that Newton should have devoted his life to inventing better agricultural tools. And it might have been a good argument -- applied to someone other than Newton. (They could really have used better agricultural tools, no doubt.)

If you don't feel satisfied doing math, or think you could make a better contribution doing something else, you shouldn't be doing it. But don't make the mistake of pretending that your argument generalizes.

And see here regarding the nature of mathematics' usefulness, which doesn't reside in specific "applications".

Paul, have you tried to reverse engineer why your brain made you become interested in doing pure math in the first place? I ask because it sounds like you came up with this list of explicit arguments about the value of such research after you already became interested in it on an intuitive level.

Do you think you now understand what your intuition was doing? Does it now seem like a (subconscious) miscalculation, and if so, can you possibly explain what is the nature of that miscalculation?

I'm getting a PhD in math and I have had similar thoughts. A few quick remarks about your specific arguments followed by my personal take:

In rebuttal 1, you mention number theory as an example where the application to crypto took a long time to be apparent. I'm a number theorist and I don't find this argument compelling. Much of the number theory that is used in crypto is not deep. There's nothing about Diffie-Hellman or RSA that requires the hundreds of years of number theory research that has gone into it. One could explain the algorithms for such procedures to a mathematician in the early 1800s with little effort (although the idea of having very efficient methods of arithmetic might strike them as very odd). Moreover, while other parts of number theory have turned out to be relevant it is still a very tiny fraction of all number theory.

A possibly better example would be non-Euclidean geometries which really were studied in detail in the 19th century before they were found to have practical applications.

As for myself, I enjoy math a lot, and I suspect that I will be more productive at areas I enjoy than areas I enjoy less. This might be a rationalization but it connects to another aspect: I'm not a good utilitarian. Given the choice between me having a happy life and being somewhat productive for humanity and given the choice between being less happy and more productive for humanity I'll choose being more happy. (When I phrase it that way it triggers far more reactions that lead me to not want to do that. But what decisions I make on a day to day basis about what to think about and what to do with my time are very much near mode).

For many of us, choosing a career path has a dominant effect on our contribution to the society. For those of us who care what happens to society, this makes it one of the most important decisions we make.

Do you really see this as a one-time choice that you are stuck with for the rest of your life? I think that most people (in the US, at least) find themselves making decisions that change their career paths right up until retirement (and beyond).

A suggestion. Write down the names of three senior people in your field whose contributions you admire. Now add three people who made important discoveries in their youth thirty years ago and are still alive. Add three people who you think have made significant contributions to society in some way related to your field. And finally, three people who made significant contributions to society from any field.

Take a look at the CVs and/or biographical sketches of these people. Look particularly at the career decisions they made - at your age and later in their careers. I'm not sure what you will find, but I have my guesses:

• Very few people spend a lifetime doing pure research.
• The people who contributed most to society did not consciously set out to do so at the start of their careers.
• You get lots of opportunities along the way to steer your career path. But the true helmsman of notable people's careers is serendipity.

I guess you feel the need to maximise your utility in society? You are certain that this is a moral necessity for you? I got into research because I thought it would be interesting, and pay reasonably well, and I didn't want to commit to a real world job. Oh, and potentially i might make more a difference than in other fields I'd considered. I'd also argue that its not that easy to switch fields- I went into statistics because thats where I found my knowledge easiest to apply. I suspect I could have done as well in politics, or in historical study, or in the financial sector. I chose my sector because I felt I'd enjoy doing it.

There are many types of math, with differing sorts of value, but I can say a little about the sort of math I find moving.

I agree with you. For the most part, applied souls dream up their advances and make them without relying on the mathematical machine. They invent the math they need to describe their ideas. Or perhaps they use a little of the pure mathematician's machine, but quickly develop it in ways that are more important to their work than the previous mathematical meanderings.

I think you underestimate the role of mathematics as the grand expositor. It is the tortoise that trails forever beyond the hare of applied science. It takes the insights of applications, of calculus for example, and digests them. It reworks them, understands them, connects them, rigorizes them.

The work of mathematics is not useful in your mind because a mathematician does not make a truly new applied advance. A mathematician invents and connects notations to ease the traversal, the learning, and most importantly the storage in working memory of past insights.

What is the purpose of a category? An operad? A type theory? A vector bundle? The digit 0? When these languages were introduced, it could always be claimed they were worthless because the old languages could express the same content as these new languages. But somehow the new language makes it easier to conceptualize and think about the old ideas; it increases the working human RAM.

And what of the poor student? He who must learn so many subjects is grateful when it is realized that many of those subjects are in fact the same: http://arxiv.org/abs/0903.0340 . Mathematics digests theories and rewrites them as branches of a common base. It makes it possible to learn more insights quickly and to communicate them to the next generation.

So young applied scientists, perhaps generations later, benefit by more compactly and elegantly understanding the insights of their forebearers. Then, the mathematician dreams, they are freer to envision the next great ideas: http://arxiv.org/abs/1109.0955

So why the mathematician's focus on solving specific problems? Why so much energy to characterize finite groups? It is not that these problems are important. It is that they serve as testbeds for new languages, for new characterizations of old insights. The problems of pure math are invented as challenges to understand an old applied language, not to invent a new one.

After writing the third paragraph below, it would appear I think your rebuttals to Arguments 4 & 7 are most salient. I believe the idea that fundamental research is very important for future discoveries to be possible, but I have an argument for that which you don't list. The great minds that gave us theories of gravitation, evolution, and quantum mechanics all learned their fields by doing research. Some of them did basic theoretical work, and others did applied work. But, even the ones who did applied work may have learned significantly from people who were more suited, or who more enjoyed, theoretical work.

So, if you really, really love the theoretical stuff, and you're just worried that you'll feel your life has been a waste at the end, being afraid of lifelong commitment is not a good reason by itself to fail to commit. Honest curiosity about other fields is valid, and if that's at least part of what you're feeling, read on. (Well, you may read on anyway, I'm aware :)

What else have you tried doing? Have you ever worked in a position where you did something other than pure research? If the answer is "No," I would say you should definitely value trying something else, at least for a little while. If you are currently a research graduate student, you are in the perfect position to take a year off and do just that. Apply for an internship to do math modeling for an oil company (or work at a radio telescope, or something else that has a practical application.) I did a one year internship at Los Alamos National Labs as a spacecraft payload operator based on my undergrad physics degree. In a "real job," you have several different kind of responsibilities--not just different responsibilities, but different kinds. I checked the daily health reports on the satellite, yes. But I also attended meetings of top astrophysicists, getting insight into how they think and what they do. (One of those scientists, Roger Fenimore, taught me the lesson that the people who really make important things happen often get experience from multiple disparate fields, and then notice important connections between them.) I investigated small failures in the satellite data, learning about materials science and clean room procedures along the way. I gave tours of our facility to visitors. I participated in a student council, helping to improve student life in a small, isolated town.

If you're a professor already, you're in a bit more of a pickle, because there's no guarantee of a place to come back to if you leave. Still, it might be worth the risk.

[26 Feb 2011: Edited for intended generality. I do not think working for an oil company is really your only choice. It's just an example of something a math friend of mine did.]

The way most people can best contribute to society is to make as much money as possible and donate much of it to a charity that offers a high social return per dollar.

If you contribute to a charity that increases by one part in a trillion the probability of mankind surviving the next century and if conditional on this survival mankind will colonize the universe and create a trillion times a trillion sentient lifeforms then your donation will on average save a trillion lives.

If you think that people working in synthetic biology and bioengineering are doing worthwhile work (and I entirely agree that they are), then go help them. Why the ennui? Set yourself to spend a month investigating these fields and find if you are able to suss out interesting ideas that might (how can you know?) be of use. If your imagination is sparked, then you should find a job in a lab on a trial basis and take your investigations further. I would encourage anyone with a good mind to go into this area of research, as it will doubtless benefit me (I cannot speak to society).

I think your arguments against the utility of mathematics can be applied generally to any science, which is why I reject them. However, the weakness of my objection (it relies on unstable induction) is also the weakness of your argument. Look, sure, you cannot KNOW that what you are doing is going to result in something useful. But I see no evidence at all that anyone who has made worthwhile discoveries knew otherwise. It just is not true, we have no evidence for it, that Newton set out to lay down the mathematical foundations of physics for the benefit of anyone. He seems to have done it for reasons of curiosity and perhaps ambition. I imagine he had a bit of fun with it. Like it or not, this is why people do things, especially when said things require years of work.

I would posit (but do not know), that if you do want to make a useful contribution, the state of ignorance is exactly the right position to be. The x-ray, the laser, the computer, antibiotics, physics, Greek geometry, etc. etc. down the line are the result of accident, aimless research motivated only by curiosity, or people having fun with ideas. Some of these might even have been the result of chaps trying to get the girl. That is how it goes. I see almost no evidence at all (with exceptions of specific technologies, the airplane, for example) that the best way to go about making discoveries is to trying to make specific discoveries. You get interested in something and, if you find something useful, good for you, but most people do not. Given this, that we would find that the most successful scientists are motivated by curiosity, playfulness, and perhaps a little ambition. A survey of the history of science reveals, I think unquestionably, that yes, this is exactly the case!

It is certainly possible, even likely, that if you do spend your time doing theoretical math, that you will do nothing of importance or use to society. The chances, I think, are, at best, only very slightly better if you switch fields to do something else. You should do what gets you excited and interested, because only then, no matter what your pursuit, can you really increase your chances of doing something useful for yourself and society. At the very least, you will be happy, and that is not nothing.

I think this is a very important topic. Clearly the real goal would be a general algorithm that would allow young people to decide on their career paths in such a way to have maximum positive benefit to society. Such an algorithm would necessarily give different answers in different cases (i.e. it obviously could not output "do math research" for everyone - that would be a catastrophe).

I don't know how to design that algorithm, but one heuristic rule I think would be useful is to ask: "are you independently wealthy?" If so, you should think more about careers in low-paying areas like mathematics, physics, literature, or art. If not, you should think about how to become wealthy (then your kids can become poets).

Overall, I think more people should be encouraged to pursue quotidian careers in areas that actually build tangible wealth, like construction, manufacturing, import/export, and traditional business (i.e. not dot-com or biotech). The fact that we don't encourage more people to do this kind of work stems from our weird cultural fascination with "education" and "knowledge industries".

One aspects of fundamental research (and research in general) that I see missing from this post and many other explanations of why it is not the best use of your time, is being incremental. With some very rare exceptions, the maths you actually need, even if developed at the time where it was needed, depend on many things that had to be found prior to that.

The example that comes to mind, and was not mentioned in the post or the comments (as far as I know), is the birth of computer science. You can say: yay, Turing "invented" (with a lot of other people) theoretical computer science to solve concrete problems, when it was needed. But that would completely obstruct the fact that Turing builds heavily on top of Gödel, which solved questions of a purely mathematical nature. Among the ideas Gödel's work essential to the birth of computer science, diagonalization goes back to Cantor, whose work concerns some of the most pure and abstract maths ever.

That being said, I do agree from experience that many arguments one makes about justifying doing maths or theoretical computer science do not hold under scrutiny. Yet for the reason I give above, I still think pure theoretical research is necessary.

There is a shortage of intelligent, rational people in pretty much every area of human activity. I would go so far as to claim this is the limiting input for most fields.

Uhhh I find this statement extremely surprising. I mean, come on: which labor markets see wages skyrocketing for the lack of intelligent, rational, trained professionals? Certainly not most of them. In fact, outside of a few bubble and rent-seeking fields like software development and investment banking, I do believe most of the developed world's job markets for professionals are in gluts right now, with few shortages of workers apparent anywhere. And it was more so in 2011, when some of the world's economies were growing a bit slower.

I am surprised that no one brought up the following point yet (this being LW after all): if you believe that strong AI will be developed before applications for the theoretical research are found, then, from purely practical point of view, you shouldn't dedicate your efforts to the research. It is more efficient to assist the development of strong AI instead of doing the research yourself.

As an applied mathematician, I disagree very strongly with these claims. See the paper "The Dawning of the Age of Stochasticity" by David Mumford:

http://www.dam.brown.edu/people/mumford/Papers/OverviewPapers/DawningAgeStoch.pdf.

I think the fundamental flaw in this line of reasoning is that you are forgetting just how recently calculus was invented. It was not at all that either Leibniz or Newton sat down at the desk and decided to develop the calculus. I can hardly think of a more apt moment to quote Newton himself: "What Descartes did was a good step. You have added much several ways, and especially in taking the colours of thin plates into philosophical consideration. If I have seen a little further it is by standing on the shoulders of Giants."

I would argue that the deterministic developments of what we currently consider to be "the great mathematicians" were, more or less, low-hanging fruit. That, in fact, turning a corner with Godel and Turing, mathematics research, especially in probabilistic modeling, is actually accelerating.

As to whether the pursuit of pure research is the right career choice for a given person, or whether that person can adequately assess how much of a contribution he or she can make in a research field, I feel that there are many additional factors to be weighed. For instance, many people who pursue research also pursue creative endeavors of other types and would prefer to have an unstructured career that affords them time. Being a research professor tends to concentrate stress and working hours into clumps throughout the year, leaving other spans of time with significantly fewer responsibilities. If a person wants to make a small mark in a research domain and also wishes to seriously pursue an art, such as writing or photography, this is a perfectly sensible career choice. If, however, they are strictly trying to maximize the social benefit of their having existed on the Earth, then research may not be the right pursuit. But, in that case, the computational effort expended to try to model one's own eventual contribution could be highly variable and depends on certain types of honesty which people, especially at the ages when long-term career choices are made, rarely employ.

I would also be interested in thinking more about a type of Moore's Law related to the dramatic effect of mathematical progress. If you normalize the "amount of improvement" due to a "tiny mathematical victory" in modern research by the amount of improvement of say calculus of variations, you certainly won't feel that you've accomplished a lot. The tiny blood vessels in the ends of your fingertips might not be seen to be as important as your femoral artery to a layman, but a medical scientist knows that, to some degree, "it don't work that way."

very few people are good at creating a large surplus. it is the subset of people who are good at making money and bad at spending it. if there is a chance you are one of these rare individuals who can throw off excess wealth you should pursue it and donate money to SIAI and SENS. This sort of person is much much rarer than decent researchers.