Metaphilosophical Mysteries

One issue that one runs into with your question is how one defines a new field being spun off. Some people have argued that biology didn't really split off from philosophy until the 1850s and 60s, especially with the work of Darwin and Wallace. This is a popular view among Kuhnians who mark a field as becoming science when it gains an overarching accepted paradigm. (However, one could argue that the field left philosophy before it entered science.)

The word "scientist" was first used in 1833, and prior to that "natural philosopher" was used. But certainly by the late 1700s, they were practicing what we could call science. So that argument fails even if one extends the date.

Economics is generally thought of having split off from philosophy when Adam Smith wrote The Wealth of Nations, and that's in the late 18th century. But arguably, merchantilist ideas were a form of economics that predated Smith and were separate from philosophy. And you could push the date farther up, pointing out that until fairly late most of the people thinking about economics are people like Bentham who we think of as philosophers.

Possibly the best example of an area that split off recently might be psychology. Wilhelm Wundt is sometimes regarded as the individual who split that off, doing actual controlled scientific experiments in the late 19th century. But there was research being done by scientists/biologists/natural philosophers much earlier in the 19th century, especially in regards to whether the nervous system was the source of cognition. Wikipedia claims that that work started as early as 1802 with Cabanis (this is surprising to me since I didn't realize he was that early). One could argue given all the subsequent Freudian and Jungian material that psychology didn't really split off from philosophy until that was removed from mainstream psychology which was in the 1960s and 70s. However, that seems like a weak argument.

Linguistics might be another example, but again, how you define the split matters. It also runs into the not tiny issue that much of linguistics spun off from issues of philology, a field already distinct from philosophy. But other areas of linguistics broke off later, and some people still seem to think of issues like Sapir-Whorf as philosophical questions.

So a lot of this seems to depend on definitions, but regardless of definitions it seems clear that no field has spun off in the last 30 years. Going back farther makes the question murkier, but a decent argument can be made that there has been no such spin off in the last 150 years.

It depends how big you need a "field" to be. Much of philosophical logic split off to become mathematical logic (split complete by about 1930). That left the philosophers pondering things like entailment, tense logic, modality, epistemic logic, etc. But around 1960, Kripke put that stuff on a solid basis, and these kinds of logics are now an important topic in computer science. Certainly utility theory, decision theory, and subjective probability have only come over from philosophy (to econ, math, and AI) within the past 150 years. And there are still philosophers involved at the forefront of all these fields.

While people say this sometimes, I don't think this is accurate. Most of the "AI" advances, as far as I know, haven't shed a lot of light on intelligence. They may have solved problems traditionally classified as AI, but that doesn't make the solutions AI; it means we were actually wrong about what the problems required. I'm thinking specifically of statistical natural language processing, which is essentially based on finding algorithms to analyze a corpus, and then using the results on novel text. It's a useful hack, and it does give good results, but it just tells us that those problems are less interesting than we thought.

Another example is chess and Go computing, where chess programs have gotten very very good just based on pruning and brute-force computation; the advances in chess computer ability were brought on by computing power, not some kind of AI advance. It's looking like the same will be true of Go programs in the next 10 to 20 years, based on Monte Carlo techniques, but this just means that chess and Go are less interesting games than we thought. You can't brute-force a traditional "AI" problem with a really fast computer and say that you've achieved AI.

I was going to suggest that, but didn't want to stretch the layout too much. How long ago did we start using the stars to navigate? I wouldn't imagine it would pre-date agriculture, and can't think evidence we could have to suggest our ancestors from before then used the stars to navigate.

That said, I think (1), that astronomic ability is almost universal in eye space, is true. (At least for eyes that can see predators and obstacles in the dark, which much more strongly selected for than being able to navigate by starlight is.)

Huh, not sure. If I look at the sky with my left eye without wearing my glasses, I can barely see the stars, but I'm pretty sure I could see predators.

Pray tell, what is the right thing to update?

For instrumental rationality, yes; for epistemic rationality, no. If the reason the EU-maximizer loses money is because it believes that the encoding will be different than it actually is, then it is irrational.

Wei Dai distinguishes between objective rationality and subjectively obective morality:

in ethics we find it hard to do better than to identify "morality" with a huge blob of computation which is particular to human minds, but it appears that in decision theory "rationality" isn't similarly dependent on complex details unique to humanity.

Morality does seem to be more complex than decision theory. But aren't they both subjectively objective? Just as we can define the class of moral agents as those which respond to moral arguments, we can define the (larger) class of rational agents as those which respond to arguments about rationality.

I'm familiar with that. As I understand it, EY does not say that Bayesian updating is suboptimal. If anything, he says the opposite, that standard rationality gives you the right answer.

Could you be more specific about where you believe that article claims Bayesian updating is the wrong thing to do?

This additional point is controversial even here?

Also, does the "professional philosopher community" have reality-grounded standards for what constitutes "good philosophy"? And could they say what the consequences would be of making such errors (relative to the current body of knowledge)?

Because without that, then being rejected by mainstream academic philosophy is no more worrisome than if you were criticized for not being up-to-date with the top theology, or not knowing which writers were truly "post-colonial".

From some authors, I get the impression that their standard is no more rigorous than, "what all my buddies in major philosophy departments agree with".

Really? As far as I can tell, the consensus for Bayesian updating and expected utility maximization among professional philosophers is near total. Most of them haven't heard of UDT yet, but on Less Wrong and at SIAI there also seems to be a consensus that UDT is, if not quite right, at least on the right track.

From my (anecdotal but varied) experience talking to professional philosophers about them, I'd (off-the-cuff) estimate 80% are not familiar with expected utility maximization (in the sense of multiplying the probability of outcome by the utility) or Bayesian updating, and of the rest, a significant portion think that the Bayesian approach to probability is wrong or nonsensical, or that "expected utility maximization" is obviously wrongheaded because it sounds like Utilitarianism.

I run into a fair number of epistemologists who are not keen on describing beliefs in terms of probabilities and want to use binary "believe" vs "not believe" terms, or binary "justification." Bayesian updating and utility-maximization decision theory are pretty dominant among philosophers of probability and decision theorists, but not universal among philosophers.

Just skim the Stanford Encyclopedia of Philosophy articles on probability and see how uncontroversial philosophers in general regard Bayesian inference. I think you'll see that they consider it problematic and controversial in general.

Really? As far as I can tell, the consensus for Bayesian updating and expected utility maximization among professional philosophers is near total.

According to the The PhilPapers Survey, 25.8% (ETA:Wrong number, 23.6% is the correct value. I quoted from the wrong entry) of surveyed philosophers were consequentialists of some form. That makes it hard to argue for a consensus about maximizing expected utility.

But how do you mathematicize an area, except by doing philosophy? I mean real world problems do not come to you in the form of equations to be solved, or algorithms to be run.

This seems to run into SilasBarta's inquiry above about what you mean by philosophy. I wouldn't for example think of the work of people like Galileo and Newton to be doing philosophy, but they took physics and put it on solid mathematical grounding. Similar remarks apply to Lavoisier or many people in other fields.

I may be repeating what Vladimir said here, but it seems to me your objection is basically "Oh shit! We can diagonalize!" (Which if we then collapse the levels can get us a Berry paradox, and others...)

So, yes, it follows that any system of description we can think of, there's some potential truth its corresponding "universal prior" (question - do those exist in general?) won't be able to infer. But the fact that this applies to any such system means we can't use it as a criterion to decide between them. At some point we have to just stop and say, "No, you are not allowed to refer to this concept in formulating descriptions." Maybe computability isn't the best one, but you don't seem to have actually given evidence that would support any other such system over it.

Or am I just missing something big here?

Let me distill down what timtyler and Dre have written into a concisely stated question:

Premise 1: a human's brain implements a computable algorithm
Premise 2: a human can update on evidence and become convinced that a halting oracle exists
I'm not sure if it is true that: when exposed to evidence from an actually existing halting oracle, the posterior probability of an algorithm implementing the predictions of a Premise 2-type human will exceed that of an algorithm that assigns zero prior probability to an uncomputable universe
but if so, then -- Conclusion: an agent with a Solomonoff prior can become convinced that the universe contains a halting oracle.

And my question: did I do a stupid?

You objection to the universal prior is: "what if Occam's razor breaks"?!?

Engineers would not normally be concerned with such things. Occam's razor has been fine for hundreds of years. If it ever breaks, we will update our probability expectations with whatever the new distribution is. Surely that is no big deal.

Original priors are not too important, anway. As soon as an agent is born, it is flooded with information from the world about the actual frequencies of things - and its original priors are quickly washed away and replaced with experience. If their experiences involve encounting the uncomputable, agents will simply update accordingly.

So: this topic seems like angels and pinheads - at least to me.

We should perhaps expect that, but I didn't actually do so until you mentioned it; not for the reasons you stated, at least not as quite that short and obvious a sentence.

Bravo.

Consider a universal prior based on an arbitrary logical language L, and a device that can decide the truth value of any sentence in that language. Such a device has no finite description in L (according to Tarski's undefinability theorem), so the universal prior based on L would assign it zero probability.

What do you mean by "decide the truth value"? Most statements aren't valid or unsatisfiable, there is no truth value for them. We are not assuming any models here, just assigning plausibility to (statement) elements of language's Lindenbaum algebra.

Such a device has no finite description in L (according to Tarski's undefinability theorem), so the universal prior based on L would assign it zero probability.

Whatever model you have in mind, it will be categorized on one side of each statement of the language. We are assigning plausibility to statements, and hence classes of structures, not individual structures (which are like individual points for a continuous distribution).

Maybe the human is a bad philosopher in this case and is simply wrong.

Re: "Consider a universal prior based on an arbitrary logical language L, and a device that can decide the truth value of any sentence in that language. Such a device has no finite description in L (according to Tarski's undefinability theorem), so the universal prior based on L would assign it zero probability."

It would never see the infinite description with the 0 probability, though - not enough space-time.

The evidence of the Oracle that the agent would get to see would be in the form of finite sensory inputs - and those would not be assigned zero probability. So: it could update on that evidence just fine - with no problems.

If the agent sees a tiny box with an Oracle inside it, that is just more finite sense-data about the state of the universe to update on - no problem - and no silly p=0 for an empirical observation.

Why "new terms"? If the language can finitely express a concept, my scheme gives that concept plausibility. Maybe this could be extended to lengths of programs that generate axioms for a given theory (even enumerable sets of axioms), rather than lengths of individual finite statements, but I guess that can be stated within some logical language just as well.

This is one of those situations where we should probably take multiple approaches simultaneously.

This is reasonable, and I agree.

The opposite explanation also works: we use so many unrelated heuristics that there's no single area where they all fail simultaneously.

we can't be sure at this point that we have zero self-shadowing blind-spots, I think there's a reasonable chance that's in fact the case.

there is a single "ability to philosophize" that is sufficient (given enough raw intelligence) to overcome all such blind spots.

So, one way to attack this position would be to construct a toy-model of a system that has an "ability to philosophize" but that still fails in some cases.

An example would be the bayesian AI that self-modifies to one-box on all newcomblike problems where omega examines it after that self-modification event. So it realizes that it is better to be "rationally irrational", but only in a limited sense.

A less controversial example is the case where some devout catholics were convinced that even thinking about whether or not god might not exist would cause them to be sent straight to hell.

For any agent or community of agents, there are some cherished beliefs that the agent/community refuses to challenge. Sometimes for good reason. Even I have some, and LW certainly has some.

In this latter case, a false belief shields itself from criticism by convincing the community or agent involved that even the act of questioning the belief is of highly negative value.

The timeless/updateless/acausal intuitions come from the human intuitions about pride, dignity, honor, etc, which were developed because humans interacted with other humans.

Umm... I first thought of the updateless idea while trying to figure out anthropic reasoning. I fail to see how that had anything to do with pride, dignity, honor, etc.

I think this is correct. Using my formulation, the Bayseian system is what I call a "theory of reality", the timeless one is the "theory of mind", which I see as the trade-off along the autism spectrum.

AIXI with a TM-based universal prior will always produce predictions about the black box, and predictions about the rest of the universe based on what the black box says, that are just as good as any prediction the human can come up with. After all, the human is in there somewhere. If you think of AIXI as embodying all computable ways of predicting the universe, rather than all computable models of the universe, you may begin to see that's not quite as narrow as you thought.

For example, if our universe does in fact contain halting problem oracles, the Bayesian superintelligence with a TM-based universal prior will never be able to believe that.

I think this problem would vanish if you spelled out what "believe" means. The Bayesian superintelligence would quickly learn to trust the opinion of the halting problem oracle; therefore, it would "believe" it.

P(true) = 0 - or p(false) = 1 - seem like trivial mistakes to avoid.

A "expected utility maximizer programmed with a TM-based universal prior" would surely not care very much if it was programmed with wrong priors after a while - since it would not be depending on the details of its priors much any more - due to having a big mountain of experience concerning what the actual expected frequency of events was. Its priors would be swamped by data - unless its priors were completely crazy.

The OP must be thinking of some different type of construct from me - and he doesn't seem to explain what it is.

Maybe it's just too hard-core for the average LWer's tastes? High score maybe corresponds to a combination of sufficiently clever but also somewhat down-to-earth?

Philosophy has never provided the basic definitions, concepts, or even motivations for math. The historical influence was the other way around: The successful use of math inspired the invention of philosophy as a rational discipline.