'Is' and 'Ought' and Rationality

It may not be great, but I did give an argument. Roughly, again,

a) wants do entail oughts (plausible) b) wanting = being in unproblematically naturalistic state ABC (from assumption of naturalism) c) from a and b, there is some true statement of the form 'being in naturalistic state ABC entails an ought' d) but no claim of the form 'being in naturalistic state ABC entails an ought' is plausible

I infer from the contradiction between c and d to the falsity of b. If you could formulate your dissatisfaction as a criticism of a premise or of the reasoning, I'd be happy to listen. In particular, if you can come up with a plausible counter-example to (d), I would like to hear it.

Suppose I hear Bob say "I want to eat an apple." Am I justified in assigning a higher probability to "Bob wants to eat an apple" after I hear this than before (assuming I don't have some other evidence to the contrary, like someone is holding a gun to Bob's head)?

I think this question hits the nail on the head. You are justified in assigning a higher probability to "Bob wants to eat an apple" just in case you are already justified in taking Bob to be a rational agent (other things being equal...). If Bob isn't at least minimally rational, you can't even get so far as construing his words as English, let alone to trust that his intent in uttering them is to convey that he wants to eat an apple (think about assessing a wannabe AI chatbot, here). But in taking Bob to be rational, you are already taking him to have preferences and beliefs, and for there to be things which he ought or ought not to do. In other words, you have already crossed beyond what mere natural science provides for. This, anyway, is what I'm trying to argue.

Rational-ought beliefs and actions are the ones optimal for achieving your goals. Goals and optimallity can be explained in scientific language. Rational-ought is not moral-ought. Moral-ought is harder to explain because it about the goals an agent should have,not the ones they happen to have.

I'm sincerely not sure about the rational-ought/moral-ought distinction - I haven't thought enough about it. But anyway, I think moral-ought is a red herring, here. As far as I can see, the claims made in the post apply to rational-oughts. That was certainly the intention. In other posts on LW about fallacies and the details of rational thinking, it's a commonplace to use quite normative language in connection with rationality. Indeed, a primary goal is to help people to think and behave more rationally, because this is seen for each of us to be a good. 'One ought not to procrastinate', 'One ought to compensate for one's biases', etc..

Try somehow to shoehorn normative facts into a naturalistic world-view, at the possible peril of the coherence of that world-view.

Easily done with non-moral norms such as rationality.

Would love to see the details... :-)

Not if you think of purpose as a metaphysical fundamental. Easily, if a purpose is just a particular idea in the mind. If I intend to but a lawnmower, and I write "buy lawnmower" on a piece of paper, there is nothing mysterious about the note, or about the state of mind that preceded it.

I'm not sure I get this. The intention behind drawing the initial distinction between is/ought problems was to make clear the focus is not on, as it were, the mind of the beholder. The question is a less specific variant of the question as to how any mere physical being comes to have intentions (e.g., to buy a lawnmower) in the first place.

That you want to do something does not mean you ought to do it in the categorial, unconditional sense of moral-ought.

I agree, but I think it does mean you ought to in a qualified sense. Your merely being in a physical or computational state, however, by itself doesn't, or so the thought goes.

There is no 'open question' issue here - "mistake", like "exothermic", does not have any prior metaphysical meaning. We are free to define it as we wish, naturalistically.

I'm having trouble with the word "metaphysical". In order for me to make sense of the claim that "mistake" and "exothermic" do not have prior metaphysical meanings, I would like to see some examples of words that do have prior metaphysical meanings, so that I can try to figure out from contrasting examples of having and not having prior metaphysical meanings what it means to have a prior metaphysical meaning. Because at the moment I don't know what you're talking about.

An agent morally ought not to do something iff it tends to generate consequences contrary to the agent's interests, those negative consequences arising from the reactions of disapproval coming from other agents.

That doesn't work. It would mean conformists are always in the right, irrespective of what they are conforming to.

Can we also define this kind of ought naturalistically? I think that we can, because once again I deny that "wrong" has any prior metaphysical meaning. The trick is to make the new (by definition) meaning not clash too harshly with the existing metaphysical connotations

The main trick seems to be getting people to agree on a definition. For instance this:.

How is this for a first attempt at a naturalistic definition of the moral ought as a subset of the practical ought? An agent morally ought not to do something iff it tends to generate consequences contrary to the agent's interests, those negative consequences arising from the reactions of disapproval coming from other agents.

...aims rather low. That just tells people to do what they would do anyway. Part of the social function of morality is to give people an ideal to personally aim towards. Another part of the social function of morality is to provide people with an ideal form of behaviour, in order to manipulate others into behaving "better". Another part of the social function of morality is to allow people to signal their goodness by broadcasting their moral code. Done right, that makes them seem more trustworthy and predictable. Your proposal does not score very well on these fronts.

I think this is right, except possibly for the part about no prior metaphysical meaning. The later explanation of that part didn't clarify it for me. Instead, I'll just indicate what prior meaning I find attached to the idea that "the virus replicated wrongly."

In biology, the idea that organs and behaviors and so on have functions is quite common and useful. The novice medical student can make many correct inferences about the heart by supposing that its function is to pump blood, for example. The idea preceded Darwin, but post-Darwin, we can give a proper naturalistic reduction for it. Roughly speaking, an organ's function is F iff in the ancestral environment, the organ's performance of F is what it was selected for. Various RNA features in a virus might have functions in this sense, and if so, that gives the meaning of saying that in a particular case, the viral reproduction mechanism failed to operate correctly.

That's not a moral norm. It's not even the kind of norm relating to an agent's interests, in my view. But it is a norm.

There was a pre-existing meaning of "biological function" before Darwin came around. So, a Darwinian definition of biological function was not a purely stipulative one. It succeeded only because it captured enough of the tentatively or firmly accepted notions about "biological function" to make reasonably good sense of all that.

If I bet higher than 1/6th on a fair die's rolling 6 because in the last ten rolls 6 hasn't come up -meaning it's now 'due'- I make a mistake. I commit an error of reasoning; I do something wrong; I act in a manner I ought not to.

What about the virus particle which, in the course of sloshing about in an appropriate medium, participates in the coming into existence of a particle composed of RNA which, as it happens, is mostly identical but differs from itself in a few places. Are you saying that this particle makes a mistake in the same sense of 'mistake' as I do in making my bet?

Option (1): The sense is precisely the same (and it is unproblematically naturalistic). In this case I have to ask what the principles are by which one infers to conclusions about a virus's mistakes from facts about replication. What are the physical laws, how are their consequences (the consequences, again, being claims about what a virus ought to do) measured or verified, and so on?

Option (2): The senses are different. This was the point of calling the RNA mistake metaphorical. It was to convey that the sense is importantly different than it is in the betting case. The idea is that the sense, if any, in which a virus makes a 'mistake' in giving rise to a non-exact replica of itself is not enough to sustain the kind of norms required for rationality. It is not enough to sustain the conclusions about my betting behaviour. Is this fair?

Compare What Do We Mean By Rationality.

You're right that I didn't mean this necessarily to be about a specifically moral sense of 'ought'. As for the suggested inference about baby-punching, I would push that onto the 'other things being equal' clause, which covers a multitude of sins. No acceptable theory can entail that one ought to be punching babies, I agree.

The picture I want to suggest should be taken seriously is that on one side of the fence are naturalistic properties, on the other are properties such as rationality, having wants/beliefs, being bound by norms as to right and wrong (where this can be as meagre a sense of right and wrong as "it is right to predicate 'cat' of this animal and wrong to predicate 'dog' of it" -or, "it is right to eat this round, red thing if you desire an apple, it is wrong to eat it if you desire a pear"), oughts, goals, values, and so on. And that the fence isn't as trivial to break down as one might think.

I'm understanding a utility function is something like a mapping of states of affairs (possible worlds?) onto, say, real numbers. In this context, the question would be giving naturalistic sense to the notion of value -that is, of something's being better or worse for the agent in question- which the numbers here are meant to correlate to. It's the notion of some states of affairs being more or less optimal for the agent -which I think is part of the concept of utility function- which I want to argue is outside the scope of natural science. Please correct me on utility functions if I've got the wrong end of the stick.

To be clear - the intent isn't to attack the idea that there can be interesting and fruitful theories involving utility functions, rationality and related notions. It's just that these aren't consistent with a certain view of science and facthood.