An argument that consequentialism is incomplete

We often imagine a "consequence" as a state of the world at a particular time. But we could also include processes that stretch out in time under the label "consequence". More generally, we could allow the truth of any proposition as a potential consequence. This wouldn't be restricted to a state, and not even to a single process.

I think this is intuitive. Generally, when we want something, we do wish for something to be true. E.g. I want to climb a mountain: I want it to be true that I climb a mountain.

For this argument, consequentialism is like kinetic theory of gases. The point is not that it's wrong and doesn't work (where it should), but that it's not a relevant tool for many purposes.

I started giving up on consequentialism when thinking about concepts of alignment like corrigibility and then membranes (respect for autonomy). They could in principle be framed as particular preferences, but that doesn't appear to be a natural way of thinking about them, of formulating them more clearly. Even in decision theory, with the aim of getting certain outcomes to pass, my current preferred ontology of simulation-structure of things [LW(p) · GW(p)] points more towards convincing other computations to move the world in certain ways than towards anticipating their behavior before they decide what it should be themselves. It's still a sort of "consequentialism", but the property of preferences being unchanging is not a centerpiece, and the updateless manipulation of everything else is more of a technical error (like two-boxing in ASP) than a methodology.

In human thinking, issues with consequentialism seem to be about losing sight of chasing the void [LW · GW]. Reflectively endorsed hedonistic goals (in a broad sense, which could include enjoyment of achievement) are a bit of a dead end, denying the process of looking for different kinds of aims, sometimes cynical reveling in knowing the secrets of human nature.

Doesn’t rule consequentialism (as opposed to act consequentialism) solve all of these problems (and also all^[1] other problems that people sometimes bring up as alleged “arguments against consequentialism”)?

This post strikes me as saying something extremely obvious and uncontroversial, like “I care about what happens in the future, but I also care about other things, e.g. not getting tortured right now”. OK, yeah duh, was anyone disputing that??

non-consequentialist wishes can make you go in circles

I feel like you’re responding to an objection that doesn’t make sense in the first place for more basic reasons. Why is “going around in circles” bad? Well, it’s bad by consequentialist lights—if your preferences exclusively involve the state of the world in the distant future, then going around in circles is bad according to your preferences. But that’s begging the question. If your care about other things too, then there isn’t necessarily any problem with “going around in circles”. See my silly “restaurant customer” example here [LW · GW].

I think there's two parts of the argument here:

• Issues of expressing our values in a consequentialist form
• Whether or not consequentialism is the ideal method for humans

The first I consider not a major problem. Mountain climbing is not what you can put into the slot to maximize, but you do put happiness/interest/variety/realness/etc. into that slot. This then falls back into questions of "what are our values". Consequentialism provides an easy answer here: mountain climbing is preferable along important axes to sitting inside today. This isn't always entirely clear to us, we don't always think natively in terms of consequentialism, but I disagree with:

There are many reasons to do things - not everything has to be justified by consequences.

We just don't usually think in terms of consequences, we think in terms of the emotional feeling of "going mountain climbing would be fun". This is a heuristic, but is ultimately about consequences: that we would enjoy the outcome of mountain climbing better than the alternatives immediately available to our thoughts.

This segues into the second part. Is consequentialism what we should be considering? There's been posts about this before, of whether our values are actually best represented in the consequentialist framework.
For mountain climbing, despite the heuristic of "I feel like mountain climbing today", if I learned that I would actually enjoy going running for an hour then heading back home more, then I would do that instead. When I'm playing with some project, part of that is driven by in-the-moment desires, but ultimately from a sense that this would be an enjoyable route.This is part of why I view the consequentialist lens as a natural extension of most if not all of our heuristics.
An agent that really wanted to go in circles doesn't necessarily have to stop, but for humans we do care about that.
There's certainly a possible better language/formalization to talk about agents that are mixes of consequentialist parts and non-consequentialist parts, which would be useful for describing humans, but I also am skeptical about your arguments for non-consequentialist elements of human desires.

This may be a complaint about legibilism, not specifically consequentialism.  Godel was pretty clear - a formal system is either incomplete or inconsistent.  Any moral or decision system that demands that everything important about a decision is clear and well-understood is going to have similar problems.  Your TRUE reasons for a lot of things are not accessible, so you will look for legible reasons to do what you want, and you will find yourself a rationalizing agent, rather than a rational one.

That said, consequentialism is still a useful framework for evaluating how closely your analytic self matches with your acting self.  It's not going to be perfect, but you can choose to get closer, and you can get better at understanding which consequences actually matter to you.

Climbing a mountain has a lot of consequences that you didn't mention, but probably should consider.  It connects you to people in new ways.  It gives you interesting stories to tell at parties.  It's a framework for improving your body in various ways.  If you die, it lets you serve as a warning to others.  It changes your self-image (honestly, this one may be the most important impact).  

I think this post is asking questions a lot of people ask, and most of the natural responses are already in the comments.

I'd note that when evaluating a choice or action, a consequentialist should ideally consider all of its effects, throughout the entire future light cone (aka there's no such thing as an end result, only a succession of intermediate states that ripples out forever). That's obviously computationally infeasible, but we also aren't so limited that we have to phrase desires with short sentences that can be so trivially shown are not what we really want. 

Beyond that, consequentialism doesn't tell you what you're allowed to want. There's nothing non-consequentialist with a desire being, "No I want to actually climb the mountain, the old fashioned way." But it does give you a framework from which to say, "Given that I want to climb the mountain, the course of action where you put me up there or give me a feeling or false memory of having climbed up there doesn't achieve it." Similarly, there's nothing non-consequentialist about wanting to play, or wanting to respect others' desires. You want what you want. Consequentialism evaluates whether a given choice or action gets you more or less of what you want.

Then I can give you the feeling of having climbed it!

Related: Total Recall

I don't see any limit to or problem with consequentialism here, only an overly narrow conception of consequences.

In the mountain example, well, it depends on what you, in fact, want. Some people (like my 12yo past self) actually do want to reach the top of the mountain. Other people, like my current self, want things like take a break from work, get light physical exercise, socialize, look at nature for a while because I think it's psychologically healthy, or get a sense of accomplishment after having gotten up early and hiked all the way up. All of those are consequences, and I don't see what you'd want that isn't a consequence.

Whether consequentialism is impractical to think about everyday things is question I'd want to keep striclty separate from the philosophical component... but I don't see the impracticality in this example, either. When I debated going hiking this summer, I made a consequentialist cost-benefit analysis, however imperfectly.

This seems to be discounting the consequentialist value of short term pleasure seeking. Doing something because you enjoy the process has immediate positive consequences. Doing something because it is enriching for your life has both positive short term and long term consequences.

To discount short term pleasures as hedonism (as some might) is to miss the point of consequentialism (well of Utilitarianism at least), which is to increase well-being (which can be either short or long term). Well-being can only be measured (ultimately) in terms of pleasure and pain.

Though I agree consequentialism is necessarily incomplete as we don't have perfect predictive powers.

To use a physics analogy, utility often isn't a potential function over state of affairs, and for many depends on path taken.

However, state of affairs is but a projection; state of world also includes mind states, and you might be indifferent between any quantum paths to worlds involving same mind state (including memory, beliefs) for you. (As a matter of values, I am not indifferent between paths either; rather, I endorse some integrated-utility up to an unspecified pinning point in future.)

Broadly, consequentialism requires us to ignore many of the consequences of choosing consequentialism. And since that is what matters in consequentialism it is to that exact degree self-refuting. Other ethical systems like Deontology and Virtue Ethics are not self-refuting and thus should be preferred to the degree we can't prove similar fatal weaknesses. (Virtue Ethics is the most flexible system to consider, as you can simply include other systems as virtues! Considering the consequences is virtuous, just not the only virtue! Coming up with broadly applicable rules that you follow even when they aren't what you most prefer is a combination of honor and duty, both virtues.)