The Real Rules Have No Exceptions

That's something you see in movies, yes, but as I understand what Paul Scharre is saying, it's not something that's actually true. According to him, the laws of war "care about what you do, not who you are." If you are behaving in a soldierly fashion, you are a soldier, whether you are a young man, old man, woman, or child.

On a more depressing note one might look into the events in the Korean War "except for women and children" was not applied. The movie is called A Little Pond (it was available on Amazon Prime a year or so back not sure if it's there now though) about the events at Nogunri.

Now, the movie also depicts the more human side of a soldier when confronted directly with that act -- rather than the impersonal shapes from hundreds of meters away -- near the end of the movie.

I would also add, regarding the whole permissible versus exception, that I suspect it is even grayer than suggested. The 5 year old with a radio is hardly and less a part of the fighting force than the civilians providing all the logistics and production supporting any of the military actions. So where is that line?

I'm not sure the AI will do much worse or much better than those making the plans and issuing the orders far from the battle ground and not exposed to the bloodshed and human carnage.

I think my point is different, although I have to admit I don’t entirely grasp your objection to Nostalgebraist’s objection.

Oh, I don’t object to what nostalgebraist says! I think it’s entirely right. (Also, to be clear, his post was written some time before my comment, so it’s not in any way a response to the latter.)

I say only that despite what he says seemingly being a serious challenge to (or even contradiction of) my post, nonetheless the post’s thesis survives the challenge intact, if not unscathed—mostly because no alternative approach to mine deals with the challenge any better.

I guess a better example is: the various flavours of utilitarianism …

Actually… I think this is a much worse example—because, in fact, I think such difficulties are entirely fatal to utilitarianism! (In fact I think that utilitarianism’s inadequacy as a moral theory is overdetermined—that is, that there are several reasons to reject it, each one sufficient on its own—but the sorts of problems you mention are certainly among those reasons.)

But let me return to your original examples—physics and business. Having thought about the matter a bit, it now seems to me that the position you are arguing against, which you (by implication) ascribe to me, is somewhat of a strawman.

The sort of situation I am referring to is one where you have (a) a rule that is applicable to a given class of situations, and (b) some phenomenon by which exceptions to the rule [i.e., specific situations where you don’t follow the rule, but instead do something else] arise. The claim I am making at the end of the post is that (b) is not some unfathomable black box from which, unexpectedly and unpredictably, exceptional cases spring, but rather a comprehensible set of criteria; and that (a) and (b) together constitute the actual “rule”—which, by construction, lacks exceptions. (And then there is the additional claim that there’s a benefit to making all of this explicit, and basing your decisions on it; this is the primary subject of the post.)

Now, it seems to me (and please correct me if I’m wrong here) that you are misreading me in two ways.

Firstly, it seems as if you are reading me as saying that (a) and (b) actually should be, or are, not two separate things but actually just one thing (and perhaps even that this one thing is, or should be, a simple thing). But I’m not saying anything of the sort! For instance, you say:

In fact, astrophysicists would have no difficulty identifying when it’s the right time to stop using Newtonian gravity.

Well and good! This is entirely consistent with my point. Here the “actual rule” would be something like: “relativity, plus whatever criteria we use to determine when to use Newtonian physics instead”. Clearly, this rule has no exceptions! (And if it does, well, whence those exceptions? How do physicists decide those are exceptions? However they did, whatever criteria they used—into the rule they go…)

Secondly, the situations I am referring to are, as I said, those where you have a rule that’s applicable to a given class of situations. By this I mean that you have some rule that tells you precisely what to do, but sometimes instead of doing that thing, you do (or, at least, are tempted to do) a different thing (i.e., you sometimes encounter [potentially] exceptional cases).

For example, if you have the rule “don’t eat cookies”, and you encounter a cookie, your rule is very clear on what you are to do: don’t eat the cookie. There’s no ambiguity here, no confusion or uncertainty. Should you eat this cookie? The rule says: no. You should not eat the cookie. End of story. That you are sometimes tempted to ignore, a.k.a. break, the rule, does not change the fact that the rule unambiguously dictates your actions. (The question, then, is why you’re tempted to make the exception, and exactly in what sorts of cases, etc.)

But note that this is not the case in your examples! If the rule, supposedly, is “use the Navier-Stokes equation”, but that equation is, in practice, impossible to calculate, then the rule doesn’t actually dictate your actions! It’s not that you know exactly what the answer is but you are unwilling to accept it; you just don’t have the answer! The supposed “rule” isn’t really any such thing. And in business it’s even worse: yes, “just turn a huge profit”, but what actually do I do? Specifically? I don’t know! I’m not tempted to break the rule, not at all; actually, I’d love to follow it, if only I knew how… but I don’t have any idea how! So, I have to use something other than this purported “rule”, in order to decide what to do.

The question of formalization (a.k.a. “what does ‘expect’ mean in a technical sense”) is a good one; I don’t have an answer for you. (As I said, the idea which I have in mind is like the idea of “conservation of expected evidence”, but, as you say, it’s not quite the same thing.) My mathematical skills do not suffice to provide any technical characterization of the term as I am using it.

It seems to me that the informal sense of the word suffices here; a formalization would be useful, no doubt (and if someone can construct one, more power to them)… but I do not see that the lack of one seriously undermines the concept’s validity or applicability.

In particular, your list of examples is composed almost entirely of cases which quite miss the point. Before going through them, though, I’ll note two things:

First—the purpose of the exercise is to construct more effective rules with which to govern our own behavior (as individuals), and the behavior of groups or organizations in which we participate. This general goal is often threatened by the existence of so-called “exceptions” to our ostensible rules, which can easily turn some apparently clear and straightforward rule against itself, and against the ostensible intent of the rule’s formulator(s). My aim in the OP is to provide a conceptual tool that counteracts this threat, by pointing out that the existence of “exceptions” is, in fact, a sign that there actually exists some real rule which is not identical to the stated rule (and which is the generator for the “exceptions”).

Second—the point I make in the OP is twofold: descriptive and prescriptive. The descriptive component is “the real rules have no exceptions”. The prescriptive component is “here is how you ought to deal with encountered apparent ‘legitimate exceptions’”. You seem to be objecting, here, to the prescriptive component. I do not think your objection holds (as I’ll try to demonstrate shortly), but note that even if you continue to find my prescription unconvincing, nevertheless the description remains true! There is some underlying pattern which is generating “legitimate exceptions”, and it will continue, unseen, to govern your behavior (and to undermine the predictability thereof)… unless you identify it, and either integrate or alter it.

We’ll do well to remember these two points as we consider the examples you offer. You propose that the following seem like potentially expectable exceptions to the “No Cookies” rule:

1. except if someone invents a cookie that’s good for my health

Once again, recall that the point of the rule in the first place is to effectively govern your own behavior. The difficulty, after all, is what? It’s that you know that you shouldn’t eat cookies all the time (or perhaps, almost ever), but you also know that without some device with which to restrain yourself, you’ll eat lots of cookies, because they’re delicious. (We can express this in terms of first- and second-order desires, or “goals” vs. “urges”, or some framework along such lines, but I think that the point here ought to be simple enough in any case.) A No Cookies rule is such a device. Its purpose is to enforce upon yourself some rule which you wish enforced upon yourself, in the service of achieving, and maintaining, some goal of yours.

Now, what happens when you encounter the best cookies in the state, and they seem to you to be a legitimate exception to your No Cookies rule? Roughly, what you have discovered thereby is that in addition to your goal of maintaining your health, you also have some other goal(s), which compete with it (such as, perhaps, “avoid turning life into a joyless existence, devoid entirely of sensory pleasures”, or “don’t let rare experiences pass you by, as they are precious and enriching”). Any explicit rule meant to govern the given class of situations, which purports to embody your goals and preferences, must capture this competing goal, along with the “maintain health” goal.

But under this view, the quoted example of a purported exception isn’t any such thing after all! The purpose of the No Cookies rule was to stop yourself from eating lots of cookies and thereby harming your health in the pursuit of momentary pleasures… but this hypothetical “health cookie” doesn’t interfere at all with the “health maintenance” goal, and is entirely consonant with the purpose of the existing rule. If you like, you can say that we take “cookies” to be a stand-in for “delicious but unhealthy sweets”—and “health cookies” don’t fit the bill. (Indeed, such a broad interpretation is needed anyway, as otherwise we would have the absurd situation of abjuring cookies but gorging on brownies—thus utterly ruining the purpose of the rule—and having to engage in philosophical debates about whether “bar cookies” are cookies or a distinct culinary product called bars, etc.)

2. except if someone points a gun at my head and orders me to eat a cookie

Well, first of all, should you encounter such a conundrum, you really have bigger problems than how best to formulate a rule governing your dietary practices.

Nothing I wrote in the OP (indeed, you may assume, nothing I ever write) is intended to replace common sense. I am not Eliezer; I do not write with the ultimate aim of applying my points to AI design. My prescription is meant for people—not for robots.

That having been said, there is, in fact, a non-ad-hoc way of handling just such cases; one prominent example of such an approach is seen in Jewish religious law, in the concept of pikuach nefesh. Briefly, the point is that there is no need to write into every rule a clause to the effect that “this rule shall be suspended if someone’s pointing a gun at my head”; instead, you have a general rule that if your life’s in danger, almost all other rules are suspended. Whatever goals and purposes your rules serve, they’re not so important as to be worth your life. (This doesn’t apply to all rules, just most of them… but certainly that “most” includes dietary restrictions.)

3. except if a doctor prescribes cookies because of some medical or psychiatric reason

Essentially the same response applies as that for example #1. If the cookies in question are, in fact, necessary to maintain your health, then eating them serves the goal for which the No Cookies rule was formulated. There is no question, here, of whether this is a “legitimate exception”; no uncertainty, no temptation.

Again, remember that the No Cookies rule is made by you, to serve your goals, to guard those goals against your impulses and your weaknesses. Consider again the notion of “legitimate exceptions”. We have already covered the meaning of legitimate exceptions (they are manifestations of underlying intuitions which serve competing goals), but what about illegitimate exceptions? The possibility of such is implied, isn’t it? But what are they? Well, they’re the manifestations, not of competing goals, but of precisely the impulses or urges which the rule is aimed at restraining in the first place! The question of “legitimacy” of an exception is, then, the question: “I have an intuition that I ought to except this situation from application of the rule, but does that intuition spring from a competing goal which I endorse, or does it spring from the desire I am trying to restrain?”

But in the given example, the question does not arise, because the exception is not generated by your intuition, but by an entirely endogenous factor: your doctor. (And, it must be noted, the question of how the health-related goal of your No Cookies rule stacks up to whatever medical reason your doctor has for prescribing you cookies, can, and should, be discussed with your doctor!)

4. except if I’m in a social situation where not eating a cookie is a serious faux pas (e.g., it will seriously offend the person offering me a cookie)

(Skipping this one for now; see below.)

5. except if I’m diagnosed with a terminal disease so I have no reason to care about my long term health anymore

Well, then you can drop the No Cookies rule entirely, and need no longer worry about what does, or does not, constitute an exception to it.

6. except if I’m presented with convincing evidence that I’m living in a simulation and eating cookies has no real negative consequences

The same response applies as to example #5.

Now, let’s return to the example I skipped:

4. except if I’m in a social situation where not eating a cookie is a serious faux pas (e.g., it will seriously offend the person offering me a cookie)

Ah! Now, here we have a genuine difficulty—and it is precisely the sort of difficulty which the concept I describe in the OP is intended to handle.

First, a note. In my post and my comments, I have talked about “encountering” various situations (and, relatedly, “expecting” to encounter them). Yet as you demonstrate, one can imagine encountering all sorts of situations, before ever actually encountering them.

Well, and what is the problem with that? This, it seems to me, is a feature, not a bug. Surely it’s a good thing, and not at all a bad thing, to think through the implications of your rules, and to consider how they may be applied in this or that situation you might run into. Suppose, after all, that you run into such a social situation (where refusing an offered cookie is a faux pas), having never before considered the possibility of doing so. You are likely to experience some indecision; you may act in a way you will later come to regret; you will, in short, handle the situation more poorly than you might’ve, had you instead given the matter some thought in advance.

You may think of this, if you like, of “encountering” the situation in your mind, which (assuming that your imagined scenario contains no gross distortions of the likely reality) may stand in for encountering the situation in fact. If the imagined scenario contains an apparently legitimate exception to your rule, you can then apply the same approach I describe in my post (i.e., analyze the generator of the exception, then either integrate the exception by updating the rule, or keep the rule and judge the exception to be illegitimate after all).

(Of course, such things shouldn’t be overdone. It’s no good to be paralyzed into anxiety by the constant contemplation of all possible situations you may ever encounter. But this problem is, I think, beyond the scope of this discussion.)

Now, to the specific example. You have, we have said, a rule: No Cookies. But you find yourself in some social situation where applying this rule has negative social consequences. This would seem to be one of those legitimate exceptions. And why is this? Well, we may suppose that you’ve got (as most people have) a general goal along the lines of “maintain good social standing”; or, perhaps, the operative goal is something more like “maintain a good relationship with this specific individual”.

The question before you, then, is how to weigh this social goal of yours against the health goal served by the No Cookies rule. That is something you (that is, our hypothetical person with the No Cookies rule) must answer for yourself; there is no a priori correct answer. In some cases, for some people, the social goal overrides the health goal. But for others, the health goal takes precedence. In such a case, it is a very good idea to have considered such situations in advance, and to have decided, in advance, to stand firm—to reject, in other words, the intuitive judgment of the exception’s legitimacy, having analyzed it and given due consideration to its source (i.e., the goal of maintaining social status or a personal relationship).

Such advance consideration is valuable not only because it saves you from making on-the-spot decisions you would later regret, but also because it allows you to take steps to mitigate the effects of choosing one way or the other—to turn an “either way, I lose something important” situation into a win-win.

Take the case of a No Cookies rule which is challenged by the refusal of an offered cookie being a social faux pas. Suppose you decide that in such a case (or in a specific such case), you will give precedence to your social goal(s), and eat the cookie. What steps might you take to mitigate the effects of this? For one, you might consider the impact of this violation of your No Cookies rule on the goal the rule serves, and compensate by reducing your sugar intake for the day / week / month. Alternatively, you might anticipate the possibility of entirely ruining your diet by frequent encounters of such socially challenging cookie-related situations, and proactively ensure that you only rarely find yourself at cookie-tasting parties (or whatever).

Conversely, suppose you decided that in such a case (or in a specific such case), you will give precedence to your health goal(s), and refuse the cookie. What steps might you take to mitigate the seriousness of the faux pas? Well, you might warn your cookie-offering acquaintance in advance that you are on a No Cookies diet, apologize in advance for refusing their offer of a cookie, and assure them (and solicit credible witnesses to bolster your assurance) that your refusal isn’t a judgment on their cookie-baking skills, but rather is forced by your dietary needs.

I could just go to full consequentialism and say the real rule is “no cookies except if the benefits of eating a cookie outweigh the costs” but presumably that’s not the point of this post?

The point, as I say above, is to provide a conceptual tool with which to better govern your own behavior, and that of organizations and groups in which you participate. Consequentialism is very well and good, and I have no quarrel with it; but act consequentialism is impractical (for humans). Consider my post to be a suggestion for a certain sort of rule-consequentialist “implementation detail” for your consequentialist principles.

It’s hard to give examples, because you don’t actually explain the specific function of not stating the “real rule” in any particular case.

By no means is it hard to give examples. Indeed, I did give several examples in an earlier comment [LW(p) · GW(p)].

As for reasons to keep the real rule unstated, they seem clear enough to me. I did not state them because I considered them too obvious to belabor… of course, it’s possible that I was wrong about this!

I can make my views of this explicit, if you like, but I don’t think I will be adding much to the understanding of signaling already common in this forum. In fact, I wonder if anyone else (perhaps one of the folks who liked or benefited from this post) would like to try their hand at explaining this? It would give me useful info about whether readers of this post understood it as I intended it to be understood (and would help to clarify the post for anyone confused, of course).

I feel like I use act consequentialism all the time, for example when deciding what restaurant to go to for dinner …

Really? When deciding what restaurant to go to for dinner, you examine all possible consequences of all the choices at your disposal (and the probability distributions across them), evaluate or rank them all, and select one? You don’t use any rules at all?

… which can’t be made into a rule since it depends on so many variables like where I am located on a particular day, what the weather is like, and what foods I’ve eaten recently

Why do you think that there needs to be a rule, instead of, say, multiple rules (some combination of which may bear on any given situation)? And why can’t rules depend on variables? (Or contain heuristics, etc.?)

… I’m guessing the neural networks implementing my intuitions have been trained to do something like [expected utility computations]

This seems stupendously unlikely. My reason for thinking otherwise is that this just isn’t consistent with anything we know about how people make decisions.

Because the choices I make usually respond to changing circumstances in a way that seems consistent with doing something like EU maximization.

Are you just saying that the preferences revealed in your choices conform to the VNM axioms (or some other formalism—if so, which?)? (If you are, then you know that this implies nothing at all about whether your brain is actually doing any expected utility computations.) Or are you making some stronger claim? If so, what is it?

Requires that we already had the criteria for deciding if the as yet unknown exception arising from the know information was in fact well formed and able to deal with the unknown information.

No. It does not. Nothing that I wrote requires this.

There are other ways of building in flexibility. For instance:

In cases many where the environment (causes) is very unpredictable, it is still possible to establish guidelines with reference to effects.

At the same time, the “rule” can explicitly state criteria for turning off the intervention, thereby reducing the risk that the intervention become a new normal.

Types of interventions can also be limited to a pre-existing list of alternatives, which can be criticised and vetted before the emergency is triggered.

Yes, all of these things can be done. And if you do any or all of them, and do not have a provision for applying judgment that stands above any of these provisions, your system will be exploitable. This is nostalgebraist’s point.

In practice, there will always exist “no, actually, we’re not following that rule in this case” exceptions. Our options are as follows:

1. Pretend strenuously that no exceptions exist. Apply the rules as inflexibly as possible, even (in fact, especially) in cases where it really seems like we shouldn’t apply them, to maintain the illusion that no exceptions exist.

2. Admit the fact that exceptions exist; attempt to make explicit, clarify, and rationalize the criteria for exception-making (in other words, fold them into the explicit rules); nevertheless maintain the option to exercise judgment, in contravention to the rules.

Crucially, in case #1, there will still be exceptions. They will be “snuck in” via creative interpretation of the rules, or via expansion or alteration of the rule set with rules that are bad rules (and exist only to allow a certain class of cases which the rules would otherwise forbid), or via manipulations outside the rule system that make the rule-based decision itself irrelevant, or in any number of other ways. Look to our criminal justice system for examples, and our political system also; and any number of other systems of rules.

Note three things:

First, that nostalgebraist’s advice does not in any way prevent you from formulating guidelines, and attempting to minimize and to circumscribe the scope of your “extralegal” judgments. In fact, you should do this—especially if you find yourself making such “extralegal” exceptions often! You should seek a pattern in them, and see if they (or at least some of them) can be made into a rule; or, perhaps, if the existing rules need to be altered.

Second, the “fail-safe” clause is not meant to be used only by one “side” or party in some relationship—quite the opposite! Consider what nostalgebraist says:

Of course, this option can itself be abused, particularly if it is used freely and unthinkingly. If the rules hold only until the moment you don’t like their consequences, then the rules don’t really hold at all.

But if someone does that, you can take the same option yourself: “yeah, I said you should have this option, but using it like that is wrong.” Because the option does not have specific rules (by construction), no one can use rules lawyering to take it away from you. If someone uses “calling bullshit” for bad ends, you can just call bullshit on them.

If someone is using “sometimes I will just use my judgment” to be a capricious, unpredictable dictator, then you say: “you are acting improperly”. By construction, it will do the would-be tyrant no good to say “ah, but we agreed that I can use my judgment whenever I see fit, so your objection is invalid”!

Third, you say:

In the realm of psychology and politics, rules gain legitimacy when they are adhered to over a long period of time and when they are seen to consistently protect against bad outcomes.

There is a case for flexible interpretation, but an agent who abandons rules too frequently, and with slight incentive will eventually lose confidence in his ability to abide by rules.

Here you are speaking first about legitimacy—how rules meant to bind others are perceived by the public—and then, apparently, about self-control and will. These are not the same thing, and it does no good to discuss them both in the same breath.