My Marriage Vows

post by Vanessa Kosoy (vanessa-kosoy) · 2021-07-21T10:48:24.443Z · LW · GW · 53 comments

I'm getting married. We decided to take marriage vows very seriously, and write vows that we will be fully committed to uphold. These vows are going to be a commitment no weaker than any promise I ever made or any contract I ever signed. Therefore, it is very important to avoid serious errors in their content.

I'm interested to hear feedback of the form "making these vows might turn out to be a big mistake for you, and here is why"[1] or of the form "here is how the spirit of these vows can be implemented better". Given that this is a community which nurtures security mindset, I have great expectations :) More precisely, I am less interested in extreme nitpicking / rule-lawyering, since that should be neutralized by the Vow of Good Faith anyway (but tell me if you think I'm wrong about this!) and more in serious problems that can arise in at least semi-realistic situations. (Of course, since many of us here expect a Singularity in a few decades, semi-realistic is not a very high bar ;)

Without further ado, the vows:

[EDIT 2022-07-15: The following text has been edited to match the final version of the Vows (that we took on 2021-08-29)]

I, [name], solemnly pledge to [name] three sacred Vows as I take [pronoun] to be my [spouse]. These vows are completely sincere, literal, binding and irrevocable from the moment both of us take the Vows, unless the marriage is dissolved or my [spouse] unconscionably[2] breaks [pronoun]’s own Vows which I believe in all likelihood will never happen. Let everyone present be my witness.

The First Vow is that of Honesty. I will never set out to deceive my [spouse] on purpose without [pronoun]’s unambiguous consent[3], without exception. I will also never withhold information that [pronoun] would in hindsight prefer to know[4]. The only exception to the latter is when this information was given to me in confidence by a third party as part of an agreement which was made in compliance with all Vows[5]. If for any reason I break my vow, I will act to repair the error as fast as reasonably possible.

The Second Vow is that of Concord. Everything I do will be according to the policy which is the Nash solution to the bargaining problem defined by my [spouse]’s and my own priors and utility functions, with the disagreement point set at the counterfactual in which we did not marry. I will act as if we made all precommitments that would a priori be beneficial from a Nash bargaining point of view[6]. If our utility functions change, we will effectively perform another Nash bargaining with the previous policy as the disagreement point. Moreover, if I deviate from this policy for any reason then I will return to optimal behavior as soon as possible, while preserving my [spouse]’s a priori expected utility if at all possible[7]. Finally, a hypothetical act of dissolving this marriage would also fall under the purview of this Vow[8].

The Third Vow is that of Good Faith, which augments and clarifies all three Vows. The spirit of the Vows takes precedence over the letter. When there’s some doubt or dispute as to how to interpret the Vows, the chosen interpretation should be that which my [spouse] and I would agree on at the time of our wedding, in the counterfactual in which the source of said doubt or dispute would be revealed to us and understood by us with all of its implications at that time as well as we understand it at the time it actually surfaced[9].


  1. Conditional on the assumption that my decision to marry is about as well-grounded as one can expect. I am not soliciting criticism of my choice of spouse! ↩︎

  2. Meaning that it's a grave or persistent violation rather than a minor lapse. ↩︎

  3. Consent is mentioned to allow us to e.g. play tabletop games where you're supposed to deceive each other. ↩︎

  4. That is, information X such that if the spouse knew X, they would believe it's good that they found out about it. This excludes information which is not important (knowing X is practically useless) and infohazards (knowing X is actively harmful). ↩︎

  5. If I enter an agreement with a third party in violation of the Vow of Concord, the Vow of Honesty takes precedence over the agreement and I might have to violate the latter and pay whatever fine is necessary. ↩︎

  6. We are taking an "updateless" perspective here. The disagreement point is fixed in the counterfactual in which we didn't marry in the first place, it does not move to the counterfactual of divorce. Notice also that marriage is guaranteed to be an a priori Pareto improvement over no-marriage because this is our current estimate, even if it turns out to be false a posteriori. ↩︎

  7. If the violation shifts the Pareto frontier such that the previous optimum is outside of it, the new Pareto optimum is chosen s.t. the violating party bears the cost. ↩︎

  8. This makes all of the Vows weightier than they otherwise would be. The Vows can be unmade by dissolving the marriage, but the act of dissolving the marriage is in itself subject to the Vow of Concord, which limits the ability to dissolve it unilaterally. ↩︎

  9. In other words, interpretation is according to the extrapolated volition of us at the time of our wedding, where the extrapolation is towards our knowledge and intellectual ability at the time of making the judgment. ↩︎

53 comments

Comments sorted by top scores.

comment by orthonormal · 2021-07-21T17:55:14.356Z · LW(p) · GW(p)

The Vows can be unmade by dissolving the marriage, but the act of dissolving the marriage is in itself subject to the Vow of Concord, which limits the ability to dissolve it unilaterally.

My ex and I included a more informal version of this in our own vows, and it was the only vow I ever broke. You cannot exclude from possibility a situation where the marriage is unhealthy, one spouse is suffering, and the other cannot bear the idea of letting go.

My ex was desperate to make things work, and I was trying with all my might, but there was no progress on the problems that blew up as soon as we moved in together. The first highly recommended couples therapist couldn't help us after over a year, then the next one threw up her hands and said she didn't think she could help any more.

Could I have convinced my ex to agree to a divorce while still living together? It seemed impossible to me - my guilt, and my fear of letting loved ones down, would drag me back.

(I'm in so much healthier a place, by the way, and my ex now seems to be happier as well. Our divorce was amicable after the first few weeks.)

I don't know what the more human-safe version of this clause would be, but it's not this. Unilateral exit should be a difficult option—to be done only in great emergencies or after a large amount of effort has been expended—but please don't take it off the table.

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-21T18:51:04.112Z · LW(p) · GW(p)

Thank you for sharing. I'm sorry it worked out so poorly for you!

It sounds like your situation was not at all Pareto efficient? If so, this Vow of Concord would not preclude you from divorce? Notice that the Vow does not say that both spouses must locally prefer divorce for divorce to happen. It only says that divorce must be part of the bargaining-optimal policy.

For example, consider the following scenario:

  • If we wouldn't get married, our payoffs would be .
  • With probability we will have a mutually beneficial marriage in which each has payoff .
  • With probability the marriage will be detrimental to me (payoff ) and beneficial to my spouse (payoff ).
  • With probability the marriage will be beneficial to me (payoff ) and detrimental (w.r.t counterfactual) to my spouse (payoff ).

(Here "beneficial" and "detrimental" are to be understood relatively to the counterfactual in which this marriage didn't happen.)

Then, the policy "never get divorced" has payoff vector whereas the policy "stay married iff the marriage is mutually beneficial" has payoff vector . The latter is Pareto dominant therefore the Vow of Concord will select the latter.

Now let's change the payoffs:

  • If we wouldn't get married, our payoffs would be .
  • With probability we will have a mutually beneficial marriage in which each has payoff .
  • With probability the marriage will be detrimental to me (payoff ) and beneficial to my spouse (payoff ).
  • With probability the marriage will be beneficial to me (payoff ) and detrimental to my spouse (payoff ).

Now "never get divorced" has payoff vector while "stay married iff the marriage is mutually beneficial" still has payoff vector . The Vow of Concord requires us to stay married. Without the Vow (or decision-theoretic ability to simulate the Vow), both of us would be worse off in expectation!

In practice, I think that miserable marriages are virtually guaranteed to land in the "get divorced" territory, since they tend to make both spouses miserable.

Replies from: orthonormal
comment by orthonormal · 2021-07-23T22:58:37.931Z · LW(p) · GW(p)

I have a hard time trusting any mere humans to think straight on the decision theory of divorce; the stakes are so high that emotions come to the fore.

There must be conditions, even conditions short of abuse, where unilateral exit is allowed regardless of whether the other thinks that is a mistake. The conditions are a safety valve for motivated thinking. They can be things like "if you're miserable, having more fights than intimacy, have tried couples therapy for at least 6 months, stayed apart for a month and felt better alone, then you can divorce if you want".

Obviously that would be clunky in the vows, so there may be a lower-entropy way of saying that this marriage has some unlikely conditions for exit as well as voice.

(If you don't have this, you risk "one spouse trying to convince the other, unwilling, spouse to accept a divorce", which is pretty damn bad.)

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-24T08:18:43.750Z · LW(p) · GW(p)

I had an extremely painful and emotional divorce myself, so I am aware. Although, I tend to reject the idea that emotions prevent you from thinking straight. I think that's a form of strategic self-deception.

Strictly speaking, the vows don't say all decisions must be unanimous (although if they aren't it becomes kinda tricky to define the bargaining solution). However, arguably, if both of us follow the vows and we have common knowledge about this, we should arrive at unanimous decisions[1]. This is the desirable state. On the other hand, it's also possible that one of us breaks the vows, or erroneously beliefs that the other broke the vows[2], in which case unilateral action might be consistent with the vows. So, there's no implication that unilateral divorce is completely forbidden.


  1. By Aumann agreement, but even if we have different priors so Aumann agreement doesn't apply, we should still be able to state those priors and compute the bargaining solution on that basis. ↩︎

  2. Further levels of recursion are ruled out if we assume that one is not allowed to dismiss the vows on account of an unconscionable violation by the other party without declaring this to the other party, which is probably a good clause to add. ↩︎

comment by philip_b (crabman) · 2021-07-21T11:29:08.603Z · LW(p) · GW(p)

Security mindset, here we go.

I will never set out to deceive my [spouse] on purpose without [pronoun]’s unambiguous consent, without exception.

Perhaps you two will realize that it's better for you if your partner deceives you in some class of situations for stupid-monkey-brain reasons. I guess you can patch the agreement then by consenting to such deception.

Another potential failure mode: you will think that you're not deceiving your partner in some area while actually deceiving them. According to "The Elephant in the Brain", this is probable. From having read the vows, I am unsure how they are supposed to treat such a situation. Also, maybe in such a situation your partner will start despising the vows thinking that it's unfair that you're better at deceiving-without-knowing-that-you're-deceiving than them and that because of these, the vows are better for you. Or it could be vice versa.

Also, for elephant-in-the-brain reasons, I think it would be good to make the agreement such that there would be some incentives to not act against your partner via unconscious elephant-in-the-brain mechanisms.

The only exception to the latter is when this information was given to me in confidence by a third party as part of an agreement which was made in compliance with all Vows

This might make third parties less likely to tell you important things, since you might tell your spouse.

I will also never withhold information that [pronoun] would in hindsight prefer to know.

What happens if someone learns about this agreement of yours and thus kidnaps and tortures your spouse until they tell the kidnapper all your passwords or something like that? Suppose your spouse didn't know your passwords before this event and now they're on the phone and asking for your passwords. Are you obligated to give them your passwords because right now they prefer to know? Or are you free form the obligation because before this actually happened, both you and your spouse would prefer that you don't give passwords?

Another thing, I think this whole agreement models you as a rational agent with a utility function. And it might become bad for you if you start thinking about yourself another way, e.g. if you start seeing yourself less agentic, less rational, with more monkey-brain bugs.

Replies from: vanessa-kosoy, mary-chernyshenko
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-21T12:17:29.541Z · LW(p) · GW(p)

Another potential failure mode: you will think that you're not deceiving your partner in some area while actually deceiving them. According to "The Elephant in the Brain", this is probable.

I believe that essentially the elephant is the agent making all decisions, so it's the elephant taking the vows and bearing full responsibility for upholding them. Self-deception is not a valid excuse for deception.

The only exception to the latter is when this information was given to me in confidence by a third party as part of an agreement which was made in compliance with all Vows

This might make third parties less likely to tell you important things, since you might tell your spouse.

I am confused. Which part makes it less likely? The part where an agreement that goes against the Vows is invalidated? This is true, but seems not specific to these vows: any time I enter an agreement with someone, they are relying on me not to have prior conflicting agreements, and I am liable to compensate them if it turns out that I did.

What happens if someone learns about this agreement of yours and thus kidnaps and tortures your spouse until they tell the kidnapper all your passwords or something like that?

This is an interesting point. I guess the correct interpretation of "would in hindsight prefer to know" is "would rank the policy of revealing the information above the policy of not revealing the information from an updateless perspective i.e. according to their a priori expected utility".

Another thing, I think this whole agreement models you as a rational agent with a utility function. And it might become bad for you if you start thinking about yourself another way, e.g. if you start seeing yourself less agentic, less rational, with more monkey-brain bugs.

Well, I do tend to model humans as more rational than the typical position of this community. However, I think that if it turns out that I am less agentic then the concept of "bad for me" becomes less coherent s.t. the effect more or less cancels out.

Replies from: TekhneMakre
comment by TekhneMakre · 2021-07-22T05:45:35.949Z · LW(p) · GW(p)

>I believe that essentially the elephant is the agent making all decisions, so it's the elephant taking the vows and bearing full responsibility for upholding them. Self-deception is not a valid excuse for deception.


So then, these vows could only be made if you have an extremely high level of already having untangled yourself / the elephant, such that it's even possible for you to not (self-)deceive. Are the vows assuming this? If not, maybe there should be a clause describing a derivative or trajectory, rather than a state. In other words, how sure are you that you / they aren't already deceiving each other about some important stuff?


>set out to deceive my [spouse] on purpose

Maybe you're saying "set out", meaning, once the marriage starts, there won't be any *new* deception. Hard to tell how the boundary is drawn, if a preexisting deep deception could spin up new shallow deceptions (without you explicitly noticing this, i.e. being in bad faith). What's "on purpose" doing here? It sort of sounds like "on purpose (...but if it's the elephant, not *me*, then it's less bad)", which I don't think you want to say?

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-22T20:30:24.299Z · LW(p) · GW(p)

So then, these vows could only be made if you have an extremely high level of already having untangled yourself / the elephant, such that it's even possible for you to not (self-)deceive.

I believe that it's always possible for you to not self-deceive. The only real agent is the "elephant". The conscious self is just a "mask" this agent wears, by choice. It can equally well choose to wear a different mask if that benefits it.

What's "on purpose" doing here?

I just mean that there is an intent to deceive, rather than an accidental miscommunication.

Replies from: TekhneMakre
comment by TekhneMakre · 2021-07-22T21:58:37.418Z · LW(p) · GW(p)

What I'm saying is, do you think that there's no ongoing deep hidden deception (or, situation that would call forth deception) in you or your spouse? I this seems possible to me, it's just that empirically it's very rare. I'm wondering if your vows are proofed against this possibility. Maybe you don't think the probability is high enough to worry about; maybe you think the vow ought to be nullified / broken if there is such deception; maybe by

If for any reason I break my vow, I will act to repair the error as fast as reasonably possible.

you mean to say, yes it was a breach to make this vow given that there was hidden deception, and you'll repair it. Maybe this is how vows are supposed to work--making them, knowing that there's a good chance they'll be partly broken, and then working to uphold them with the understanding that the good faith clause will keep the agreement intact--rather than trying to explicitly say what (/whether) there's circumstances in which the agreement is definitively not intact. IDK. I guess my worry is that hidden deceptions (that is, a deception that you're doing but aren't aware of, i.e. don't have clear access to with most of your mind) will adaptively keep themselves hidden if there's no clear recourse for keeping the agreement intact (including an amicable separation) when they become revealed.

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-23T10:55:23.002Z · LW(p) · GW(p)

We do have margin for minor violations of the vows, as long as they are not "unconscionable". Granted, we don't have a precise definition of "unconscionable", but certainly if both of us agree that a violation is not unconscionable then it isn't.

comment by Mary Chernyshenko (mary-chernyshenko) · 2021-07-23T18:35:17.428Z · LW(p) · GW(p)

...if someone kidnapped and tortured a loved one of yours and you didn't reveal your passwords, there'd have to be a rather important reason for your reticence, vow or no vow.

Replies from: Different
comment by Different · 2021-07-23T18:48:29.427Z · LW(p) · GW(p)
comment by dreeves · 2021-07-22T17:55:57.641Z · LW(p) · GW(p)

I love this so much and Bee (my spouse) and I have started talking about it. Our first question is whether you intend to merge your finances. We think you shouldn't! Because having separate finances means having transferrable utility which puts more powerful and efficient and fair decision/bargaining mechanisms at your disposal.

My next question is why the KS solution vs the Nash solution to the bargaining problem?

But also are you sure the Shapley value doesn't make more sense here? (There's a Hart & Mas-Colell paper that looks relevant.) Either way, this may be drastically simplifiable for the 2-player case.

Thanks so much for sharing this. It's so sweet and nerdy and heart-warming and wonderful! And congratulations!

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-22T21:17:06.311Z · LW(p) · GW(p)

Our first question is whether you intend to merge your finances.

We do, at least because I'm the only one who has income.

My next question is why the KS solution vs the Nash solution to the bargaining problem?

I'm actually not sure about this. Initially I favored KS because monotonicity seemed more natural than independence of irrelevant alternatives. But then I realized than in sequential decision making, IIA is important because it allows you to consistently optimize your policy on a certain branch of the decision tree even if you made suboptimal actions in the past. There seems to be a way of working around this violation of IIA in KS, but it makes KS look more like a hack.

But also are you sure the Shapley value doesn't make more sense here?

Shapley value is for distributing transferable values, like money. For general utility functions there's no easy way to convert 1 Vanessa-utilon to 1 Marcus-utilon.

Thanks so much for sharing this. It's so sweet and nerdy and heart-warming and wonderful! And congratulations!

Thank you <3

Replies from: dreeves
comment by dreeves · 2021-07-22T22:26:41.951Z · LW(p) · GW(p)

Ooh, this is exciting! We have real disagreements, I think!

It might all be prefaced on this: Rather than merge finances, include in your vows an agreement to, say, split all outside income 50/50. Or, maybe a bit more principled, explicitly pay your spouse for their contributions to the household.

One way or another, rectify whatever unfairness there is in the income disparity directly, with lump-sum payments. Then you have financial autonomy and can proceed with mechanisms and solution concepts that require transferrable utility!

Replies from: vanessa-kosoy, Different
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-23T10:51:09.986Z · LW(p) · GW(p)

Marcus has chronic illness. This means their contribution to the household can vary unpredictably, practically on any timescale. As a result, it's hard to think of any split that's not going to be skewed in one or other direction in some scenarios. Moreover, they are unable to hold to job, so their time doesn't have opportunity cost in a financial sense.

Replies from: dreeves, Different
comment by dreeves · 2021-07-24T00:10:20.245Z · LW(p) · GW(p)

Ah, super fair. Splitting any outside income 50/50 would still work, I think. But maybe that's not psychologically right in y'all's case, I don't know. For Bee and me, the ability to do pure utility transfers feels like powerful magic!

Me to Bee while hashing out a decision auction today that almost felt contentious, due to messy bifurcating options, but then wasn't:

I love you and care deeply about your utility function and if I want to X more than you want to Y then I vow to transfer to you U_you(Y)-U_you(X) of pure utility! [Our decision auction mechanism in fact guarantees that.]

Then we had a fun philosophical discussion about how much better this is than the hollywood concept of selfless love where you set your own utility function to all zeros in order for the other's utility function to dominate. (This falls apart, of course, because of symmetry. Both of us do that and where does that leave us?? With no hair, an ivory comb, no watch, and a gold watchband, is where!)

Replies from: dreeves, vanessa-kosoy
comment by dreeves · 2021-07-24T00:32:20.566Z · LW(p) · GW(p)

PS: Of course this was also prompted by us nerding out about your and Marcus's vows so thank you again for sharing this. I'm all heart-eyes every time I think about it!

comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-24T08:32:06.940Z · LW(p) · GW(p)

Ah, super fair. Splitting any outside income 50/50 would still work, I think.

I think that a 50/50 splits creates the wrong incentives, but I am reluctant to discuss this in public.

PS: Of course this was also prompted by us nerding out about your and Marcus's vows so thank you again for sharing this. I'm all heart-eyes every time I think about it!

<3

comment by Different · 2021-07-23T14:47:13.377Z · LW(p) · GW(p)
comment by Different · 2021-07-23T00:06:39.550Z · LW(p) · GW(p)
comment by Ericf · 2021-07-21T13:12:53.371Z · LW(p) · GW(p)

You and your spouse will each and together be different people 10 years from now. It will be impossible and undesirable to use "[the interpretation] which my [spouse] and I would agree on at the time of our wedding"

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-21T13:18:25.801Z · LW(p) · GW(p)

Why impossible or undesirable?

A related thing that came up in our discussion after I wrote this post is how to apply the Vow of Concord in the face of utility functions that change over time. The amendment we tentatively agreed on is: if the utility functions change, we do new KS bargaining where the disagreement point is the policy that resulted from the previous bargaining. This choice of disagreement point avoids perverse incentives to change your own utility function.

On a more pedestrian note, I was previously married for 10 years, so I'm not completely naïve in that regard.

Replies from: Ericf, ofer
comment by Ericf · 2021-07-22T01:35:06.510Z · LW(p) · GW(p)

Committing to a decision algorithm now implies that you expect to do worse in the future. Even though future you will have more information and experience. And, as you noted, potentially a different utility function. And, as a practical matter, are you even capable of making decisions as-if you were yourself in the past?

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-22T20:41:57.082Z · LW(p) · GW(p)

That's why I wrote "in the counterfactual in which the source of said doubt or dispute would be revealed to us and understood by us with all of its implications at that time as well as we understand it at the time it actually surfaced", so we do use the new information and experience.

The reason I want to anchor it to our present selves is because at present we are fairly aligned. We have pretty good common understanding of what we want form these vows. On the other hand, the appearance of a dispute in the future might be the result of us becoming unaligned. This would create the temptation for each of us to interpret the vows in their own favor. The wedding-time anchor mitigates that somewhat, because it requires us to argue with a straight face that our wedding-time-selves would endorse the given interpretation.

comment by Ofer (ofer) · 2021-07-26T14:40:19.597Z · LW(p) · GW(p)

A related thing that came up in our discussion after I wrote this post is how to apply the Vow of Concord in the face of utility functions that change over time.

That seems like a very important point. Also, you may end up living for more than a billion years (via future technology). The fraction of your future life in which your ~preferences/goal-system will be similar to your current ones may be extremely small.

comment by Richard_Ngo (ricraz) · 2021-07-21T22:26:05.155Z · LW(p) · GW(p)

I will also never withhold information that [pronoun] would in hindsight prefer to know

This phrasing seems a little strange. Both because the only thing you can actually commit to is never withholding information which you believe that [pronoun] will end up wishing they'd known; but also because the optimal communication policy has some non-zero rate of false negatives.

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-22T20:46:13.463Z · LW(p) · GW(p)

A more precise formulation would be: "when choosing what information to pass on, optimize solely for your best estimate of the spouse's utility function".

comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-07-21T12:03:02.269Z · LW(p) · GW(p)
Everything I do will be according to the policy which is the Kalai-Smorodinski solution to the bargaining problem defined by my [spouse]’s and my own priors and utility functions, with the disagreement point set at the counterfactual in which we did not marry. This policy is deemed to be determined a priori and not a posteriori. That is, it requires us to act as if we made all precommitments that would a priori be beneficial from a Kalai-Smorodinksi bargaining point of view[6] [LW · GW]. Moreover, if I deviate from this policy for any reason then I will return to optimal behavior as soon as possible, while preserving my [spouse]’s a priori expected utility if at all possible

Idk, I have a bad feeling about this, for reasons I attempted to articulate in this post. [AF · GW] The notion of optimal behavior you are using here may in fact be bad, and I question whether the benefits outweigh the costs. What are the benefits exactly? Why use all this specific, concrete decision theory jargon when you can just say "I promise to take my partner's interest (as they judge it, not as I judge it) into account to a significant extent" or something like that. Much more vague, but I think that's a feature not a bug since you have the good faith clause and since you are both nice people who presumably don't have really fucked up notions of good faith.

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-21T12:28:09.330Z · LW(p) · GW(p)

Idk, I have a bad feeling about this, for reasons I attempted to articulate in this post.

I'm not sure how commitment races are relevant here? We're not committing against each other here, we're just considering the set of all possible mutual commitments to compute the Pareto frontier. If you apply this principle to Chicken then the result is, flip a coin to determine who goes first and let them go first, there's no "throwing out the steering wheel" dynamics. Or, you mean commitment races between us and other agents? The intent here is making decision theoretic commitments towards each other, not necessarily committing to any decision theory towards the outside more than we normally would be.

Why use all this specific, concrete decision theory jargon when you can just say "I promise to take my partner's interest (as they judge it, not as I judge it) into account to a significant extent" or something like that.

Well, we could but this formal specification shows just how significant the extent is (which is very significant).

Replies from: daniel-kokotajlo
comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-07-21T14:37:38.683Z · LW(p) · GW(p)
Or, you mean commitment races between us and other agents? The intent here is making decision theoretic commitments towards each other, not necessarily committing to any decision theory towards the outside more than we normally would be.

Ah, good, that negates most of my concern. If you didn't already you should specify that this only applies to your actions and commitments "towards each other." This is an awkward source of vagueness perhaps, since many actions and commitments affect both your spouse and other entities in the world and thus are hard to classify.

Re: the usefulness of precision: Perhaps you could put a line at the end of the policy that says "We aren't actually committing to all that preceding stuff. However, we do commit to take each other's interests into account to a similar extent to the extent implied by the preceding text."

Replies from: daniel-kokotajlo, vanessa-kosoy
comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-07-21T14:40:59.978Z · LW(p) · GW(p)

Also: Congratulations by the way! I'm happy for you! Also, I think it's really cool that you are putting this much thought into your vows. :)

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-21T14:53:04.270Z · LW(p) · GW(p)

To phrase my intent more precisely: whatever the decision theory we will come to believe in[1] is, we vow to behave in a way which is the closest analogue in that decision theory of the formal specification we gave here in the framework of ordinary Bayesian sequential decision making.


  1. It is also possible we will disagree about decision theory. In that case, I guess we need to defer to whatever is the most concrete "metadecision theory" we can agree upon. ↩︎

Replies from: daniel-kokotajlo
comment by Daniel Kokotajlo (daniel-kokotajlo) · 2021-07-21T14:56:24.817Z · LW(p) · GW(p)

I like where you are going with this. One issue with that phrasing is that it may be hard to fulfill that vow, since you don't yet know what decision theory you will come to believe in.

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-21T14:58:26.200Z · LW(p) · GW(p)

Well, at any given moment we will use the best-guess decision theory we have at the time.

comment by ADifferentAnonymous · 2021-07-23T00:30:06.514Z · LW(p) · GW(p)

First of all, this is awesome.

I didn't know about KS bargaining before reading this, thinking through it now... 

It seems kind of odd that terrible solutions like (1000, -10^100) could determine the outcome (I realize they can't be the outcome, but still). I would hesitate to use KS bargaining unless I felt that  values were in some sense 'reasonable' outcomes. Do you have a general sense of what a life of maximizing your spouse's utility would look like (and vice versa)? 

Trying to imagine this myself wrt my own partner, figuring out my utility function is a little tricky. The issue is that I think I have some concern for fairness baked in. Like, do I want my partner to do 100% of chores? My reaction is to say 'no, that would be unfair, I don't want to be unfair'. But if you're referencing your utility function in a bargaining procedure to decide what 'fair' is, I don't think that works. So, would I want my partner to do 100% of chores if that were fair? I can simulate that by imagining she offered to do this temporarily as part of a trade or bet and asking myself if I'd consider that a better deal than, say,  her doing 75% of chores. And yes, yes I would. But I'd consider 'she does 100% of chores no matter what, I'm not allowed to help' a worse deal than 'she does 100% of chores unless it becomes too costly to her' for some definitions of 'too costly'.

Assuming that my utility function is like that about most things, and that hers is as well, I'd say our  values are actually reasonable counterfactuals to consider. Which inclines me to think yours are as well. 

Still, 'everything I do' is a big solution space to make assumptions about. The Vow of Concord pretty much requires you to look for edge cases where your spouse's utility can be increased by disproportionate sacrifices of yours; I'd suggest you start looking now (if you haven't yet), before you've Vowed to let them guide your decisions.

Replies from: vanessa-kosoy, Different
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-23T11:03:44.429Z · LW(p) · GW(p)

First of all, this is awesome.

Thank you :)

It seems kind of odd that terrible solutions like (1000, -10^100) could determine the outcome (I realize they can't be the outcome, but still).

I think you might be misunderstanding how KS works. The "best" values in KS are those that result when you optimize one player's payoff under the constraint that the second player's payoff is higher than the disagreement payoff. So, you completely ignore outcomes where one of us would be worse off in expectation than if we didn't marry.

Replies from: ADifferentAnonymous
comment by ADifferentAnonymous · 2021-07-23T12:04:01.682Z · LW(p) · GW(p)

The "best" values in KS are those that result when you optimize one player's payoff under the constraint that the second player's payoff is higher than the disagreement payoff.

I'm not sure this is the case? Wiki does say "It is assumed that the problem is nontrivial, i.e, the agreements in [the feasible set] are better for both parties than the disagreement", but this is ambiguous as to whether they mean some or all. Googling further, I see graphs like this where non-Pareto-improvement solutions visibly do count.

I agree that your version seems more reasonable, but I think you lose monotonicity over the set of all policies, because a weak improvement to player 1's payoffs could turn a (-1, 1000) point into a (0.1, 1000) point, make it able to affect the solution, and make the solution for player 1 worse. Though you'll still have monotonicity over the restricted set of policies.

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-23T14:39:36.637Z · LW(p) · GW(p)

In the original paper they have "Assumption 4" which clearly states they disregard solutions that don't dominate the disagreement point. But, you have a good point that when those solutions are taken into account, you don't really have monotonicity.

comment by Different · 2021-07-23T14:57:49.264Z · LW(p) · GW(p)
comment by Oz Agar (oz-agar) · 2021-07-24T06:01:15.522Z · LW(p) · GW(p)

I would recommend a revision (close to a re-write) of the second vow for four reasons:

First, negotiations theory has progressed past game theory solutions to a more psychologically based methodology.  This approach has been demonstrated to be more effective in two well tested and well studied environments: FBI hostage negotiations (I'd recommend starting with Never Split the Difference by Chris Voss as a light, entertaining but still useful introductory work), and intelligence community asset recruitment. You can look up the transition from the relatively explanatory MICE - money, ideology, compromise, ego - to RASCLS - reciprocity, authority, scarcity, consistency, liking, social proof.  Note that in RASCLS all approaches essentially link to ego, though they may use the other MICE factors to get there.  

Essentially, a lot of the time a dispute isn't about the outcome, but about the process and establishing understanding for respective points of view.  I understand that a less mathematically rigorous approach may at first seem less rational and less desirable.  But ultimately from a utilitarian viewpoint, it's most rational to choose the most effective method, even if that method is less "rational" than an alternate method.

Second, the second vow focuses too much on the target (a KS solution bargain to the disagreement) and too little on the process.  Agreeing on how you will dispute is more important for maintaining concord than how you determine the outcome.  Using a more specific procedure for arguments (especially serious ones) is useful, as it makes both parties know that they will have a fair opportunity to express themselves.  

I would recommend sourcing from debate standards at least for the most serious arguments, possibly recorded so you can both review afterwards (British parliamentary debate rules are quite good, and there are various shortened forms).  But even within the framework of searching for a KS solution, it's important to have a pre-established method of presenting possible solutions, establishing scores for how desirable those solutions are for each party, etc., without further escalating the disagreement.  Essentially some analogue to how facts and evidence can be established in a legal dispute.

Periodic (perhaps weekly) check ins to air grievances and make sure everyone is on the same page can also be useful.

Third, KS systems (like other game theory approaches) are difficult to quantify.  It's hard to assign a dollar value to taking out the trash versus doing the dishes.  Time is a possible approach, but difficult to reconcile with how much you like/dislike an activity.  Accompanying your partner for 30 minutes of listening to screaming metal on full blast may not be equivalent to 30 minutes of manual labor (in either direction).  Adding in a scale of love <-> hate (say, 0-10) that then multiplies by that time is similarly a poor approach because people may prefer a short sharp pain to a longer but milder pain, but absolutely avoid a very short agonizing pain.  Like most approaches that require a useful quantitative score, KS breaks down in the messy (and often inconsistent!) application of preferences.

Fourth, a KS/game-theory negotiations approach to disputes promotes an accounting approach to your relationship.  "You should do X because I did Y..." and so on.  This will provide a constant state of two parties keeping their own track of relative performance, which even without taking into account the likely bias towards their own viewpoints is likely to have fair divergence due to differences in accounting for various actions.  Though small, there will be countless and never ending disagreements and dissatisfaction due to this approach, providing for a grit which will slowly (or not so slowly) erode your relationship.

The exact wording is something you may want to work out with your partner.  It wouldn't be unreasonable to refer to some agreement between you on methods of dispute resolution which can be changed at a later date with both parties' agreement.  But I think you'd find this sort of approach more effective and more rewarding.

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-24T08:50:10.426Z · LW(p) · GW(p)

First, negotiations theory has progressed past game theory solutions to a more psychologically based methodology.

Hmm. Do you have a reference which is not, like, an entire book?

Second, the second vow focuses too much on the target (a KS solution bargain to the disagreement) and too little on the process.

Well, the process is important, but I feel like the discourse norms exemplified by this community already have us covered there, give or take.

Third, KS systems (like other game theory approaches) are difficult to quantify. It's hard to assign a dollar value to taking out the trash versus doing the dishes.

It's not dollar value, it's utilon value. I agree that quantification is challenging, but IMO it only reflects the complexity of the underlying reality that we have to deal with one way or the other. In principle, you can always quantify the utility function by asking enough questions of the form "do I prefer this lottery over outcomes to that lottery over outcomes".

Fourth, a KS/game-theory negotiations approach to disputes promotes an accounting approach to your relationship.

I think that all relationships are already accounting, people are just not always honest about it. Problems arise from people having different expectations / standards of fairness that they expect others to follow while never negotiating them explicitly. The latter is what we want to avoid here.

comment by Joe_Collman · 2021-07-21T23:57:11.927Z · LW(p) · GW(p)

Very interesting - and congratulations!

A few thoughts:
It strikes me that the first vow will sometimes conflict with the second. If your idea is that any conflict with the second vow would be a (mild) information hazard, then ok - but I'm not sure what the first vow adds in this case.

Have you considered going meta?:
"I make the set of vows determined by the Kalai-Smorodinski solution to the bargaining problem..."
"...I expect these vows to be something like [original vows go here] but the former description is definitive."

This has the nice upside of automatically catching problems you haven't considered, but not requiring you to be super-human. Specifically, the "Everything I do will be according to the policy..." clause just isn't achievable. Committing to the set of vows such a policy would have you make is achievable (you might not follow them perfectly, but there'd automatically be a balance between achievability and other desiderata).

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-22T20:50:34.551Z · LW(p) · GW(p)

Very interesting - and congratulations!

Thank you :)

It strikes me that the first vow will sometimes conflict with the second.

Well, yes, the intent it is that the Vow of Honest takes precedence over the Vow of Concord.

Have you considered going meta? "I make the set of vows determined by the Kalai-Smorodinski solution to the bargaining problem..."

I'm not sure what's the difference between "set of vows" and "policy"? When I say "policy" I refer to the set of behaviors we are actually capable of choosing from, including computational and other constraints.

Replies from: Joe_Collman
comment by Joe_Collman · 2021-07-23T01:06:24.811Z · LW(p) · GW(p)

Ah ok, if the honesty vow takes precedence. I still think it's a difficult one in edge cases, but I don't see effective resolutions that do better than using vows 2 and 3 to decide on those.

I'm not sure what's the difference between "set of vows" and "policy"? 

The point isn't in choosing "set of vows" over "policy", but rather in choosing "I make the set of vows..." over "Everything I do will be according to...". You're able to make the set of vows (albeit implicitly), and the vows themselves will have the optimal amount of wiggle-room, achievability, flexibility, emphasis on good faith... built in.

To say "Everything I do will be according to..." seems to set the bar unachievably high, since it just won't be true. You can aim in that direction, but your actions won't even usually be optimal w.r.t. that policy. (thoughts on trying-to-try [LW · GW] notwithstanding, I do think vows that are taken seriously should at least be realistically possible to achieve)

To put it another way, to get the "Everything I do..." formulation to be equivalent to the "I make the set of vows..." formulation, I think the former would need to be self-referential - i.e. something like "... according to the policy which is the KS solution... given its inclusion in this vow". That self-reference will insert the optimal degree of wiggle-room etc.

I think you need either the extra indirection or the self-reference (or I'm confused, which is always possible :)).

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-23T14:52:35.950Z · LW(p) · GW(p)

One way to interpret this is "I will do my best effort to follow the optimal policy". On the other hand, when you're optimizing for just your own utility function, one could argue that the "best effort" is exactly equal to the optimal policy once you take constraints and computational/logical uncertainty into account. On the third hand, perhaps for bargaining the case for identifying "best effort" and "optimal" is weaker. In practice, what's important is that even if you followed a suboptimal policy for a while, there's a well-defined way to return to optimal behavior. This is true for Nash bargaining (because of independence of irrelevant alternatives), less so for KS! Which is why I'm leaning towards switching to Nash. And if I fail to even make the best effort, there's the clause about how to amend.

comment by Geoffrey Wood (geoffrey-wood) · 2021-07-21T22:51:02.411Z · LW(p) · GW(p)

 "We decided to take marriage vows very seriously", I this this is good, however I think you are making a basic error in what this should mean you have in your vows. I think you are trying to write a specific contract, or a set of programming instructions.

We, as in humans, are poorly defined, barely conscious, irrational, lumps of meat. We are not aware of our own utility functions let alone those of others, especially as they change over time and chaotically in the course of a day. We are unable to follow a precise recipe like the one you have outlined.

I think the only thing you can communicate with your vows is the spirit of your words not their contractual meaning. That's why you often hear in others vows poorly defined meaningless things like "I feel you and I were destined to be together" or "I love you with all of my heart". The words are not the point, nobody remembers them anyway.

I also took my vows seriously, i simply tried to be a bit more concrete that the average set of vows.

I went with:

I vow to mow the lawns, vacuum the floor, clean the bathroom and if the mood strikes, do the dishes
I vow to always do my best to keep you happy, healthy in mind and body and well pampered
I vow to cook for you all sorts of delectable, interesting and marginally sub-lethal dishes
I vow to be there for you when you need me, and far away when you don't
I vow to always attempt to improve myself and help you grow in any direction you so desire
I vow to always give you something to laugh about, a reason to smile
I vow to think
I vow to help
I vow to listen
I vow to love you for the rest of your life

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-22T21:02:11.000Z · LW(p) · GW(p)

We, as in humans, are poorly defined, barely conscious, irrational, lumps of meat. We are not aware of our own utility functions let alone those of others, especially as they change over time and chaotically in the course of a day. We are unable to follow a precise recipe like the one you have outlined.

I'm not convinced. I have a rather favorable view of human agency and rationality compared to the distribution of opinions in this community, and I think it's not the place to hash out these differences. For our present purpose, just assume that we are able (or at least give concrete examples of where we will fail).

I think the only thing you can communicate with your vows is the spirit of your words not their contractual meaning. That's why you often hear in others vows poorly defined meaningless things like "I feel you and I were destined to be together" or "I love you with all of my heart".

I think that marriage is a contract, except that usually it's unwritten. Which often enough leads to disaster when the two spouses have different notions of what the contract entails. People prefer meaningless things over explicit contracts for signaling reasons, but we explicitly decided to pursue a different strategy.

Replies from: geoffrey-wood
comment by Geoffrey Wood (geoffrey-wood) · 2021-07-26T20:49:40.830Z · LW(p) · GW(p)

I agree that marriage is an unwritten contract, I mean you literally sign something as part of the ceremony that legally binds you in the eyes of the government after making serious promises in front of everyone you care about. In some ways this contract is already  agreed to well before the wedding, during the period you are dating, living together and sorting out what each other thinks about things.

Nevertheless due to my more pessimistic view of human agency, I wouldn't write it up, instead relying on constant good communication about each others feelings on things. (Not implying that this is a perfect recipe or that people should be blamed for being bad communicators if a relationship fails, sometimes there truly are irreconcilable differences).

I've been thinking about this over the last few days and I feel that the need to get it all nailed down in this manner could perhaps be coming from a place of insecurity? It might be an idea to address this separately? Id like to say that anyone who could write vows like the ones above with their partner is probably in an excellent place in their relationship.

In your other responses i saw that you are the primary breadwinner and that your beloved is at least partially dependent on you. This situation is similar to one of my friends and i don't think they have been handling it well, I think he doesn't realise the extent of the power imbalance this causes. His weakly held opinions have more impact on his wife than he realises and I think that her relative lack of ability to argue with him has caused them to make some poor decisions in the past. I know this is slightly off topic un-solicited advice, but It might be helpful to you to realise (on the off chance that you hadn't already thought about it in this way).

I wish you and your fiancé the best in your life together :)

comment by Gunnar_Zarncke · 2021-07-21T12:14:23.985Z · LW(p) · GW(p)

Your vows are very precise and specific and I think intended to be optimal in some sense. Which is cool. It may be that the real world with children and difficult times may make it difficult to even have time to think through all the vows at all times. I think you should have something that accounts for that. I'm not saying something like a buffer for errors but something that compensates for errors on the other side. I think the Vow of Good Faith is doing some work in that direction but it is focussing most on the intended purpose, not on the practicalities of the implementation. I think something like a Vow of Forgiving might be nice which heals any errors if the partner made remediation steps (this is hard to formalize though).

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-21T13:13:44.203Z · LW(p) · GW(p)

Well, this is bounded rationality: the optimization we're talking about is understood to be within the computational constraints of humans. As to including an explicit Vow of Forgiving, I am concerned it might be too exploitable.

comment by frontier64 · 2021-07-22T02:45:33.228Z · LW(p) · GW(p)

I think modeling yourselves as agents for the purpose of the vows is a good idea. It'll both reinforce agent-like behavior and form a stronger commitment between you and your spouse.

I have a couple of minor quibbles. For the Vow of Honesty I think you should keep the vow as it is in public, but privately commit to full honesty with your husband disregarding agreements with third parties. You should not be bound to keep a secret from your spouse even if it fits under the Vow of Concord and you were sworn by a third party. If you are committed to honesty it should be a full absolute commitment instead of a commitment with a very difficult to achieve exception. But third parties will be less likely to ever share information with you in confidence if you publicly commit to not ever keeping a secret from your spouse. Having a separate public/private vow of honesty gives you the best of both worlds.

I have 2 corrections for this line, "These vows are completely sincere, literal, binding and irrevocable from the moment both of us take the Vows and as long as we both live, or until the marriage is dissolved or until my [spouse]’s unconscionably[2] breaks [pronoun]’s own Vows..." Firstly I think that " 's " is a grammar mistake and it should just read "...or until my [spouse] breaks..." instead.

Also I think that out should be removed even if it's made grammatically correct. Allowing yourself to cancel following your vows because your spouse willfully stopped following theirs is a little dangerous. It leads to situations where you might rather justify your own breach of the vows by pointing to their breach instead of trying to make things right. This is an issue in contracts sometimes where one side wants to be able to prove the other committed a material breach so they have the insurance policy that they can cancel the contract whenever they want to. You would never want to be in a situation where you want your spouse to break their vows so you can feel ok breaking them yourself.

Replies from: vanessa-kosoy
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-22T21:07:05.504Z · LW(p) · GW(p)

Firstly I think that " 's " is a grammar mistake and it should just read "...or until my [spouse] breaks..." instead.

You're right, thanks!

Allowing yourself to cancel following your vows because your spouse willfully stopped following theirs is a little dangerous. It leads to situations where you might rather justify your own breach of the vows by pointing to their breach instead of trying to make things right.

I agree that it's a possible failure mode, but the alternative seems worse? Suppose that my spouse starts completely disregarding the vows and breaking them egregiously. Do you really think I should still follow my own vows to the letter?

Replies from: frontier64
comment by frontier64 · 2021-07-24T23:26:04.802Z · LW(p) · GW(p)

Yes I do think you should follow your vows to the letter even if your spouse is breaking them egregiously. I have strong feelings about this, but I'm not sure if I have a good explanation as to why. Its my general feeling that you really shouldn't be able to consider any sort of exit plan for a marriage. Of course you definitely do need an exit plan, but it shouldn't be something that you're aware of until it's necessary.

A marriage is different from a typical mutually beneficial contract. A marriage should partially realign the husband and wife's utility functions such that expected utility for one spouse counts for substantial expected utility to the other spouse. So unless your spouse is behaving so egregiously that you're losing enough expected utility from the marriage to put you below your disagreement point, violating your vows shouldn't come into play. But of course at that point you would be considering divorce anyway if you thought the situation couldn't be fixed while you remain in the marriage. I think that's the crux of it for me: if you don't have breaking your vows or divorce on the table you'll really try to fix whatever issues you have in the marriage (if there are issues) before you have to go nuclear.

As I've said I don't quite understand my own position in a straightforward sense so don't give it too much weight. I'm not sure if my explanation for why is really rational or just a rationalization.

Thanks for the post and congratulations!

comment by ThomasReiner · 2021-07-23T03:52:08.695Z · LW(p) · GW(p)

Let me first state, that this is quite inspiring!
Your application to probability is fascinating, and the vows in themselves are inspiring.
I will also like to share that I think about marriage quite a lot, due to my want to have a big family and a healthy family structure - but I cannot help but constantly return to this one statistical fact:

70% of the couples that marry these days (meaning, Millenials and Generation X) are subject to get a divorce within a decade. Moreover, if the couple are in their twenties, the time for the divorce is then reduced to 5 years instead of 10.

*I wanted to inquire, without judgement, how do you reconcile with this fact?

I find myself often talking to potential partners and other men whom are married, and most of the time, if they are 3 years in the marriage they report having struggles like any other couple yet I believe 3 years is just not long enough to reach any conclusion.

*(The art of love is indeed a subtle subject during the technological era, where our primal needs are stimulated constantly from the use of technologies. And I firmly believe that restricting all such technological stimuli can be beneficial to any relationship. Previous generations did not have WhatsApp, and so they missed each other truly. They did not have Instagram, and so they did not lust after things they do not have nor did they constantly focused on what is 'missing' in their life. They did not have facebook or Linkedin, and so they did not spend the time running the rat-race, chasing passions to prove to others a point.)

(Obviously, I generalize here, please bare with me.)

 

Replies from: vanessa-kosoy, Different
comment by Vanessa Kosoy (vanessa-kosoy) · 2021-07-23T15:00:54.902Z · LW(p) · GW(p)

Let me first state, that this is quite inspiring!

Thank you!

70% of the couples that marry these days (meaning, Millenials and Generation X) are subject to get a divorce within a decade...

I wanted to inquire, without judgement, how do you reconcile with this fact?

I am divorced myself, and my previous marriage lasted about a decade. Still, I don't know if there's much to reconcile. Obviously there is always risk that the marriage will fail. Equally obviously, staying without a primary lover forever is a worse alternative (for me).

Previous generations did not have WhatsApp, and so they missed each other truly. They did not have Instagram, and so they did not lust after things they do not have nor did they constantly focused on what is 'missing' in their life. They did not have facebook or Linkedin, and so they did not spend the time running the rat-race, chasing passions to prove to others a point.

Luckily I don't have Instagram or Facebook, and although I have a LinkedIn account, I don't engage with the content there. I do have WhatsApp but I'm skeptical that it's really so bad.

comment by Different · 2021-07-23T04:36:30.417Z · LW(p) · GW(p)