Seeking Power is Instrumentally Convergent in MDPs 2019-12-05T02:33:34.321Z · score: 109 (32 votes)
"Mild Hallucination" Test 2019-10-10T17:57:42.471Z · score: 9 (4 votes)
Finding Cruxes 2019-09-20T23:54:47.532Z · score: 17 (4 votes)
False Dilemmas w/ exercises 2019-09-17T22:35:33.882Z · score: 17 (4 votes)
Category Qualifications (w/ exercises) 2019-09-15T16:28:53.149Z · score: 24 (9 votes)
Proving Too Much (w/ exercises) 2019-09-15T02:28:51.812Z · score: 12 (7 votes)
Arguing Well Sequence 2019-09-15T02:01:30.976Z · score: 14 (3 votes)
Trauma, Meditation, and a Cool Scar 2019-08-06T16:17:39.912Z · score: 83 (35 votes)
Kissing Scars 2019-05-09T16:00:59.596Z · score: 45 (20 votes)
Towards a Quieter Life 2019-04-07T18:28:15.225Z · score: 17 (11 votes)
Modelling Model Comparisons 2019-04-04T17:26:45.565Z · score: 12 (3 votes)
Formalizing Ideal Generalization 2018-10-29T19:46:59.355Z · score: 3 (3 votes)
Saving the world in 80 days: Epilogue 2018-07-28T17:04:25.998Z · score: 56 (23 votes)
Today a Tragedy 2018-06-13T01:58:05.056Z · score: 58 (22 votes)
Trajectory 2018-06-02T18:29:06.023Z · score: 18 (5 votes)
Gaining Approval: Insights From "How To Prove It" 2018-05-13T18:34:54.891Z · score: 21 (8 votes)
Saving the world in 80 days: Prologue 2018-05-09T21:16:03.875Z · score: 34 (10 votes)
Mental TAPs 2018-02-08T17:26:36.774Z · score: 29 (8 votes)


Comment by elriggs on Does equanimity prevent negative utility? · 2020-06-17T02:56:11.637Z · score: 1 (1 votes) · LW · GW

Based on your comment on Ricraz's answer, "something that is bad for me", I will make a guess at what you mean. Let me know if it answers your question.

Objectively (outside-perspective):

"Bad" requires defining. Define the utility function, and the answer falls out.

Depending on your goals and the context of being hurt, it might be negative, positive, or a mix of both! (ex. being unintentionally burned while cooking, being a masochist, and being burned to protect a clumsy loved one, respectively)


If you mean negative utility as in the negative valence of an observation, then I would argue that negative valence is a signal telling you how well you're achieving a goal. (this is from Kaj's Non-mystical sequence)

From a multi-agent view, you may have an agent giving you valence on how well you're doing at a goal (say a video game). If you're really invested in the game, you might fuse with that sub-agent (identify that with a "self" tag), and suffer when you fail at the game. If you're separated from the game, you can still receive information about how well you're doing, but you don't suffer.

The more equanimity you have (you're okay with things as they are), the less you personally suffer. Though you can still be aware of the negative/positive signal of valence.

Comment by elriggs on Today a Tragedy · 2020-06-13T01:43:53.703Z · score: 12 (3 votes) · LW · GW

I think I've convinced my girlfriend that it's okay for me to be sad because of what happened to you. She used to try to cheer me up, but I would tell her that it's okay for me to be sad. It sucks and it's okay if I act like it sucks.

I had honestly thought the day was July 15th. Then I saw my calendar and saw that it was today. As soon as I noticed, I started watching youtube, I guess to distract myself. When I stopped, it all just weighed on me again.

It's hard to accept your death. You had your goals and friends and all of your expectations, and then it just ended.

Your sister still posts for you on Facebook. Your friends still think of you.

I remember you. I miss you man

Comment by elriggs on Corrigibility as outside view · 2020-05-12T17:27:01.006Z · score: 3 (2 votes) · LW · GW

Okay, the outside view analogy makes sense. If I were to explain it to me, I would say:

Locally, an action may seem good, but looking at the outside view, drawing information from similar instances of my past or other people like me, that same action may seem bad.

In the same way, an agent can access the outside view to see if it’s action is good by drawing on similar instances. But how does it get this outside view information? Assuming the agent has a model of human interactions and a list of “possible values for humans”, it can simulate different people with different values to see how well it learned their values by the time it’s considering a specific action.

Considering the action “disable the off-switch”. It simulates itself interacting with Bob who values long walks on the beach. By the time it considers the disable action, it can check it’s simulated self’s prediction of Bob’s value. If the prediction is “Bob likes long walks on the beach”, then that’s an update towards doing the disable action. If it’s a different prediction, that’s an update against the disable action.

Repeat 100 times for different people with different values and you’ll have a better understanding of which actions are safe or not. (I think a picture of a double-thought bubble like the one in this post would help explain this specific example.)

Comment by elriggs on Meditation: the screen-and-watcher model of the human mind, and how to use it · 2020-05-03T02:06:00.800Z · score: 2 (2 votes) · LW · GW

I pattern match this to the Buddhist idea of interdependence, where what you are is reliant on the environment and the environment is reliant on you (or embedded agency).

Comment by elriggs on My experience with the "rationalist uncanny valley" · 2020-04-24T02:40:38.855Z · score: 3 (2 votes) · LW · GW

If I understand you right, you value some things (finding them meaningful) because you robustly value them regardless of circumstances (like I value human life regardless of whether I had coffee this morning). Is this correct?

But you also mentioned that this only accounts for some values, and other things you value and find meaningful aren’t robust?

Comment by elriggs on Today a Tragedy · 2020-04-11T00:03:46.650Z · score: 12 (3 votes) · LW · GW

Happy Birthday Will,

I remember in 9th grade you started dating my ex right after consoling me. I was so mad! Haha. I never told you this, but me and the others on our forensics team saw y’all just sitting, holding hands, and having a good time, and Jennifer suggested that me and her hold hands and sit next to y’all giggling.

I said no, though it would’ve made a better story if I went through with it, haha. I think we started getting along again after she moved, although I can’t remember saying anything mean to you because of it.

I’m not sure she knows what happened to ya. I know y’all kept in touch when she moved, and maybe she checks Facebook more than I do.

Anyways, a lot of us are back home cause of the Coronavirus, and I would love to be able to give you a call and see how your life’s progressed these past few years.

Love you Will,


Comment by elriggs on Link Retrospective for 2020 Q1 · 2020-04-10T01:55:46.637Z · score: 5 (3 votes) · LW · GW

Thanks for the links and I hope you post another next quarter!

Comment by elriggs on "No evidence" as a Valley of Bad Rationality · 2020-04-01T21:20:41.390Z · score: 3 (2 votes) · LW · GW

Correct, favoring hypothesis H or NOT H simply because you label one "null hypothesis" are both bad. Equally bad when you don't have evidence either way.

In this case, intuition favors "more chemo should kill more cancer cells", and intuition counts as some evidence. The doctor ignores intuition (which is the only evidence we have here) and favors the opposite hypothesis because it's labeled "null hypothesis".

Comment by elriggs on Attainable Utility Preservation: Scaling to Superhuman · 2020-02-27T18:26:33.926Z · score: 1 (1 votes) · LW · GW

Thanks for the link (and the excellent write-up of the problem)!

Regarding the setting, how would the agent gain the ability to create a sub-agent, roll a rock, or limit it's own abilities initially? Throughout AUP, you normally start with a high penalty for acquiring power, and then you scale it down to reach reasonable, non-catastrophic plans, but your post begins with having higher power.

I don't think AUP prevents abuse of power you have currently have (?), but prevents gaining that power in the first place.

Comment by elriggs on Attainable Utility Preservation: Scaling to Superhuman · 2020-02-27T12:46:15.577Z · score: 1 (1 votes) · LW · GW

I expect AUP to fail in embedded agency problems (which I interpret the subagent problem to be included). Do you expect it to fail in other areas?

Comment by elriggs on Firming Up Not-Lying Around Its Edge-Cases Is Less Broadly Useful Than One Might Initially Think · 2020-01-13T08:20:43.566Z · score: 1 (1 votes) · LW · GW

I realized afterwards that only “not sharing others secrets” is an example of “it’s ethical to lie if someone asks a direct question”. The other two were more “don’t go out of your way to tell the whole truth in this situation (but wait for a better situation)”

I do believe my ethics is composed of wanting what’s “best” for others and truthful communication is just an instrumental goal.

If I had to blatantly lie every day, so that all my loved ones could be perfectly healthy and feel great, I would lie every day.

I don’t think anyone would terminally value honesty (in any of it’s forms).

Comment by elriggs on Firming Up Not-Lying Around Its Edge-Cases Is Less Broadly Useful Than One Might Initially Think · 2020-01-13T07:47:58.526Z · score: 1 (1 votes) · LW · GW

Thanks for the clarification.

For me the answer is no, I don’t believe it’s ethically mandatory to share all information I know to everyone if they happen to ask the right question. I can’t give a complete formalization of why, but three specific situations are 1) keeping someone else’s information secret & 2) when I predict the other person will assume harmful implications that aren’t true &3) when the other person isn’t in the right mind to hear the true information.

Ex for #3: you would like your husband to change more diapers and help clean up a little more before they leave work every day, but you just thought of it right when he came home from a long work day. It would be better to wait to give a criticism when you’re sure they’re in a good mood.

An example for #2: I had a friend have positive thoughts towards a girl that wasn’t his girlfriend. He was confused about this and TOLD HIS GIRLFRIEND WHEN THEY WERE DATING LONG DISTANCE. The two girls have had an estranged relationship for years since.

If I was my friend, I would understand that positive thoughts towards a pretty girl my age doesn’t imply that I am required to romantically engage them. Telling my girlfriend about these thoughts might be truthful and honest, but it would likely cause her to feel insecure and jealous, even though she has nothing to worry about.

Comment by elriggs on Firming Up Not-Lying Around Its Edge-Cases Is Less Broadly Useful Than One Might Initially Think · 2020-01-13T07:08:50.632Z · score: 1 (1 votes) · LW · GW

Ethical is undefined here, but if it was a defined standard, you’d just pick the available action that scores well on that standard, even if it doesn’t satisfy the constraint “behave as if you know all information you in fact know” (which I think the hiding Jews from a Nazi is the classic example)

If the point of solving the puzzle is to better understand the concept “ethics in relation to truth-acting” then I don’t think I’ve added much by the Nazi example or the games & performances ones.

What do you believe the point of the puzzle is? What would a good solution entail or imply?

Comment by elriggs on Firming Up Not-Lying Around Its Edge-Cases Is Less Broadly Useful Than One Might Initially Think · 2020-01-13T05:50:31.686Z · score: 1 (1 votes) · LW · GW

If I understand your puzzle right, then poker, surprise parties/engagements, and those lying games you play with your friends where some people are “murderers” but are trying to hide the fact.

Is your puzzle different than that?

Comment by elriggs on "Mild Hallucination" Test · 2019-10-12T20:01:06.934Z · score: 2 (2 votes) · LW · GW

Thanks, and curious questions are welcomed.

I don't think it's affected it, though I don't have an easy way to compare. I lost most of that vision last August, have been meditating for a year, and have learned to see these perceptions in the past week. The vision in my left eye is definitely much, much noisier!

You can sort of recreate it by covering one eye and checking, though the difference is my left eye has no lens, no iris, and some retinal detachment.

Comment by elriggs on "Mild Hallucination" Test · 2019-10-12T17:05:27.373Z · score: 2 (2 votes) · LW · GW

Thanks, I've honestly learned so much throughout our comment thread.

One thing I'm confused about it why/how local contexts recognized by neurotypicals.

Maybe "mimic high-status members of in-group" explains most of it(?), or "what's other people doing?" or "what would someone else in my current role do?"

I think that's confused because if I know what "role" I'm in, then I already have a context in mind, and I'm trying to figure out how that context is derived in the first place!

Maybe contexts feel more solid/real to neurotypicals. "School" feels like a real/solid thing (even though it's just a building where kids ...). "Money" feels like it's real/solid (even though it's just paper or a number in a database with a socially agreed upon value attached). Being a "Good Student" feels real/tangible (even though it's just writing notes directly from the board and ...)

Those 3 examples are definitely things I felt were real/solid/tangible and I didn't connect the "even though it's ..." definitions until highschool/ undergrad.

Comment by elriggs on "Mild Hallucination" Test · 2019-10-12T16:39:47.031Z · score: 2 (2 votes) · LW · GW

Thanks for the info!

I also especially liked the saccadic masking, specifically

This can easily be duplicated by looking into a mirror, and looking from one eye to another. The eyes can never be observed in motion, yet an external observer clearly sees the motion of the eyes.

Which I remember trying and failing to do a few years ago. I recently lost my vision in one of my eyes, so it seems impossible to try the mirror test now (Although I still don't notice movement in my peripheral switching from my good eye to my nose, so maybe?).

Comment by elriggs on "Mild Hallucination" Test · 2019-10-12T10:44:17.146Z · score: 1 (1 votes) · LW · GW

Could you give examples of s1 "knowing" things until s2 inquires? I can understand how it "knows" visual snow, and by doing these tests we are "inquiring" about it. But I'm sure there are other contexts (other than visual information) where this concept is true.

Comment by elriggs on "Mild Hallucination" Test · 2019-10-12T10:38:33.286Z · score: 1 (1 votes) · LW · GW
At one extreme all data could be sent all the time to all functionalities but each functionality only really digests a small portion of it.

Is this the context-blind extreme? and

Or in reverse a brain that gets easily confused by garbage data might limit by only transitting information really required for the operations

is the other extreme?

Rephrasing: All people need to filter data, but from which data? Context-blind filters global variables while context-sensitive filters from local variables.

Also, what about games/activities with explicit rules such as chess or programming languages? Wouldn't everyone be able to identify those contexts and apply the right rules? (Assume they know the rules)

Comment by elriggs on "Mild Hallucination" Test · 2019-10-12T10:24:30.109Z · score: 1 (1 votes) · LW · GW

I agree. These all feel like very real sensory information. This is in contrast to being in sleep paralysis and creating extra sensory information or in very vivid dreams, since in both of these cases I realize afterwards "Oh, those weren't real" as in, I didn't actually receive that sensory information.

Also, I made a mistake in my initial post, my correction is separating different things that might be confused with "visual snow" such as:

1. Visual Snow - Like a million very tiny dots. Very much like static/white noise in the wiki. More visible in low light conditions or when you're tired. I saw it for the first time this (8/12) morning in low-light conditions.
2. Patterned lines (?) - Like the geometric/kaleidoscopic shape in this picture. Doesn't have to be that consistent or patterned but is better described by "lines" than either of the other two. This is what I meant by "jumpy spiderwebs made out of light" and what I thought visual snow was.
3. Blue-sky Sprites - The picture is a nice animation (can be seen without looking at the blue sky but apparently it's more prominent in that case). Dots and wisps the size of a mm or a little bigger. Maybe 5-100 at a time vs the million in "visual snow". Resembles afterimages and the "black stars" when feeling faint.
4. (Also very possible there's more that I've missed)

Did you see visual snow as in #1? And the others?

Comment by elriggs on "Mild Hallucination" Test · 2019-10-12T10:04:16.913Z · score: 1 (1 votes) · LW · GW

Would you be willing to comment on the role of neurodivergence, meditation, psychedelic experiences, or a 4th alternative, that may explain why you can already see all of them?

Comment by elriggs on "Mild Hallucination" Test · 2019-10-12T10:02:42.710Z · score: 1 (1 votes) · LW · GW

Honestly I mixed up different phenomena for "visual snow" in my description:

1. Visual Snow - Like a million very tiny dots. Very much like static/white noise

2. Patterned lines (?) - Like the geometric shape in this picture. Doesn't have to be that consistent or patterned but is better described by "lines" than either of the other two.

3. Blue-sky Sprites - The picture is a nice animation. This can be seen without looking at the blue sky but apparently it's more prominent in that case.

Did you mean #1 for the "visual snow"?

Comment by elriggs on "Mild Hallucination" Test · 2019-10-12T09:51:48.460Z · score: 2 (2 votes) · LW · GW

That’s a really good animation for the blue-sky sprites. When teaching a friend to see visual snow, they could only see these.

Do you see the kaleidoscopic, patterned lines like the picture from slate star codex’s article? It’s not always regular or geometric, but it’s separate from visual snow and blue-sky sprites. Actually, I had never seen visual snow until this morning in low light conditions. I thought the patterned lines were visual snow the whole time!

Comment by elriggs on "Mild Hallucination" Test · 2019-10-11T17:34:35.285Z · score: 3 (2 votes) · LW · GW

Is it similar to when colors are more vibrant on a cloudy day? (When the blue glare of the sky is gone)

Comment by elriggs on "Mild Hallucination" Test · 2019-10-11T16:15:49.348Z · score: 1 (1 votes) · LW · GW

No wait, this doesn't make sense framed this way. I think everyone isn't context-blind when the rules are explicit. If we're playing tag, or chess, or programming in Python, (I think) most people know which rules apply in this context because those rules are more explicit.

If so, maybe it's contexts with implicit rules? And implicit rules are learned by mimicking other's reactions?

Comment by elriggs on "Mild Hallucination" Test · 2019-10-11T15:38:21.717Z · score: 2 (2 votes) · LW · GW
One of the theories for the type of divergence I have is context-blindness. That would explain that if a more typical brain has very strong magisteria for each kind of context they can't cross-pollute as easily. Thus low-level pattern matching would be encapsulated to be invisible to the rest of the brain.

Thanks for pointing out "context-blindness". Let me see if I've got this straight.

A neurotypical has these different contexts/magisteria where different rules and interpretations apply. Someone who is context-blind has trouble identifying different contexts and so applies a global set of rules and interpretations in all situation.(?)

And this relates to low-level patterns because these different contexts are actually just sort of arbitrary, or just social constructs, so they're impossible to see when you're only paying attention to low-level details (?)

Comment by elriggs on "Mild Hallucination" Test · 2019-10-11T15:27:04.088Z · score: 1 (1 votes) · LW · GW

I think I understand your first point connecting "not seeing rawer data" and a synesthetic person having a mismatched letter/color. I think your main point is: You do/don't see rawer data depending on the context. (Also, can you choose to see the rawer data, or choose to only the abstract deduction?)

I guess with meditation black boxes become more white. The effect would depend a lot how how boxed things were to begin with. And it probably isn't activity that is generated but just acknowledged. Thus it is not really hallucinations

What is "not really hallucinations" here? The 3 tests above? Also, what do you mean by hallucinations in this context?

Comment by elriggs on "Mild Hallucination" Test · 2019-10-11T14:59:23.243Z · score: 1 (1 votes) · LW · GW

Do you think you could write your own type of test for seeing that type of "glowing". Like what's the ideal environment and what should one look for?

Comment by elriggs on "Mild Hallucination" Test · 2019-10-11T14:56:59.658Z · score: 1 (1 votes) · LW · GW

Thanks for taking the time make another account!

That's interesting. I see the "kaleidoscope patterns" like the picture, but also like ~100 pulsating/popping tiny dots that resemble afterimages (is this what others think for visual snow?).

I view these as two separate low-level information because I could see the first yesterday, but I can now see the second today. Someone I asked today could only see the second. But, you said

Note it wasn't visual snow with this pattern overlaid - the visual snow had become this pattern.

which makes it seem like you view them as one and the same?

Also, that insight seems related to concepts explored in the book Seeing that Frees, and the podcast Deconstructing Yourself. I'd also be more than happy to discuss that insight with you though I am by no means an expert.

Comment by elriggs on "Mild Hallucination" Test · 2019-10-11T14:39:28.165Z · score: 1 (1 votes) · LW · GW

Thanks for pointing that out. While trying to notice the "glides/shifts/jerks", I was looking through a door and it looked like I was on a rocking boat. Like different "background layers" (like at a play or picture book) where shifting in different directions.

Comment by elriggs on "Mild Hallucination" Test · 2019-10-11T13:08:32.783Z · score: 1 (1 votes) · LW · GW

Thanks for commenting that like I requested!

What are your thoughts on the connection to seeing these and neurodivergence, meditation, and psychedelic drug use?

Comment by elriggs on "Mild Hallucination" Test · 2019-10-11T13:00:47.065Z · score: 2 (2 votes) · LW · GW

I think everyone sees afterimages with lights and the wiki picture linked. For sure not everyone sees afterimages around objects while they’re currently looking at them with their eyes open.

I say that because I never noticed afterimages around objects until recently.

Were you unable to see visual snow after a minute using the instructions?

Comment by elriggs on "Mild Hallucination" Test · 2019-10-11T12:54:36.769Z · score: 2 (2 votes) · LW · GW

I appreciate that, but really your comment was fine and provided useful information. I’m just excited about this currently and want to talk about it more.

Also, the recommended reading I linked was more for your own curiosity’s sake as opposed to a pre-req, in case it came across that way.

Comment by elriggs on "Mild Hallucination" Test · 2019-10-11T00:41:58.497Z · score: 1 (1 votes) · LW · GW

Thanks for commenting that it's already obvious to you.

Would you comment on which parts of my theory (in my comment) are true for you. Feel free to PM me instead if you don't want to publish that information online.

Also, if you haven't already, I recommend reading the linked post (Lot's of People Going Around with Mild Hallucinations All the Time).

Comment by elriggs on "Mild Hallucination" Test · 2019-10-10T19:51:45.544Z · score: 1 (1 votes) · LW · GW

When reading up on this, lots of people said these mild hallucinations were associated with tiredness, migraines, low light levels, and other things.

I assume you've recently spent a lot of time working with computers, and you still see visual snow now? If not, can you see it with the instructions given?

Comment by elriggs on "Mild Hallucination" Test · 2019-10-10T19:47:42.855Z · score: 3 (2 votes) · LW · GW

By "glowing effects" do you mean something like the light rays coming off the street lamp in this picture?

Regarding "tinnitus", in the Mind Illuminated, it claims that master of "stage 8" is...

When the eyes perceive only an inner light, the ears perceive only an inner sound, the body is suffused with a sense of pleasure and comfort, and your mental state is one of intense joy.

Do you think this "tinnitus" is the same as this inner sound? I can hear this ringing "inner sound" when I "let go" of hearing. Same with the inner light when I "let go" of seeing. And to add, bodily sensations feel like very fast vibrations when I "let go" of sensing them. (note: I don't claim to have "mastered stage 8"). I think the "click" for me is "letting go", like relaxing a muscle but with sensory information. Is that similar to your experience or not at all?

Comment by elriggs on "Mild Hallucination" Test · 2019-10-10T18:12:24.393Z · score: 9 (3 votes) · LW · GW

Here, I would like to share my current theory on the type of people who are more prone to see this and why. Definitely inspired by Scott Alexander's previous writings.

I believe these hallucinations are just low-level processing information. I believe that being neurodivergent, taking psychedelics, and meditating all help more naturally see low-level sensory information.

I think Psychadelics causing these hallucinations is already agreed upon, so I won't defend it here.

Regarding meditation, I'm neurotypical and haven't taken psychedelics. I wasn't able to see these hallucinations until meditating consistently for a year, although I can't say I ever tried very hard to do these things. Like, would I have seen them if I tried the tests in this post a few years ago?

Regarding neurodivergence, I tried this with my neurodivergent friend. They could see all of them beforehand and was confused because they thought everyone could see them (but that everyone just ignored it and didn't talk about it).

Overall, this is weak evidence, and I'd like to hear other people's experiences and models of this.

Comment by elriggs on Proving Too Much (w/ exercises) · 2019-10-10T16:31:46.800Z · score: 3 (2 votes) · LW · GW

Wow, this is exactly the type of feedback I wanted, thank you!

I’ve changed my view on this, and my current model is the frame “I can prove anything in the set A because of reason X”

Like I can prove a certain set of facts about Natural numbers using induction, but to claim that induction proves all things about Real numbers or morality or... is proving too much.

I would rewrite the post to focus on questions regarding that such as:

  1. What set of claims do you think reason X proves?
  2. How do you know that reason X proves those types of claims? (And of course figure out how to phrase these things more tactfully)

Also, I’ve also enjoyed Thinking Physics and TurnTrout’s AU sequence type questions over my “pattern match to low-status belief” ones (I do like my generalization and algorithm question though), so I think I understand your point there.

Comment by elriggs on Finding Cruxes · 2019-09-27T01:37:05.851Z · score: 2 (2 votes) · LW · GW

Of course, I’ve really appreciated your input.

I like using this formula as a guideline for introspection, and the overall purpose is understanding the other person (which is related to curiosity, but not my purpose).

A negative after step 2, “no, I would still be just as confident” helps focus the conversation on actual cruxes. However, I did have a guy, having already understood I was asking for cruxes, say that the reason was a crux, but it wasn’t the complete reason (“yes, if God didn’t give grace I wouldn’t believe in him at all, but the Bible and 2000 year history are also important”)

Maybe if I was talking to someone else, they wouldn’t be able to say the extra reasons, being more timid or less introspective. But I’m pretty good at noticing when someone doesn’t react like “Oh, I’ve been 100% convinced and nothing is wrong with this logic”. This skill is very useful, and isn’t mentioned in the post.

The best use of this method is definitely drawing the picture using the picture to keep up with all the reasons and reasons for reasons. It makes it so much easier for both of us to stay on track and remember what was said.

Okay, so that’s the benefits and caveats of the method, though I’m confused on your “the strategy wants the other to tell a story on how they’d arrive to a new stance”.

I don’t understand this. If I believe in ghosts. And you use this method. The story would be how I would arrive to not believing in ghosts? Like just the negative of the original belief, not anything else new, right?

If so, then I don’t think that story is very hard if, after introspecting using this method, I figure out my reasons for believing in ghosts are flawed.

But maybe if the belief is very important like a religious one, properly setting someone’s expectations would be good. Like I might need to tell them “yes, you can still be a good person, be happy, have great friends, find love, etc” if they change their belief.

Was that your point?

Comment by elriggs on Finding Cruxes · 2019-09-27T01:06:06.545Z · score: 1 (1 votes) · LW · GW

Thanks for explaining the differences and going into detail for the combinations. I defined crux in this post as anything with probability attached to it, such that if it wasn’t true, the confidence of the belief would lower. This is more general and covers cases that have multiple reasons that lead to a belief.

For ex. I believe with 99% confidence that I picked the fair coin after I flipped it 10 times. Each of those 10 flips contributes to the belief, and each is a crux.

I don’t quite understand the disambiguating of the last line. Some people do interpret it as “give me a defense/good arguments for that belief“, but I don’t see how “no really, ...” couldn’t also be misinterpreted the same way.

To clarify why I trust one statement and not another, I used this technique a couple of days ago with two guys. I asked the “why do you believe this?”, and didn’t get a crux. I asked if he’d be just as confident if his reason didn’t exist and he said he’d be just as confident.

After then explaining that I’m looking for reasons that contribute to the confidence of his belief, he said “oh, I get it” and we had a very productive conversation.

I think that drawing the picture, asking for their confidence, asking why, and asking if it’s a crux helps tremendously towards a productive conversation. I think this process (which takes like 1-2 minutes per iteration) disambiguates the “why” question mentioned above. (Though, if you have better phrasing’s that are clearer, I’d be happy to hear them)

Comment by elriggs on Finding Cruxes · 2019-09-27T00:31:28.385Z · score: 1 (1 votes) · LW · GW

Thanks for explaining. I’m more convinced you’re right math wise, though I haven’t verified for myself.

I don’t think understanding this or working it out correctly will help in actual conversations with people about their beliefs though. (In fact, I get the most out of it by just drawing the picture of beliefs and connected reasons, and writing estimates probabilities. It really helps keep track of what’s said and makes circular reasoning very clear.)

Are you saying there is a practical reason for doing so? I can’t imagine one for the average university student I run into, let alone less technical people. Maybe with oneself or someone technical?

Comment by elriggs on Finding Cruxes · 2019-09-23T23:45:12.558Z · score: 1 (1 votes) · LW · GW

I don't understand how your comment relates to mine. Are you claiming the math to update the confidence is wrong?

Are you claiming that I haven't properly defined how to calculate the probabilities and that this is bad for a reason?

Comment by elriggs on Finding Cruxes · 2019-09-23T04:12:27.207Z · score: 1 (1 votes) · LW · GW

I don’t know what you mean by “true reason”. I’ve defined a “crux” as holding probability such that it being true or not actually affects the confidence in the belief. Could you define “true reason” in this level of detail?

I’m confused about where we disagree:

  1. When someone gives me a reason for why they believe in something, I don’t assume that they gave me a crux.

  2. When I ask someone “if that reason turned out to not be true, would you still be just as confident in your belief?”, I’ll usually trust them when they say “yes” or “no”.

  3. If after 2, and I show them that their reason is actually false, they say “oh, that actually didn’t change my mind like I predicted”, then most people would feel weird/bad about being inconsistent and would try to resolve it. This situation is also good, but I predict it’s unlikely.

Comment by elriggs on Finding Cruxes · 2019-09-23T03:56:53.627Z · score: 1 (1 votes) · LW · GW

I think you’re saying: Someone might not know all the relevant information, or all the logical implications, and it might be good to encourage them to read more information or think through more implications(?)

Regardless, I think using the recursive finding cruxes algorithm given in this post solves any of these issues in a real life conversation. Are you claiming that it doesn’t?

Comment by elriggs on Finding Cruxes · 2019-09-23T03:43:36.015Z · score: 1 (1 votes) · LW · GW

Oh! That’s clear, thanks!

I give an example of this in the “bob is best friend” picture.

How you calculate it is just a proportion. I’m 99% sure of ghosts, and 60% of that is 60*.99=59.4 percentage points.

If I figure out that the ghost girl was actually just my brain rationalizing sleep paralysis, then my belief in ghosts loses 59.4 percentage points. So now I believe in ghosts with 99-59.4= 39.4% confidence. Note that the other two beliefs (and unmentioned beliefs not in set {A,B,C}) must now be normalized to equal 100%.

You should be able to verify that you understand this by getting the same answer I did in the “Bob is best friend” example.

With this you can also answer: how many percentage points of 99% do you lose when the ghost girl belief goes from 60% to 50%?

Comment by elriggs on Finding Cruxes · 2019-09-22T01:31:38.132Z · score: 1 (1 votes) · LW · GW

So there’s still other reasons, right? They’re just not in the set {A,B,C}?

I don’t understand your overall point. Like say I believe in ghosts with 99% confidence. Three reasons why are: A. Ghost shows I watch: 10% B. Internet stories: 15% C. Those few times a ghost girl stood at the foot of my bed and I couldn’t move or scream: 60%

Would you apply what you’re trying to say/ask to this example?

Comment by elriggs on Finding Cruxes · 2019-09-22T01:22:45.719Z · score: 1 (1 votes) · LW · GW

I realize my other comment covers your 3rd paragraph, so moving on to the 4th one:

Not sure I fully understand it. But if I got someone to say “I don’t know”, then I count that as a major win, because they’ll usually think about it more and try to figure it out. So I think we agree that’s a good thing.

I don’t think it always implies that, in this case, intelligence is a crux. After introspecting, they could say “actually, I think I care about whether or not I had them as a pet growing up”.

I don’t understand the agnosticism Is valuable to flip part.

Comment by elriggs on Finding Cruxes · 2019-09-22T01:12:26.425Z · score: 1 (1 votes) · LW · GW

I don’t understand the point you’re making in your first two paragraphs, could you explicitly relate it to finding cruxes and what you specifically disagree/agree with?

I did understand the Al part though! I never claimed that he was being purposely misleading, but I did want it to come across as “Al is giving a reason for his belief that only accounts for <10% of his confidence”, or “Al isn’t giving the main reason for his belief that accounts for the most probability”.

I agree it can account for a smaller probability, and this is mentioned in Ex. 1 as what’s subtly wrong with my phrasing.

Comment by elriggs on Finding Cruxes · 2019-09-21T15:33:29.831Z · score: 1 (1 votes) · LW · GW

{} should be 0%, unless you’re talking about a uniform distribution over all possibilities? Like {} for a coin flip is 50% for heads, and {} for a dice roll being 4 is 1/6?

Though, when talking to someone, I would probably never go so technical as to ask for their confidence for each subset. They are probably not calibrated and probably can’t even enumerate every possible reason why they believe something. [This is also related to Scott Alexander’s “Not Sounding Like a Robot”]

Iterating through the algorithm in this post does allow someone else to think through possible reasons why they believe, and whether or not that reason is a crux.

Comment by elriggs on False Dilemmas w/ exercises · 2019-09-18T15:16:26.969Z · score: 1 (1 votes) · LW · GW

I fixed it the issue on mine. I created and shared a draft with you reproducing the error.