post by [deleted] · · ? · GW · 0 comments

This is a link post for

0 comments

Comments sorted by top scores.

comment by sarahconstantin · 2018-02-15T21:58:45.231Z · LW(p) · GW(p)
psychedelic experiences hold promise as a treatment for treatment-resistant depression, and may also improve the intentions of highly capable people who have not reflected much about what matters (“the betterment of well people”).

I'd like to push back on this a little.

I recently did a lit review (https://srconstantin.wordpress.com/2018/02/12/atypical-treatment-resistant-depression/) on what's been shown to work for treatment-resistant depression, and while there is a study on psilocybin and it did show positive effects, it was small and uncontrolled, and fairly short-term (only three-month follow-up.)

My rough model, from anecdotal evidence, is that there is such a thing as a short-term, several-month "honeymoon" from depression that a single use of psychedelics or especially MDMA can trigger, but that these are frequently not permanent cures. We have pretty solid evidence that a single use of ketamine can produce dramatic, but short-term and rapidly-fading, reprieves from treatment-resistant depression. Sometimes temporary isn't bad -- if temporarily having your depression lifted allows you to access insights that you can carry forward into the rest of your life. But I think we should expect that one-time trips are less likely to give permanent effects than drugs that you can take every day.

The "improving the intentions of highly capable people who have not reflected much about what matters" part seems roughly true to me. I think that psychedelics are especially valuable to people who want to understand what minds are like, so, roughly, people whose work intersects with neuroscience, psychology, AI, or philosophy. I think theories about minds are bound to be incomplete if they don't include exposure to minds in very different states than the thinker's own habitual waking life. Other stuff that you should be exposed to, ideally through practical experience:

  1. How babies and young children think
  2. How animals think
  3. How mentally ill and cognitively disabled people think
  4. How people think in states of meditation or religious ecstasy
  5. How computer programs "think"

If you've ever read, say, Kant and thought "wow, has this guy literally ever played with a baby?! He's obviously wrong about what cognitive processes are universal or innate" then you have some idea of what the benefits of exposure to different minds are. Empirical experience with what minds are like seems important for "universal" questions about what is worth valuing & pursuing. Note that scientific/humanist generalists like Oliver Sacks have deep experience with most of 1-5 (I don't know that he ever studied computer science) as well as frequently having experience with psychedelics.

Replies from: ioannes_shade
comment by ioannes (ioannes_shade) · 2018-02-19T16:40:31.283Z · LW(p) · GW(p)

Thanks for the great comment :-)

I agree that the evidence around psychedelics for treatment-resistant depression is slim & preliminary. The effect sizes in Carhart-Harris et al. 2016 are very large, so I'm optimistic that a sizeable (though smaller) effect will occur in larger studies.

Sometimes temporary isn't bad -- if temporarily having your depression lifted allows you to access insights that you can carry forward into the rest of your life.

The mechanism I'm most excited about re: psychedelics for depression is a high-dose psychedelic experience functioning as a catalyst for other behavior & worldview changes which, when taken all together, lead people out of their depression.

comment by Qiaochu_Yuan · 2018-02-15T19:21:44.211Z · LW(p) · GW(p)

I agree that animal welfare should be deprioritized and that understanding what matters is quite important.

The EA community's current approach to understanding what matters seems terrible to me, and has resulted in what looks to me like negative progress so far.

Consider the following analogy: suppose we live in a dystopia (hint, this isn't really an analogy) in which some vital aspect of human value has been thoroughly suppressed, say sex + romantic love to be concrete. There is no concept of sex or romantic love anywhere in the culture, except as a moral aberration that should be fought at all costs, people are taught to completely ignore their sexual or romantic urges, people get married based on genetic compatibility and reproduce via artificial insemination, etc.

Now imagine founding an organization, in this dystopia, whose mission is to try to figure out what matters by thinking, reasoning, theorizing, modeling, etc. My sense is very strongly that it will never occur to such an organization that something like sex + romantic love could be a vital aspect of human value, especially if historical mentions of other societies in which sex + romantic love mattered were either suppressed or thoroughly denigrated as morally degenerate, as long as their methodology is primarily thinking-based. If anything I expect them to come to conclusions like "well, since the strange urges we get from our genitals produce a lot of suffering in us, obviously we should be putting more funding into castration research," or whatever.

I think the only methodology that might work is for the members of such an organization to learn how to listen to their bodies, especially their sexual and romantic urges; give in to those urges despite tremendous social pressure not to and probably a bunch of what sound like excellent intellectual arguments that they shouldn't; and then decide, based on the evidence from their bodies, that what happens as a result (sex + love) feels vitally important to human value. (Of course there's the question of what methodology they could possibly follow, that would seem reasonable to them, that would bring to attention the hypothesis that they should do something like listen to their bodies; I don't have a good answer to that and that scares me.)

Edit: One general methodology that might work, although I imagine many people might find it distasteful, is to update towards past humans, which is also something that I try to do a little. Compared to the present day, this suggests that the following might be neglected components of human value: religion, violence, farming, hunting...

I worry that all of the organizations you describe as working on the problem of understanding what matters are using a thinking-based methodology that is 1) heavily influenced by cultural blindspots and 2) disconnected from the direct experience of being in a human body. The methodology I've personally been following for trying to understand what matters is to try to make my life extremely good by chasing the felt sense of "this feels vitally important to human value" through a bunch of embodied experiences, and I think I've come across important stuff that EAs don't talk about at all.

As a simple example, I don't hear people worrying a ton about the difference between just-the-head cryonics and full-body cryonics, and I personally have updated heavily in the direction of just-the-head cryonics being a terrible idea, because I think much more of what makes a person who they are is in their body and not their brain than I used to.

Replies from: Davis_Kingsley, Benito, SaidAchmiz, ioannes_shade
comment by Davis_Kingsley · 2018-02-16T17:50:57.598Z · LW(p) · GW(p)

I find this claim very surprising - in many respects it seems in opposition to a lot of the models that are conventionally used here. (That doesn't, of course, mean that it's wrong.) I think the classic heuristics and biases view holds that many of our default intuitions are predictably inaccurate. In fact, I've often thought that this community doesn't go far enough in skepticism of default "embodied concepts" - for instance, I believe there are many both in broader society and in this community for whom human sexuality is outright negative on net, and yet I see extremely few willing to entertain this notion, even as a hypothesis.

I do think that the sort of view you describe is clearly much better than the old transhumanist "my body is a meat prison" view, but my sense was that comparatively few significantly endorsed that and little policy was based on it in any case.

I'd be interested in hearing examples of scenarios where you believe important policies seem disaligned with embodied values; I find this claim very bold and am quite interested in figuring out whether there's a good way to test it. If true, it could necessitate a serious realignment of many groups.

Replies from: Qiaochu_Yuan
comment by Qiaochu_Yuan · 2018-02-16T19:20:41.972Z · LW(p) · GW(p)
For instance, I think the classic heuristics and biases view holds that many of our default intuitions are predictably inaccurate.

The classic heuristics and biases literature is about things like the planning fallacy; it has very little to say about intuitions about human value, which is more in the domain of experimental moral philosophy. "Intuitions" is also a pretty broad term and arguably doesn't cleave reality at the joints; it's easy for some intuitions to fall prey to cultural blindspots (along the lines of Paul Graham's moral fashion) in exactly the way I'm worried about and I think it's an important problem to find ways of getting past blindspots in a broad sense. I trust some of my intuitions, whatever that means, much more than others for doing this.

I believe there are many both in broader society and in this community for whom human sexuality is outright negative on net, and yet I see extremely few willing to entertain this notion, even as a hypothesis.

I'm willing to entertain this as a hypothesis, although I'd be extremely sad to live in this world. I appreciate your willingness to stick up for this belief; I think this is exactly the kind of getting-past-blindspots thing we need on the meta level even if I currently disagree on the object level.

I'd be interested in hearing examples of scenarios where you believe important policies seem disaligned with embodied values; I find this claim very bold and am quite interested in figuring out whether there's a good way to test it. If true, it could necessitate a serious realignment of many groups.

So as I mentioned in another comment, I think basically all of the weird positions described in the SSC post on EAG 2017 are wrong. People who are worrying about insect suffering or particle suffering seem to me to be making philosophical mistakes and to the extent that those people are setting agendas I think they're wasting everyone's time and attention.

Replies from: tristanm, Davis_Kingsley
comment by tristanm · 2018-02-16T23:36:40.894Z · LW(p) · GW(p)

Personally I wonder how much of this disagreement can be attributed to prematurely settling on specifc fundamental positions or some hidden metaphysics that certain organizations have (perhaps unknowingly) committed to - such as dualism or pansychism. One of the most salient paragraphs from Scott's article said:

Morality wasn’t supposed to be like this. Most of the effective altruists I met were nonrealist utilitarians. They don’t believe in some objective moral law imposed by an outside Power. They just think that we should pursue our own human-parochial moral values effectively. If there was ever a recipe for a safe and milquetoast ethical system, that should be it. And yet once you start thinking about what morality is – really thinking, the kind where you try to use mathematical models and formal logic – it opens up into these dark eldritch vistas of infinities and contradictions. The effective altruists started out wanting to do good. And they did: whole nine-digit-sums worth of good, spreadsheets full of lives saved and diseases cured and disasters averted. But if you really want to understand what you’re doing – get past the point where you can catch falling apples, to the point where you have a complete theory of gravitation – you end up with something as remote from normal human tenderheartedness as the conference lunches were from normal human food.

I feel like this quote has some extremely deep but subtly stated insight that is in alignment with some of the points you made. Somehow, even if we all start from the position that there is no univeral or ultimately real morality, when we apply all of our theorizing, modeling, debate, measurement, thinking, etc., this somehow leads us to making absolutist conclusions about what the "truly most important thing" is. And I wonder if this is primarily a social phenomenon: In the process of debate and organizing groups of people to accomplish things, it's easier if we all converge to agreement about specific and easy to state questions.

A possible explanation for Scott's observed duality between the "suits" on the one hand who just want to do the most easily-measurable good, and the "weirdos" on the other hand who want to converge to rigorous answers on the toughest of philosophical questions (and those answers tend to look pretty bizarre), and the fact that these are often the same people - my guess is this has something to do with coverging to agreement on relatively formalizable questions. Those questions often appear in two forms: The "easy to measure" kind of questions (how many people are dying from malaria, how poor is this group of people, etc.), and the "easy to model or theorize about" questions (what do we mean by suffering, what counts as a conscious being, etc.), and so you see a divergence of activity and effort spent between those two forms of questions.

"Easy" is meant in a relative sense, of course. Unfortunately, it seems that the kind of questions that interest you (and which I agree are of crucial importance) fall into the "relatively hard" category, and therefore are much more difficult to organize concerted efforts around.

comment by Davis_Kingsley · 2018-02-17T01:42:48.923Z · LW(p) · GW(p)
The classic heuristics and biases literature is about things like the planning fallacy; it has very little to say about intuitions about human value, which is more in the domain of experimental moral philosophy.

Fair point, though I do think it provides at least weak evidence in this domain as well. That said, there are other examples of cases where intuitions about human value can be very wrong in the moment that are perhaps more salient, - addictions and buyer's remorse come to mind.

I'm willing to entertain this as a hypothesis, although I'd be extremely sad to live in this world. I appreciate your willingness to stick up for this belief; I think this is exactly the kind of getting-past-blindspots thing we need on the meta level even if I currently disagree on the object level.

Thanks!

So as I mentioned in another comment, I think basically all of the weird positions described in the SSC post on EAG 2017 are wrong. People who are worrying about insect suffering or particle suffering seem to me to be making philosophical mistakes and to the extent that those people are setting agendas I think they're wasting everyone's time and attention.

I agree that these positions are mistakes. That said, I have three replies:

  1. I don't think the people who are making these sorts of mistakes are setting agendas or important policies. There are a few small organizations that are concerned with these matters, but they are (as far as I can tell) not taken particularly seriously aside from a small contingent of hardcore supporters.
  2. I worry that similar arguments can very easily be applied to all weird areas, even ones that may be valid. I personally think AI alignment considerations are quite significant, but I've often seen people saying things that I would parse as "being worried about AI alignment is a philosophical mistake", for instance.
  3. It is not clear to me that the "embodied" perspective you describe offers especially useful clarification on these issues. Perhaps it does in a way that I am too unskilled with this approach to understand? I (like you) think insect suffering and particle suffering are mistaken concepts and shouldn't be taken seriously, but I don't necessarily feel like I need an embodied perspective to realize that.
comment by Ben Pace (Benito) · 2018-02-15T22:59:05.784Z · LW(p) · GW(p)

If I take your point very generally, I feel like you're re-stating Eliezer's Archimedes Chronophone in a cool, different way. It's saying "To discover new value, you don't just need to the same thing but rigorously - you need to be able to get outside your incentives and patterns of thought and have ideas that cause you to pull the rope sideways."

If I take your point very literally, it seems false. For example I think that trying to practically understand basic economics / game theory very rigorously gets you really far (cf. Eliezer's post "Moloch's Toolbox", Bryan Caplan's work on education and voter theory, etc). Trying to do statistics on the QALY output of differing health interventions in the developing world gets you important insights (cf. Toby Ord's "On the moral imperative toward cost-effectiveness").

(This comment is responding to your points about theory, not about the specific cognitive models being used within EA, which is a separate story.)

Replies from: Raemon, Qiaochu_Yuan
comment by Raemon · 2018-02-15T23:09:46.225Z · LW(p) · GW(p)

This didn't seem like it really addressed the point Qiaochu was making.

My guess is Qiaochu agrees that "economics and game theory can get you really far." But, would it specifically lead to EAs in the described dystopia discovering the value of sex and romance, and valuing them appropriately? I'm not sure if you're claiming it would, or that claiming that it doesn't matter especially, or something else.

comment by Qiaochu_Yuan · 2018-02-15T23:24:57.635Z · LW(p) · GW(p)

I'm not sure you can pass my ITT; can you try doing that first?

Why do you think QALYs matter even approximately?

Replies from: Benito
comment by Ben Pace (Benito) · 2018-02-16T01:15:39.829Z · LW(p) · GW(p)

Brief response to the second question, will see if I can come back and attempt an ITT later.

QALYs don't matter much (I think), but learning that they're power-law distributed matters a lot. It's a datapoint for the general point, akin to Paul Graham's essay on black-swan farming - it tells you that value is power law distributed and that figuring out what's the most valuable thing is super-duper important, relative to how you might otherwise have been modelling the world.

Replies from: Qiaochu_Yuan
comment by Qiaochu_Yuan · 2018-02-16T02:03:25.799Z · LW(p) · GW(p)

Okay, let me rephrase my ITT request; what I meant to say is that I'm confused, because your chronophone analogy is pretty close to the point I'm trying to make, but I don't understand what you have in mind for my point taken "very literally" because I don't understand how your second paragraph is a response to what I wrote.

I'm pretty wary of taking this power-law methodology seriously. If you think importance is power-law distributed and your estimates of what's important are multiplicatively noisy, then the top of your list of things sorted by importance will mostly be dominated by noise. This is basically my reaction to the SSC post on EAG 2017; I think basically every weird position he describes in that post is a philosophical mistake (e.g. the wild animal suffering people seem to be using a form of total utilitarianism, which I reject completely).

Replies from: Benito, Benito, Benito
comment by Ben Pace (Benito) · 2018-02-16T20:11:47.277Z · LW(p) · GW(p)

First off, I'd like to apologise, I've only now read the OP, and I was talking past you. I still think we have some genuine disagreements however, so I'll try to clarify those.

I was surprised by how much I liked the post. It separates the following approaches for improving the state of the world:

  1. Further our understanding of what matters
  2. Improve governance
  3. Improve prediction-making & foresight
  4. Reduce existential risk
  5. Increase the number of well-intentioned, highly capable people

My ordering of tractability on these is roughly as follows: 5 > 4 > 3 > 2 > 1. There is then a question of importance and neglectedness. I basically think they're all fairly neglected, and I don't have strong opinions on scope except that, probably, number 4 is slightly higher than the rest.

Understanding what matters seems really hard. For the other problems (2-5) I can see strong feedback loops based in math for people to learn (e.g. in governance, people can study microeconomic models, play with them, make predictions and learn). I don't see this for problem 1.

Sure, for all 1-5 there will be steps where you have to step sideways - notice a key variable that everyone has been avoiding even thinking about, due to various incentives on what thoughts you can think - but there's more scope for practice, and for a lot of good work to be done that isn't 80% deep philosophical insight / deep rationality abilities.

Luke Muehlhauser tried really hard and found the tractability surprisingly low (1, 2).

There are worlds consistent with what I've said above, where I would nonetheless want to devote significant resources to 1, if (say) we were making plans on a 100-200 year horizon. However, I believe that we're in a world where we're very soon going to have to score perfectly on number 4, and furthermore that scoring perfectly on number 4 will cause the other problems to get much easier - including problem number 1.

Summary: As I hadn't read the OP, I read your comment as claiming your approach to 1 was the only way to do good altruistic work. I then responded with reasons to think that other - more technical approaches - were just as good (especially for the other problems). I now pivot my response to reasons to think that working on things that aren't 1 are more important, which I think we may disagree on.

Replies from: Qiaochu_Yuan
comment by Qiaochu_Yuan · 2018-02-16T20:23:50.247Z · LW(p) · GW(p)

I think I basically agree with all of this, except that maybe I think 1 is somewhat more tractable than you do. What I wrote was mostly a response to the OP's listing of organizations working on 1, and my sense that the OP thought that these organizations were / are making positive progress, which is far from clear to me.

comment by Ben Pace (Benito) · 2018-02-16T20:09:46.983Z · LW(p) · GW(p)

I can't tell if it's still necessary, but I wanted to try anyway. Here's my ITT for your position. This story isn't literally what I think you think, but I believe it's an accurate analogy.

---

You have recently done something new, like tried LSD, or gotten really addicted to the high of exercise, or just moved to a new place and been genuinley happy and excited every minute for a whole week.

And you learned something - you didn't know life could feel this good. It's a genuine insight and feels important to you, and does change how you will make life plans forevermore.

You talk to me. I talk about the importance of figuring out what really matters in life; I've just spent many weeks reading textbooks on the subject and philosophical treatise on experience. And now you're stuck trying to communicate to me the realisation that life can be really good - you think it's causing me to make mistakes regarding all sorts of things I discuss, from population ethics to the moral value of sleep.

You know that you don't know everything there is to know about the good life, but you're pretty sure that whatever other weird realisations are out there, I'm sure not gonna find them doing what I'm doing right now.

Replies from: Qiaochu_Yuan
comment by Qiaochu_Yuan · 2018-02-16T20:27:58.654Z · LW(p) · GW(p)

I'm basically happy with this analogy as far as it goes, although it doesn't capture the part where part of the reason it's hard for me to communicate the thing is cultural blindspots; the dystopia analogy captures this really well and that's why I like it so much.

comment by Ben Pace (Benito) · 2018-02-16T20:09:10.870Z · LW(p) · GW(p)

Regarding power laws, I am attempting to make a strong claim about how reality works - reality, not methodology. It is the case that the things we value are power law distributed - whether it's health interventions, successful startup ideas, or altruistic causes, it turns out that selecting the right one is where most of the variance is.

As a result, one's ability to do good will indeed be very noisy - this is why many good funders take hits-based approaches. For example, Peter Thiel is known for asking interesting people for their three weirdest ideas, and funding at least one of them. He funded MIRI early, though I expect he probably was quite unsure of it at the time, so I consider him to have picked up some strong hits.

That's also my feeling wrt the EA post on SSC. I'm generally not happy with the low variance approaches taken within EA and feel sad at how few new ideas are being tested, but I think that to say the number of orgs doing weird things wrt figuring out what matters, is to push the number in the wrong direction.

Replies from: Qiaochu_Yuan
comment by Qiaochu_Yuan · 2018-02-16T20:25:52.987Z · LW(p) · GW(p)

Sure, I'm basically happy with this modulo taking "power law" with the appropriate grains of salt (e.g. replacing with log normal or some other heavy tailed distribution as appropriate).

comment by Said Achmiz (SaidAchmiz) · 2018-02-15T20:29:49.895Z · LW(p) · GW(p)

Have you written anything in greater detail about this (“direct experience of being in a human body”, thinking that more of what makes a person who they are is in their body, etc.)? I’d be very interested in reading it, if so!

Replies from: Qiaochu_Yuan
comment by Qiaochu_Yuan · 2018-02-15T20:35:00.486Z · LW(p) · GW(p)

Unfortunately no. My sense of the thing is shifting a lot and is mostly experiential and nonverbal so it feels premature and a little dangerous to try to set it down in writing too early, like I might crystallize it into the wrong form and miss the actual thing. At this point if I tried I would mostly generate poetry, some vague references to the enteric nervous system, and descriptions of experiences that might be difficult to believe along the lines of the Kensho post. I think I can transmit a sense of the thing in person but not via text.

Edit: Apparently someone wrote a book, although I haven't read it and can't vouch for it directly.

Replies from: SaidAchmiz
comment by Said Achmiz (SaidAchmiz) · 2018-02-15T20:50:09.204Z · LW(p) · GW(p)

Thank you—I certainly understand your position. (And I’ll check out the link.) If you do get to a point where you can articulate this, I think it would make a great LW post (or series thereof).


This part is definitely a tangent, but here’s a follow-up question. You say:

I think I can transmit a sense of the thing in person but not via text.

I’ve heard similar things from a few folks here, lately (in the context of various discussions, on various subjects). What’s going on here? Do you have in mind that gestures would be needed to convey the thing properly, or non-verbal signals, or physical demonstrations, or… what?

Replies from: Qiaochu_Yuan
comment by Qiaochu_Yuan · 2018-02-16T00:03:07.940Z · LW(p) · GW(p)

So I'll start by giving what I think is an unobjectionable example of something that can't be transmitted via text, then use it to triangulate.

Consider the example of Mary's room. Mary lives in a black and white room and studies vision. (Her own body is also black and white, she has never seen her own blood, etc.) Let's stipulate that she has a perfect mechanical understanding of how human vision works, including exactly what happens when a particular wavelength of light is processed by the eye and the visual cortex as opposed to some other wavelength. It seems intuitive, and I would defend the intuition, that nonetheless Mary would learn something if she stepped out of the room and saw a red flower for the first time, with her actual eyes. Literally there are patterns of neuronal firing in her visual cortex that would happen if she did this that couldn't happen otherwise, unless someone hypnotized her into hallucinating the experience of seeing a red flower or something like that.

So, that's one example: I don't know how to transmit via text the qualia of seeing red. Worse, I don't even know how to convince Mary that there is such a thing as the qualia of seeing red, and that if she never saw red she would be missing out on something important. (I'm reminded at this point of the lack of colors in the world of The Giver, and also the SSC post on missing out on universal human experiences.)

The thing I'd want to transmit around the direct experience of being in a body has aspects of both transmitting the direct experience of seeing red and transmitting the direct experience of playing basketball, in that it has aspects of both an underutilized human capacity and learning a skill. The simplest way to do each of these is respectively to 1) show someone a red thing and 2) play basketball with them. The best you could do via text might be something like 1) give someone instructions for finding a red thing and 2) tell someone where to sign up for basketball classes, but this is not really transmitting the experience via text, it's giving someone instructions for where to find the experience themselves.

The thing I'd try to do in person, if I wanted to transmit the direct experience of being in a body, is something like run you through exercises to get you into your body, which I would ideally need to see you doing in real time to get a sense of what adjustments need to be made to the exercise. I can also demonstrate, in real time, the difference between me being in my body and not being in my body; there are things I can do in my body that are harder to do when I'm not in my body, and I have no idea how to convince you that I can do these things via text.

Via text I can at least give instructions for finding the experience, in no particular order: go to a silent meditation retreat, take psychedelic drugs, fast for 5 days and ingest nothing but water, find someone who will guide you through holotropic breathwork, find someone to perform shamanic healing on you, go to Burning Man and do as much weird shit there as you can handle. Some of these will be a lot more effective than others and I can't make guarantees about any of them. I can guide you to a basketball class but you still might just be really bad at basketball, etc.

Oh, actually, here's an exercise you can do right now, but my experience is that people vary dramatically in their ability to do it so it could easily completely fail for you, and I haven't figured out how to get it to work reliably yet. I got it from Impro. It goes like this: try to get a clear sense of where the "locus of control" of your body is. Probably it's where your brain is by default. I visualize it as something like a blue ball of energy shooting out electricity to the rest of my body to make it move. Now, try to move the locus somewhere else in your body. In particular, try to move it as far down your spine as possible; ideally you'll be able to move it to the base of your spine. For the sake of blinding I won't tell you what's supposed to happen when you do this, although there are a pretty wide range of possible responses anyway.

Replies from: gjm, PeterBorah, SaidAchmiz
comment by gjm · 2018-02-16T16:01:15.336Z · LW(p) · GW(p)

The usual formulation of the Mary's Room thought experiment makes a very strong assumption about what Mary knows while in the room: that she has a complete understanding of every physical and physiological process that happens when she is looking at things. This of course goes way beyond what any real human being has.

With this assumption, I claim that Mary would already know that there are such things as the qualia of seeing red; she wouldn't need convincing. And she would already know quite a lot about those qualia; e.g., if seeing red tends to increase physiological arousal and spark thoughts of love, anger and serious physical injury then she would know that.

(This argument is not original to me; I found it in a book by Daniel Dennett.)

Without that assumption, some arguments based on the thought experiment (e.g., ones claiming that qualia have to be something nonphysical) collapse. Probably not yours, though, because for your purposes it doesn't matter whether Mary understands anything at all about colour vision; I think you get a better example of experiences that might not be conveyable in text if we suppose she doesn't.

Still... it seems like one could convey at least some aspects -- though certainly not all -- of the experience of seeing red through text, even to someone who had never had that experience. Along the following lines. "At present, your eyes give you a picture of the world that could be described by saying for everything how bright it is. Other people -- perhaps including you, some day -- get more information from their eyes than that: for each location in their visual field, not just a brightness but something else called colour. Different objects have different colours, so this lets us distinguish more things about them than you can from brightness alone. One kind of colour is called "red", and it turns out that our brains respond in interesting ways to red things. Red is the colour of a liquid called blood that we have inside us. When your body is badly enough damaged, blood may flow out. That's bad news, and seeing red makes your brain prepare a little bit to fight or run away: it gives you a little of the feeling you might have if you imagine that there's a tiger just around the corner waiting to jump on you. Most people's skin is slightly reddish in colour, especially when they are embarrassed or excited, and perhaps for that reason red is also associated in most people's minds with love and sex. Seeing red may make your brain prepare a little bit for a romantic or sexual encounter: it gives you a little of the feeling you might have if you imagine that someone you find very attractive is just around the corner waiting to kiss you. When there's something bright red in your field of view, your eyes will tend to be attracted by it, and distract your attention from other things. In our culture, red is associated with fighting and celebration and love and danger and urgency. The colour red is a bit like the sound of a trumpet fanfare, and a bit like the feel of either a warm embrace or a slap in the face."

Hearing all that (and the much more that one could say on the same theme) isn't the same as actually having the experience of seeing red, for sure. But I think someone who had never seen the colour red would know more about what seeing red is like after reading a few paragraphs of that sort of thing. They'd have some information about specific effects it might have; some analogies to other things they might be familiar with; some understanding of why those effects might be there, which might help them predict other effects; some information about ideas and feelings and images that people associate with redness.

Replies from: Qiaochu_Yuan
comment by Qiaochu_Yuan · 2018-02-16T18:22:10.785Z · LW(p) · GW(p)

Fair point. The version of the Mary's room setup that's relevant to this discussion is where Mary knows everything about color vision that could be transmitted via text, e.g. the best and most comprehensive possible textbook on color vision.

I agree that I could describe my experience of red to Mary, but if Mary is skeptical / doesn't trust me to report my experiences honestly I don't know how I could convince her of anything, really.

Replies from: gjm
comment by gjm · 2018-02-17T03:00:08.449Z · LW(p) · GW(p)

It seems you're very concerned about being thought a liar or a fake if you say more about your experiences. (E.g., you bring that up here, but so far as I can see no one in this thread has been suggesting anything of the kind.) Is there a particular reason for that?

(For the avoidance of doubt, I think it very unlikely that what you're saying -- or indeed what you're avoiding saying -- is lies or fakery.)

Replies from: Qiaochu_Yuan
comment by Qiaochu_Yuan · 2018-02-17T04:54:42.951Z · LW(p) · GW(p)

Mostly the response to the Kensho thread. Also some common knowledge thing along the lines of, I don't want to make strong claims on the basis of weak evidence because I don't want people to think I think other people have or should have weak epistemic standards.

comment by PeterBorah · 2018-02-16T12:30:25.777Z · LW(p) · GW(p)

I tried this exercise and found it extremely interesting. My report is rot-13'ed to respect the blinding. I highly recommend taking five minutes to try it yourself before reading.

(One more line break for further encouragement to break your train of thought and try it.)

Ok, here:

V gevrq guvf fvggvat ng zl qrfx ng jbex. V'ir cerivbhfyl unq gebhoyr jvgu rkrepvfrf gung nfxrq zr gb zbir gur ybpngvba bs "zr"/zl rtb/zl crefcrpgvir bhg bs zl urnq naq vagb zl obql, fb V jnf fxrcgvpny V jbhyq fhpprrq. Ohg Dvnbpuh'f qrfpevcgvba va grezf bs "fubbgvat bhg ryrpgevpvgl gb gur erfg bs zl obql gb znxr vg zbir" znqr zr abgvpr n qvfgvapgvba orgjrra gur ybpngvba bs "gur jngpure" be zl bofreingvba/creprcgvba, naq "gur npgbe" be zl zbirzrag/npgvba. Guvf frpbaq guvat ghearq bhg gb or abg gung uneq gb zbir, juvyr gur svefg fgnlrq va cynpr.

V qba'g unir irel fgebat zragny vzntrel, fb V qvqa'g unir nalguvat yvxr n oyhr onyy bs raretl va zvaq. Zbgvba naq fcngvny eryngvbafuvc ner angheny gb zr, fb V sbphfrq ba gur vqrn bs na nofgenpg ybphf gung "chfurq bhg" gb pnhfr zbgvba.

Gb xrrc gur sbphf ba npgvba, V zbirq zl unaq va n ercrgvgvir hc-naq-qbja zbgvba juvyr V gevrq zbivat gur ybphf. Nf fbba nf vg tbg bhg bs zl urnq vagb zl hccre purfg, zl unaq zbirzragf tbg zhpu fybjre, yrff cerpvfr, naq yrff erthyne. Vg sryg n ovg yvxr jnyxvat onpxjneqf, be zbivat jvgu zl rlrf pybfrq. Gurer jnf nyfb n fbeg bs hcjneqf cerffher ba gur ybphf, yvxr jura lbh gel gb chfu fbzrguvat ohblnag qbja vagb jngre.

V ybfg genpx n pbhcyr gvzrf naq unq gb fgneg bire, ohg ba gur guveq gel V zbirq vg vagb zl thg, naq gura gb gur onfr bs zl fcvar. Nf V nccebnpurq gur onfr bs zl fcvar, na ryrpgevp, ohmmvat, fbzrjung birefgvzhyngvat raretl pragrerq gurer orpnzr fgebatre naq zber fnyvrag. Guvf raretl vf snzvyvne gb zr sebz zrqvgngvba, ohg V'ir cerivbhfyl sbhaq vg gbb vagrafr gb "ybbx ng" sbe ybat. Gb zl fhecevfr, gung jnfa'g gehr guvf gvzr, creuncf orpnhfr V jnf pbaarpgvat vg gb zbgvba naq npgvba engure guna whfg ybbxvat ng vg.

Gur raretl nebfr naq frggyrq dhvgr dhvpxyl, va n znggre bs n pbhcyr frpbaqf. Vg jnf (naq vf, nf V jevgr guvf) fgvyy cerfrag naq fnyvrag, ohg irel znantrnoyr. Nf fbba nf vg frggyrq, zl zbgvbaf jrag onpx gb orvat pbasvqrag, snfg, naq cerpvfr. Nsgre whfg n frpbaq, V (jvgubhg pbafpvbhfyl pubbfvat gb) fgbccrq gur ercrgvgvir unaq zbgvbaf naq ghearq gurz vagb n fgergpu, svefg sbe zl unaqf naq nezf, naq gura zl jubyr obql. Zl cbfgher jrag sebz fybhpurq gb fgenvtug-onpxrq (ohg ernfbanoyl erynkrq) naq V fcernq zl yrtf gb n zber pbzsbegnoyr cbfvgvba. V abgvprq gung zl cnagf jrer hapbzsbegnoyl gvtug naq erfgevpgvat, naq nyfb abgvprq gur cyrnfher bs tvivat zl obql n fngvfslvat fgergpu naq ernqwhfgzrag.

Guvf fgngr bs tbbq cbfgher naq obqvyl njnerarff vf snzvyvne gb fbzr qrterr, nf V bppnfvbanyyl trg vagb vg nsgre ubhe-ybat zrqvgngvba fvgf. Vg'f pbaarpgrq gb trareny tebhaqrqarff naq cerfrapr va fbpvny naq zragny ernyzf nf jryy, ohg vg'f n fgngr V unira'g orra noyr gb gevttre irel eryvnoyl.

V tbg hc gb tb gb nabgure ebbz, naq sbhaq zlfrys zbivat zber dhvpxyl naq pbasvqragyl guna V'z hfrq gb. Gurer'f n fjvatvat qbbe orgjrra gjb ebbzf, naq V fbeg bs yrnarq vagb vg jvgubhg oernxvat fgevqr. Zl zbgvbaf erzvaqrq zr n ovg bs ubj V'ir frra nguyrgrf va pnfhny fvghngvbaf zbir, jvgu n fbeg bs pbasvqrapr naq frafr gung gurve obql vf na rssrpgvir gbby. Vg nyfb orpnzr dhvgr boivbhf naq fnyvrag gung V'z uhatel, fvapr V unira'g rngra oernxsnfg.

V'ir xrcg gur ybphf bs npgvba ng gur onfr bs zl fcvar juvyr jevgvat guvf zrffntr. Vg frrzf gb pheeragyl erdhver n fznyy nzbhag bs onpxtebhaq nggragvba gb xrrc vg gurer (fvzvyne gb, yvxr, crepuvat ba n oenapu juvyr ernqvat n obbx, be fbzrguvat yvxr gung), ohg qbrfa'g frrz gb abgnoyl vagresrer jvgu guvaxvat be jevgvat. V srry trarenyyl sbphfrq naq ratntrq, gubhtu vg'f uneq gb gryy jurgure gung'f na rssrpg be whfg n znggre bs orvat vagrerfgrq va guvf rkcrevzrag.

Replies from: Qiaochu_Yuan
comment by Qiaochu_Yuan · 2018-02-16T19:09:19.705Z · LW(p) · GW(p)

Awesome! This would carry more weight if I had described in advance what some of the possible outcomes are, but this is consistent with the results I've seen and what few models I have. I'd be excited to continue working with you on this at the mentor workshop, and you can also message me on Facebook or wherever.

comment by Said Achmiz (SaidAchmiz) · 2018-02-16T02:16:19.484Z · LW(p) · GW(p)

Thank you very much for the detailed reply!

Consider the example of Mary’s room. …

I’m quite familiar with Mary’s Room, yes, so this was indeed a helpful analogy.

I can also demonstrate, in real time, the difference between me being in my body and not being in my body; there are things I can do in my body that are harder to do when I’m not in my body, and I have no idea how to convince you that I can do these things via text.

Video!

(Also, convincing me aside, do you mind saying what these things are?)

Oh, actually, here’s an exercise you can do right now … It goes like this: try to get a clear sense of where the “locus of control” of your body is. Probably it’s where your brain is by default. I visualize it as something like a blue ball of energy shooting out electricity to the rest of my body to make it move.

I don’t think I have one of these things… (I don’t visualize any blue balls of energy, at any rate…) I suppose this is one of those “fundamental experiences that some people are missing”, etc.

Anyway, I do understand what you mean, now, and if you manage to get the phenomenon into words I will certainly read it!

Replies from: Qiaochu_Yuan
comment by Qiaochu_Yuan · 2018-02-16T02:47:15.581Z · LW(p) · GW(p)
Video!

I thought about it but I don't expect video to be very convincing; if I'm by myself the main differences will be changes in my body language, which I could be faking. Most of the most interesting things I can do when I'm in my body involve interacting with other people, and again on video these sorts of things could be easily faked. Try looking up videos of people being hypnotized; they're not very convincing. It's much more convincing to actually be hypnotized yourself.

(Also, convincing me aside, do you mind saying what these things are?)

I'll stick to one example: there's a thing I'm learning how to do which I'll call "talking from my guts to your guts." I'm not very good at it yet and I don't have a mechanistic explanation of how it works, but what it feels like I'm doing is an interpersonal version of Focusing. It's something like, I am trying to pick up a felt sense of what a person in front of me is feeling (e.g. from their body language, but processed mostly unconsciously), mixing that with how I'm feeling, then labeling the resulting mixed felt sense. That's not quite right, though. (Words, man.)

I've done this a few times and had it done to me a few times, and when done right I think it's a powerful tool for personal growth, although those feel like lackluster words for it.

Edit: Oops, forgot to respond to this part:

I don’t think I have one of these things… (I don’t visualize any blue balls of energy, at any rate…) I suppose this is one of those “fundamental experiences that some people are missing”, etc.

I don't mean to imply that this is something you were already doing and should be noticing that you're doing; the exercise is to start doing it. That is, start trying to conjure up the sense of a locus of control, in any way you can. I don't have a visualization that reliably works for everyone, and it's not really a visualization anyway: if visualization is something you do in your visual imagination, then the thing that's relevant here is more like doing something in your proprioceptive imagination...

comment by ioannes (ioannes_shade) · 2018-02-19T16:56:51.156Z · LW(p) · GW(p)
I worry that all of the organizations you describe as working on the problem of understanding what matters are using a thinking-based methodology that is 1) heavily influenced by cultural blindspots and 2) disconnected from the direct experience of being in a human body.

I agree that the organizations I mentioned are all taking very cerebral approaches towards the question of what matters.

I think this is because in this essay, I was only considering the question of what current EA initiatives look promising under the steering capacity framework, and all current EA initiatives are thinking-based.

There are many other projects considering the question of what matters, many of them based in lived experience (e.g. Zen & Vipassana meditation traditions). Some of these are probably very valuable.

comment by Ben Pace (Benito) · 2018-02-17T10:34:48.375Z · LW(p) · GW(p)

I really like this framing! Just a few small questions about the choice of the five attributes.

You say steering capacity has five attributes, then you make parallels to these five focus areas for altruistic work in present day work. Three q's:

  • I don't get the difference between predictive power and wisdom.
  • I would've thought predictive power would match to 'improve prediction-making & foresight', not 'number of people'.
  • Do you intend any of your points to refer to the general scientific/rationality ability of civilization, insofar as you think they're separate from prediction skills?

Replies from: ioannes_shade, ioannes_shade
comment by ioannes (ioannes_shade) · 2018-02-19T16:20:45.907Z · LW(p) · GW(p)

I don't follow your other two questions (not sure what "number of people" refers to in the second; not sure how to parse the third at all). Could you clarify or restate them?

comment by ioannes (ioannes_shade) · 2018-02-19T16:18:43.508Z · LW(p) · GW(p)
I don't get the difference between predictive power and wisdom.

I am using "predictive power" as something like "ability to see what's coming down the pipe" and "wisdom" as something like "ability to assess whether what's coming down the pipe is good or bad, according to one's value system."

comment by Said Achmiz (SaidAchmiz) · 2018-02-15T19:59:48.786Z · LW(p) · GW(p)

Meta: the links to previous posts seem to be broken.

Replies from: ioannes_shade
comment by ioannes (ioannes_shade) · 2018-02-15T20:09:14.442Z · LW(p) · GW(p)

Thanks – should be fixed now.