Under what circumstances is "don't look at existing research" good advice? 2019-12-13T13:59:52.889Z · score: 71 (21 votes)
A mechanistic model of meditation 2019-11-06T21:37:03.819Z · score: 103 (32 votes)
On Internal Family Systems and multi-agent minds: a reply to PJ Eby 2019-10-29T14:56:19.590Z · score: 38 (16 votes)
Book summary: Unlocking the Emotional Brain 2019-10-08T19:11:23.578Z · score: 180 (74 votes)
Against "System 1" and "System 2" (subagent sequence) 2019-09-25T08:39:08.011Z · score: 89 (26 votes)
Subagents, trauma and rationality 2019-08-14T13:14:46.838Z · score: 68 (35 votes)
Subagents, neural Turing machines, thought selection, and blindspots 2019-08-06T21:15:24.400Z · score: 60 (20 votes)
On pointless waiting 2019-06-10T08:58:56.018Z · score: 43 (22 votes)
Integrating disagreeing subagents 2019-05-14T14:06:55.632Z · score: 90 (24 votes)
Subagents, akrasia, and coherence in humans 2019-03-25T14:24:18.095Z · score: 87 (25 votes)
Subagents, introspective awareness, and blending 2019-03-02T12:53:47.282Z · score: 62 (22 votes)
Building up to an Internal Family Systems model 2019-01-26T12:25:11.162Z · score: 154 (56 votes)
Book Summary: Consciousness and the Brain 2019-01-16T14:43:59.202Z · score: 94 (31 votes)
Sequence introduction: non-agent and multiagent models of mind 2019-01-07T14:12:30.297Z · score: 88 (33 votes)
18-month follow-up on my self-concept work 2018-12-18T17:40:03.941Z · score: 58 (17 votes)
Tentatively considering emotional stories (IFS and “getting into Self”) 2018-11-30T07:40:02.710Z · score: 39 (11 votes)
Incorrect hypotheses point to correct observations 2018-11-20T21:10:02.867Z · score: 75 (30 votes)
Mark Eichenlaub: How to develop scientific intuition 2018-10-23T13:30:03.252Z · score: 79 (30 votes)
On insecurity as a friend 2018-10-09T18:30:03.782Z · score: 38 (20 votes)
Tradition is Smarter Than You Are 2018-09-19T17:54:32.519Z · score: 68 (24 votes)
nostalgebraist - bayes: a kinda-sorta masterpost 2018-09-04T11:08:44.170Z · score: 16 (7 votes)
New paper: Long-Term Trajectories of Human Civilization 2018-08-12T09:10:01.962Z · score: 30 (15 votes)
Finland Museum Tour 1/??: Tampere Art Museum 2018-08-03T15:00:05.749Z · score: 20 (6 votes)
What are your plans for the evening of the apocalypse? 2018-08-02T08:30:05.174Z · score: 24 (11 votes)
Anti-tribalism and positive mental health as high-value cause areas 2018-08-02T08:30:04.961Z · score: 26 (10 votes)
Fixing science via a basic income 2018-08-02T08:30:04.380Z · score: 30 (14 votes)
Study on what makes people approve or condemn mind upload technology; references LW 2018-07-10T17:14:51.753Z · score: 21 (11 votes)
Shaping economic incentives for collaborative AGI 2018-06-29T16:26:32.213Z · score: 47 (13 votes)
Against accusing people of motte and bailey 2018-06-03T21:31:24.591Z · score: 84 (29 votes)
AGI Safety Literature Review (Everitt, Lea & Hutter 2018) 2018-05-04T08:56:26.719Z · score: 37 (10 votes)
Kaj's shortform feed 2018-03-31T13:02:47.793Z · score: 13 (3 votes)
Helsinki SSC March meetup 2018-03-26T19:27:17.850Z · score: 12 (2 votes)
Is the Star Trek Federation really incapable of building AI? 2018-03-18T10:30:03.320Z · score: 29 (8 votes)
My attempt to explain Looking, insight meditation, and enlightenment in non-mysterious terms 2018-03-08T07:37:54.532Z · score: 286 (110 votes)
Some conceptual highlights from “Disjunctive Scenarios of Catastrophic AI Risk” 2018-02-12T12:30:04.401Z · score: 63 (18 votes)
On not getting swept away by mental content 2018-01-25T20:30:03.750Z · score: 24 (8 votes)
Papers for 2017 2018-01-04T13:30:01.406Z · score: 32 (8 votes)
Paper: Superintelligence as a Cause or Cure for Risks of Astronomical Suffering 2018-01-03T14:39:18.024Z · score: 1 (1 votes)
Paper: Superintelligence as a Cause or Cure for Risks of Astronomical Suffering 2018-01-03T13:57:55.979Z · score: 16 (6 votes)
Fixing science via a basic income 2017-12-08T14:20:04.623Z · score: 38 (11 votes)
Book review: The Upside of Your Dark Side: Why Being Your Whole Self–Not Just Your “Good” Self–Drives Success and Fulfillment 2017-12-04T13:10:06.995Z · score: 27 (8 votes)
Meditation and mental space 2017-11-06T13:10:03.612Z · score: 26 (7 votes)
siderea: What New Atheism says 2017-10-29T10:19:57.863Z · score: 12 (3 votes)
Postmodernism for rationalists 2017-10-17T12:20:36.139Z · score: 24 (1 votes)
Anti-tribalism and positive mental health as high-value cause areas 2017-10-17T10:20:03.359Z · score: 30 (10 votes)
You can never be universally inclusive 2017-10-14T11:30:04.250Z · score: 34 (10 votes)
Meaningfulness and the scope of experience 2017-10-05T11:30:03.863Z · score: 35 (14 votes)
Social Choice Ethics in Artificial Intelligence (paper challenging CEV-like approaches to choosing an AI's values) 2017-10-03T17:39:00.683Z · score: 8 (3 votes)
LW2.0 now in public beta (you'll need to reset your password to log in) 2017-09-23T12:00:50.345Z · score: 2 (2 votes)
Nobody does the thing that they are supposedly doing 2017-09-23T10:40:06.155Z · score: 69 (34 votes)


Comment by kaj_sotala on Reality-Revealing and Reality-Masking Puzzles · 2020-01-17T13:57:58.787Z · score: 5 (2 votes) · LW · GW

I want a similarly clear-and-understood generalization of the “reasoning vs rationalizing” distinction that applies also to processes to spread across multiple heads. I don’t have that yet. I would much appreciate help toward this.

I feel like Vaniver's interpretation of self vs. no-self is pointing at a similar thing; would you agree?

I'm not entirely happy with any of the terminology suggested in that post; something like "seeing your preferences realized" vs. "seeing the world clearly" would in my mind be better than either "self vs. no-self" or "design specifications vs. engineering constraints".

In particular, Vaniver's post makes the interesting contribution of pointing out that while "reasoning vs. rationalization" suggests that the two would be opposed, seeing the world clearly vs. seeing your preferences realized can be opposed, mutually supporting, or orthogonal. You can come to see your preferences more realized by deluding yourself, but you can also deepen both, seeing your preferences realized more because you are seeing the world more clearly.

In that ontology, instead of something being either reality-masking or reality-revealing, it can

  • A. Cause you to see your preferences more realized and the world more clearly
  • B. Cause you to see your preferences more realized but the world less clearly
  • C. Cause you to see your preferences less realized but the world more clearly
  • D. Cause you to see your preferences less realized and the world less clearly

But the problem is that a system facing a choice between several options has no general way to tell whether some option it could take is actually an instance of A, B, C or D or if there is a local maximum that means that choosing one possiblity increases one variable a little, but another option would have increased it even more in the long term.

E.g. learning about the Singularity makes you see the world more clearly, but it also makes you see that fewer of your preferences might get realized than you had thought. But then the need to stay alive and navigate the Singularly successfully, pushes you into D, where you are so focused on trying to invest all your energy into that mission that you fail to see how this prevents you from actually realizing any of your preferences... but since you see yourself as being very focused on the task and ignoring "unimportant" things, you think that you are doing A while you are actually doing D.

Comment by kaj_sotala on Toon Alfrink's sketchpad · 2020-01-17T09:51:32.506Z · score: 3 (1 votes) · LW · GW

Well, it sounds to me like it's more of a heterarchy than a hierarchy, but yeah.

Comment by kaj_sotala on Toon Alfrink's sketchpad · 2020-01-16T19:25:23.623Z · score: 3 (1 votes) · LW · GW

This BBC article discusses it a bit:

There is a further problem with Maslow's work. Margie Lachman, a psychologist who works in the same office as Maslow at his old university, Brandeis in Massachusetts, admits that her predecessor offered no empirical evidence for his theory. "He wanted to have the grand theory, the grand ideas - and he wanted someone else to put it to the hardcore scientific test," she says. "It never quite materialised."

However, after Maslow's death in 1970, researchers did undertake a more detailed investigation, with attitude-based surveys and field studies testing out the Hierarchy of Needs.

"When you analyse them, the five needs just don't drop out," says Hodgkinson. "The actual structure of motivation doesn't fit the theory. And that led to a lot of discussion and debate, and new theories evolved as a consequence."

In 1972, Clayton Alderfer whittled Maslow's five groups of needs down to three, labelled Existence, Relatedness and Growth. Although elements of a hierarchy remain, "ERG theory" held that human beings need to be satisfied in all three areas - if that's not possible then their energies are redoubled in a lower category. So for example, if it is impossible to get a promotion, an employee might talk more to colleagues and get more out of the social side of work.

More sophisticated theories followed. Maslow's triangle was chopped up, flipped on its head and pulled apart into flow diagrams.

Of course, this doesn't really contradict your point of there being separable, factorable goals. AFAIK, the current mainstream model of human motivation and basic needs is self-determination theory, which explicitly holds that there exist three separate basic needs:

Autonomy: people have a need to feel that they are the masters of their own destiny and that they have at least some control over their lives; most importantly, people have a need to feel that they are in control of their own behavior.
Competence: another need concerns our achievements, knowledge, and skills; people have a need to build their competence and develop mastery over tasks that are important to them.
Relatedness (also called Connection): people need to have a sense of belonging and connectedness with others; each of us needs other people to some degree

Comment by kaj_sotala on Ascetic aesthetic · 2020-01-15T14:29:13.846Z · score: 4 (2 votes) · LW · GW

Well there's that too. I liked this take on it:

Everything is experienced through a series of filters. Filters are created by instinct and experience… The entire world looks and feels COMPLETELY different for each person, because no ones’ set of filters is the same.

The “deeper” the filter, the more influence it has over your perception of reality. Evolutionarily developed filters (instinct) occurred from millions of years of selection, and are very deep. Filters originated in your childhood that survive into adulthood are generally deep, etc. Someone calling you a mean word adds a filter that might last an hour.

These filters are stacked on top of each other like a house of cards. The deeper the filter, the more influential. But every filter effects one’s perception of reality… There are deep filters based on physical brain chemistry, instinct based on human evolution, etc. Layers of filter can be peeled away to see reality in a more “pure” way. Peeling away layers allows one to see things(reality) in a way they didn’t before, and reconsider “their” reality. Note: peeling away layers does not necessarily mean one will then go on to form a truer view of reality.

Some of the filters defining what's beautiful and what's ugly are going to be learned and others innate, with the learned ones being formed on the basis of the innate ones.

Comment by kaj_sotala on Circling as Cousin to Rationality · 2020-01-14T20:00:38.365Z · score: 6 (2 votes) · LW · GW

That makes sense, thanks!

Comment by kaj_sotala on Ascetic aesthetic · 2020-01-14T19:32:33.809Z · score: 9 (4 votes) · LW · GW

I generally agree, but I think that there's also a sense in which aesthetics comes from facts. See Propagating Facts into Aesthetics (which is exactly about that), Identities are (Subconscious) Strategies (one's sense of identity often includes lots of aesthetic considerations as well), and Book summary: Unlocking the Emotional Brain (the kind of emotional learning described there probably drives many of these aesthetics).

Comment by kaj_sotala on Circling as Cousin to Rationality · 2020-01-14T12:03:57.016Z · score: 6 (2 votes) · LW · GW

Would be curious to hear how well you and Vaniver think that my recent post on meditation makes the case for "meditation as a form of empiricism".

Comment by kaj_sotala on Key Decision Analysis - a fundamental rationality technique · 2020-01-12T16:34:42.586Z · score: 5 (3 votes) · LW · GW

I find that the greatest challenge in starting to employ something like this, is learning to recognize the things that count as decisions to be recorded. To the extent that they are not too private, could you share more examples of the kinds of decisions that you have used this on?

Comment by kaj_sotala on Is there a moral obligation to respect disagreed analysis? · 2020-01-11T09:28:12.869Z · score: 8 (4 votes) · LW · GW

This seems abstracted enough to make it impossible to answer, as it depends on the nature of your relationship, the action, and the harm. E.g. if you are risking direct bodily injury to P, then he would have much more of a right to veto it than if your action was something like "I will carry out research into a controversial topic which might end up upsetting other people and make them angry at me, and also angry at P by association since we are known to be friends".

But in general, would you want someone else to unilaterally carry out an action that you felt had a high risk of harming you, just because you failed to convince them of this being the case?

Comment by kaj_sotala on Circling as Cousin to Rationality · 2020-01-09T11:21:48.168Z · score: 6 (2 votes) · LW · GW

I would need an example of this, before I could say for sure what I think of it.

In my experience, this kind of a thing tends to come up when there has been no explicit agreement about something, but previous experience implies a particular thing, and the other person knows that this matters for the other.

For example, say that Alice is Bob's aging mother who is lonely in her old days. Bob has explicitly promised to visit her every week. Over time, this has ended up usually meaning twice a week, on Tuesdays and Fridays.

Now one week it happens that Bob visits on Tuesday, as normal, and doesn't say anything about any changes to the normal schedule. Then on Thursday, when Alice asks what kind of dinner Bob would want to have on Friday, Bob says "oh, I'm not coming this Friday, I'll see you next Tuesday".

In this situation, Alice might on an intellectual level think that there was no betrayal. The explicit agreement was for Bob to visit once a week, and he never promised anything else. It just kind of happened that Bob ended up visiting more regularly, but he never made a promise to visit every Friday. Nor did he on Tuesday say that he would visit next Friday. Alice just kind of ended up assuming that he would, like usual.

On the other hand, she may still feel betrayed, in that she had expected Bob to visit on Friday. In particular, there may be a feeling that Bob should have known that based on him having visited on every Friday for the last six weeks, Alice would expect him to visit the coming Friday as well. Alice may feel that Bob should have understood his mother well enough to know that unless Bob specifically says that he will not be coming, Alice will plan her week under the assumption that he is coming. (Depending on how introspective Alice is, she may not be able to articulate all this, and just feel that "I know that Bob never said that he would come on Friday, but I still feel betrayed".)

Comment by kaj_sotala on Circling as Cousin to Rationality · 2020-01-09T10:48:13.801Z · score: 8 (3 votes) · LW · GW

Aren't those kind of the same thing, though? In that before you can ask yourself whether your thing is bad, you need to understand the criticism in question, and that requires verifying that your interpretation of the criticism is correct before you proceed.

It's true that these sometimes come apart: e.g. maybe I have an irrational fear of AI, but that irrational motive can still drive me to formulate correct arguments for AI risk. But in that case there exists a clean separation between the motive and the object-level argument. Whereas in this case, Bunthut seemed to be articulating reasons behind their emotional discomfort.

If you are trying to check that you correctly understood what someone is saying about their emotional discomfort, then that doesn't seem like a case where you can isolate an object-level argument that would be separate from "why the person said that". They are trying to express discomfort about something, and the specific reason why the thing is making them uncomfortable is the object-level issue.

Comment by kaj_sotala on Dominic Cummings: "we’re hiring data scientists, project managers, policy experts, assorted weirdos" · 2020-01-07T15:59:19.317Z · score: 9 (4 votes) · LW · GW

This conversation sounds like TAG uses utilitarianism to mean classic utilitarianism, where pains and pleasures are the only consequences that we care about (and rights violations are not), and like you are using it to refer to decision-theoretic utilitarianism, where the consequences can include rights violations as well.

Comment by kaj_sotala on Making decisions under moral uncertainty · 2020-01-04T13:01:36.022Z · score: 4 (2 votes) · LW · GW

Thanks! This bit in particular

I think the main way it would help advance one's thinking is by giving clues as to what one should do to resolve one's uncertainty. [...] But what I cover in this post and the following one is how to make decisions when one is morally uncertain. I.e., imagining that you are stuck with uncertainty, what do you do? This is a different question to "how do I get rid of this uncertainty and find the best answer" (resolving it).

Makes sense to me, and clarified your approach. I think I agree with it.

Comment by kaj_sotala on Making decisions under moral uncertainty · 2020-01-02T11:36:13.932Z · score: 3 (1 votes) · LW · GW

Part of me wants to say I don't think there's really that much to say about moral uncertainty itself, before getting into how to handle it.

I'm confused by you saying this, given that you indicate having read my post on types of moral uncertainty. To me the different types warrant different ways of dealing with them. For example, intrinsic moral uncertainty was defined as different parts of your brain having fundamental disagreements about what kind of a value system to endorse. That kind of situation would require entirely different kinds of approaches, ones that would be better described as psychological than decision-theoretical.

It seems to me that before outlining any method for dealing with moral uncertainty, one would need to outline what type of MU it was applicable for and why.

Comment by kaj_sotala on 2010s Predictions Review · 2019-12-31T10:04:11.236Z · score: 15 (6 votes) · LW · GW

I feel like the rate of subculture formation has actually gone down from what it used to be in the 00's. Rather than people forming small visible tribes (e.g. furries), most get sucked into large-scale visible tribes (e.g. social justice) instead.

Comment by kaj_sotala on Kaj's shortform feed · 2019-12-28T20:34:49.744Z · score: 39 (13 votes) · LW · GW

Occasionally I find myself nostalgic for the old, optimistic transhumanism of which e.g. this 2006 article is a good example. After some people argued that radical life extension would increase our population too much, the author countered that oh, that's not an issue, here are some calculations showing that our planet could support a population of 100 billion with ease!

In those days, the ethos seemed to be something like... first, let's apply a straightforward engineering approach to eliminating aging, so that nobody who's alive needs to worry about dying from old age. Then let's get nanotechnology and molecular manufacturing to eliminate scarcity and environmental problems. Then let's re-engineer the biosphere and human psychology for maximum well-being, such as by using genetic engineering to eliminate suffering and/or making it a violation of the laws of physics to try to harm or coerce someone.

So something like "let's fix the most urgent pressing problems and stabilize the world, then let's turn into a utopia". X-risk was on the radar, but the prevailing mindset seemed to be something like "oh, x-risk? yeah, we need to get to that too".

That whole mindset used to feel really nice. Alas, these days it feels like it was mostly wishful thinking. I haven't really seen that spirit in a long time; the thing that passes for optimism these days is "Moloch hasn't entirely won (yet)". If "overpopulation? no problem!" felt like a prototypical article to pick from the Old Optimistic Era, then Today's Era feels more described by Inadequate Equilibria and a post saying "if you can afford it, consider quitting your job now so that you can help create aligned AI before someone else creates unaligned AI and kills us all".

Today's philosophy seems more like "let's try to ensure that things won't be quite as horrible as they are today, and if we work really hard and put all of our effort into it, there's a chance that maybe we and all of our children won't die." Most of the world-saving energy seems to have gone into effective altruism, where people work on issues like making the US prison system suck less or distributing bednets to fight malaria. (Causes that I thoroughly support, to be clear, but also ones where the level of ambition seems quite a bit lower than in "let's make it a violation of the laws of physics to try to harm people".)

I can't exactly complain about this. Litany of Tarski and alll: if the Old Optimistic Era was hopelessly naive and over-optimistic, then I wish to believe that it was hopelessly naive and over-optimistic, and believe in the more realistic predictions instead. And it's not clear that the old optimism ever actually achieved much of anything in the way of its grandiose goals, whereas more "grounded" organizations such as GiveWell have achieved quite a lot.

But it still feels like there's something valuable that we've lost.

Comment by kaj_sotala on We need to revisit AI rewriting its source code · 2019-12-28T00:15:53.377Z · score: 12 (6 votes) · LW · GW

I'm confused. I read you as suggesting that self-modifying code has recently become possible, but I think that self-modifying code has been possible for about as long as we have had digital computers?

What specific things are possible to do now that weren't possible before, and what kind of AGI-relevant questions does that make testable?

Comment by kaj_sotala on The Curse Of The Counterfactual · 2019-12-26T11:34:43.734Z · score: 5 (2 votes) · LW · GW

Follow-up: I continued working on this issue using the approach that you outlined here. Eventually I figured out that the sense of urgency wasn't so much "I won't have the time to get enough work done" but rather "I won't have the time to get enough work done and then relax properly after that". (There might have been some other schemas which were using the sense of urgency too, which got reconsolidated during the process.) After figuring that out, I haven't had a major issue with it.

Since that was an issue that IFS etc. had failed to make a dent on for years, I then started throwing The Work/Coherence Therapy/my-model-of-your-model-as-interpreted-through-your-public-writing on a lot of other things too, and have made varying amounts of progress on at least twelve of them. Agree with you more on the weaknesses of IFS now.

Comment by kaj_sotala on We run the Center for Applied Rationality, AMA · 2019-12-25T08:47:51.029Z · score: 5 (3 votes) · LW · GW

Or are just otherwise too busy with their life to have the time for meetups.

Comment by kaj_sotala on We run the Center for Applied Rationality, AMA · 2019-12-21T14:12:45.835Z · score: 34 (12 votes) · LW · GW

Whether CFAR staff (qua CFAR staff, as above) will help educate people who later themselves produce explicit knowledge in the manner valued by Gwern, Wei Dai, or Scott Alexander, and who wouldn’t have produced (as much of) that knowledge otherwise.

This seems like a good moment to publicly note that I probably would not have started writing my multi-agent sequence without having a) participated in CFAR's mentorship training b) had conversations with/about Val and his posts.

Comment by kaj_sotala on My attempt to explain Looking, insight meditation, and enlightenment in non-mysterious terms · 2019-12-12T11:38:31.022Z · score: 15 (4 votes) · LW · GW

I still broadly agree with everything that I said in this post. I do feel that it is a little imprecise, in that I now have much more detailed and gears-y models for many of its claims. However, elaborating on those would require an entirely new post (one which I currently working on) with a sequence's worth of prerequisites. So if I were to edit this post, I would probably mostly leave it as it is, but include a pointer to the new post once it's finished.

In terms of this post being included in a book, it is worth noting that the post situates itself in the context of Valentine's Kensho post, which has not been nominated for the review and thus wouldn't be included in the book. So if this post were to be included, I should probably edit this so as to not require reading Kensho.

Comment by kaj_sotala on The LessWrong 2018 Review · 2019-12-08T22:26:39.704Z · score: 18 (5 votes) · LW · GW

Occasionally I think about writing a review, but then feel like I'm too confused to do so.

Some of my open questions:

  • I'm unsure of what to write. The post says that "A good frame of reference for the reviews are shorter versions of LessWrong or SlatestarCodex book reviews (which do a combination of epistemic spot checks, summarizing, and contextualizing)", but this feels like weird advice for reviewing a blog post, which is much shorter than a book. Especially the "summarizing" bit - for most posts the content is already too short for further summarizing to make sense. This guideline confuses me more than it helps.
  • If I just ignore the guideline and think about what would make sense to me, it would be... something like my longer nomination comments. But I already posted those as nominations. Should I re-post some of them as reviews? That seems silly.
  • I don't know which posts I should review. I won't have the chance to review all of them, so I should pick just a few. But which ones? The post says "Posts that got at least one review proceed to the voting phase", which makes it sound like reviews are like nominations / votes; a post won't be included unless it gets at least one vote. That creates an incentive for me not to review posts I don't like, since even a critical review might cause it to get to the voting stage. So I should probably focus on reviewing the posts that I like. That conclusion does not seem like it's what was intended, though.
  • Also, I'm not sure of how to review posts that I didn't like. The posts that got to this stage are generally decent quality, and I don't have major criticisms of them. If I don't think that something should be included in a collection of best posts, then my reason is generally "I didn't seem to have gotten any lasting value out of it". But someone else did, or else it would not have been nominated. There's no point in me posting a review saying "I didn't get lasting value out of this, but of course someone else might have".
Comment by kaj_sotala on The Actionable Version of "Keep Your Identity Small" · 2019-12-07T12:29:45.192Z · score: 9 (5 votes) · LW · GW

Related: Identities are (Subconscious) strategies

Identities are Strategies towards Goals

Consider a person who prides themselves on their identity as a writer: “I am a writer.” This identity is precious because there is an implicit statement of the form “I am a writer[, and therefore I will have a job, income, status, friends, lovers, and my life will be good].” The implicit statement is the goal to be obtained and the explicit identity is the strategy for achieving that goal. The value of the identity derives from the goal is supports.

I describe these plans as subconscious because more often than not they are not articulated. Many people have an identity around being intelligent, but I expect that if you ask them why this important, they will need a few moments to generate their answer. I also expect that in many cases the belief in the goodness of an identity is absorbed from society and it is social drives which motivate it for an individual. In that case, the full identity statement might go “I am a __ [and therefore society will approve of me]” whether or not an individual would admit it. In the most general case, it’s “I am a __ [and therefore goodness].”

Threats to the Identity are Threats to the Goal

Given that an identity is a strategy for achieving a goal, any threat to the identity is a threat to the goal. The degree of threat perceived is proportional to the importance of the goal and to the extent that the identity is sole strategy for achieving the goal. If someone believes that being a writer is their sole avenue for having a good and fulfilling life, they are going to get upset when that identity is challenged. This holds even if person does not consciously recognize that their identity is part of a plan. It is enough that some part of their mind, S1 or whatever, has firmly stamped “being a writer” as critical for having a good life.

Consider, though, someone who has identities both around being a writer and around being a musician. Suppose that this person has achieved considerable fame and fortune as a musician and resultantly already has wealth, friends, lovers, etc. by dint of this identity alone. I predict that this person will be less bothered by challenges to writing ability than the person who staking themselves on being a writer. If the writer-only has their manuscript rejected, it will be devastating, whereas for the writer-musician, it will be a mere disappointment.

Protect the Goal and the Identity Can Be Free

If threats to identity are really about threats to goal-attainment, then the key to working with identities becomes a) surfacing the hidden goals and, b) ensuring there is security around attaining those goals. Tell the child that they’re not cut out to be writer and they’ll tantrum, but tell them they’re not cut to be a writer yet have phenomenal painting skills, and they might just listen. Substitute one less viable plan for a new and better one. Other variations include exposing that the goal in fact has already been attained, as in the case of the writer-musician above, or recognizing that the identity in fact is going to be an ineffective plan regardless, e.g. giving up on being a goth because you realize that no one thinks goths are cool anyway.

Compare also Richard, whose brain found it really important that "confident" not be a part of his identity.

Comment by kaj_sotala on What is Abstraction? · 2019-12-07T10:03:09.596Z · score: 3 (1 votes) · LW · GW

Worth noting that 'abstraction' has different meanings in different disciplines. For example, Wikipedia has separate articles for abstraction in computer science, abstraction in mathematics, abstraction in linguistics, and abstraction in sociology.

Comment by kaj_sotala on Affordance Widths · 2019-12-07T06:52:20.266Z · score: 18 (6 votes) · LW · GW

Link, link.

Comment by kaj_sotala on A mechanistic model of meditation · 2019-12-01T21:19:06.800Z · score: 7 (3 votes) · LW · GW

Thank you very much for your kind words. :)

"Introspective awareness" sounds like the right object. Or, more specifically, it definitely feels like it's describing my own experience. And my own, homegrown hypothesis was something like: consciousness is like an echo or picture-in-picture. We can get glimpses of "ourselves" because we can look at / load partial concepts of ourselves into the working memory.

Yeah, that sounds like it's talking about the same thing. I also quite liked the way Mark Lippmann talks about "afterimages", which seems to also be very closely related:

When you look, see, or notice something, there’s a very predictable pattern that then occurs. First, there is your contact with the actual sensory experience. This is very, very brief. Almost immediately, your mind moves to phase two.

In this phase, you are no longer paying attention to the actual sensory experience, but you are instead paying attention to a sort of “afterimage” of the experience. This is what your mind actually collects and takes away from the outside world, and this is what you actually think about, make sense of, and reason about. [...]

Another way to get a sense of afterimages is to generate a short sound or some other sensory experience and then ask how you know it happened. For example, snap your fingers. Ok. How do you know you just snapped your fingers? You remember you did, right?

Unless you wait too long, part of the experience of that memory is the afterimage of you snapping your fingers. And, there’s often a special property of afterimages that you can play with: You can access the afterimage to more fully replay the experience that led to the afterimage. A replay is not available for some experiences and you might lose the replay for the experience if you wait too long before accessing it. Finally, even if a replay isn’t available, the afterimage may still contain some detail that you can inspect.

So, being aware of and using afterimages is one way that you can inspect subtle phenomena, especially phenomena that goes by very fast.

When doing so, there are some caveats to be aware of.

First, it’s good to remember that the afterimage is not a perfect replica of the experience. It is a “tag” that the experience happened, that may contain or evoke some of the structure or phenomenology of the original experience. If you’re using afterimages to investigate experience, you have to make some effort to to separate out what the experience of the afterimage is versus what remains of the original experience.

Second, it’s important to note that afterimages will always have some conceptual contamination. Afterimages are part top down and part bottom up. That is, afterimages are partially composed of what you expect to see. That’s why you can be positive you just saw a bug skitter across the flow but when you look closely it was just some very suggestive dust caught in a draft. The afterimage is what your reflexes and emotions actually react to, and the afterimage is not the same thing as what was actually there. The way to partially get around this is to try to not have preconceptions and to try to take lots of careful observations of the phenomena.

Finally, there’s a subtler point, here. It seems to be the case that you may be able to “take” or “get” an afterimage only if you already have some inkling of what you’re looking for. That is, if you already have some hint of an idea or concept of what’s there. That doesn’t mean you have to have a name for the experience. And, it doesn’t mean that you’ve had to explicitly reflect, before, on some prior occasion, on having those sorts of experiences. I just means that somewhere in your mind there has to be some sort of… familiarity for the experience before you go looking or paying attention in general.

So, how do you get that initial experience, if you can only have the experience if you’ve had the experience? It seems to “bootstrap” slowly, by simply paying attention in the vicinity of what you’re looking for. You brain eventually, faintly discerns a pattern on the edge of experience, and you gain a creeping sense of familiarity that becomes clearer and clearer, until finally you can put your finger on it, haltingly describe it with great difficulty, and maybe finally name it as a thing or break it down into further parts.

Comment by kaj_sotala on A Practical Theory of Memory Reconsolidation · 2019-12-01T15:39:50.877Z · score: 6 (2 votes) · LW · GW

Do you know any person in your life (not necessarily a close one, just someone who you've met at least occasionally) who feels like they express warmth and honesty at the same time? If you do, looking at how they express themselves is one way of getting pointers.

It might also be worth checking whether your problem is just about not knowing what to do, versus your brain having an outright objection to getting what you want. If you can, try to imagine what it would feel like to get love and affection, in exactly the way you would like to have. Take a moment to imagine it in as much detail as you can. Then pay attention to how it feels - is there any trace of unease, of anything not feeling quite right about you having achieved it?

Comment by kaj_sotala on A Practical Theory of Memory Reconsolidation · 2019-12-01T15:13:15.586Z · score: 7 (3 votes) · LW · GW

You mean a concrete example of some LW reader doing memory reconsolidation on themselves? I'm applying updates on a regular basis, though they are mostly small tweaks rather than really big stuff. Can report on some if people are interested.

Comment by kaj_sotala on What are the requirements for being "citable?" · 2019-11-30T07:28:27.868Z · score: 3 (1 votes) · LW · GW

No citations are listed, probably because of the formatting.

Comment by kaj_sotala on What are the requirements for being "citable?" · 2019-11-29T07:31:08.901Z · score: 8 (4 votes) · LW · GW

Google Scholar has already indexed (the AF version of) at least one of my posts.

Comment by kaj_sotala on 2018 AI Alignment Literature Review and Charity Comparison · 2019-11-28T17:46:36.454Z · score: 7 (3 votes) · LW · GW

Excellent and useful review. Definitely the kind of thing that I would like to encourage in the future, and which also holds historical interest - what kind of progress was made in 2018?

Comment by kaj_sotala on Mental Mountains · 2019-11-27T11:08:20.644Z · score: 10 (3 votes) · LW · GW

Exploring the connection to politics a bit more, Coherence Therapy: Practice Manual And Training Guide has this page where it claims that emotional learning forms our basic assumptions for a wide variety of domains, including ones that we would commonly think of as being the domains of rationality:


Unconscious constructs constituting people's pro-symptom positions tend to be constructs that define these areas of personal reality and felt meaning:

  • The essential nature of self/others/world (ontology/identity)
  • The necessary direction or state of affairs to pursue (purpose, teleology)
  • What necessarily results in what (causality)
  • How to be connected with others; how attachment works (attachment/boundaries)
  • How self-expression operates (identity/selfhood/boundaries/creativity)
  • Where to place responsibility and blame (causality, morality)
  • What is good and what is bad; what is wellness and what is harm; what is safety and what is danger (safety/values/morality)
  • How knowing works; how to know something (epistemology)
  • The way power operates between people (power/autonomy/dominance/survival)
  • What I am owed or what I owe (justice/accountability/duty/loyalty/entitlement)

Examples (verbalizations of unconscious, nonverbal constructs/schemas held in the limbic system and body)

Ontology: "People are attackers. If they see me, they'll try to kill me."
Causality: "If too much is going well for me, that will make a big blow happen to me."
Purpose: "I've got to keep Dad from withdrawing his love from by never, ever disagreeing with him."
Attachment: "I'll get attention and connection only if I'm visibly unwell, failing, hurting." "You'll reject and disconnect from me if I differ from you in any way."
Values: "It is selfish and bad to pay attention to my own feelings, needs and views; it is unselfish and good to be what others want me to be."
Power: "The one who has the power in a personal relationship is the one who withdraws love; the other is the powerless one."


It seems pretty easy to take some of those examples and see how they, or something like them, could form the basis of ideologies. E.g. "people are attackers" could drive support for authoritarian policing and hawkish military policy, with elaborate intellectual structures being developed to support those conclusions. On the other side, "people are intrinsically good and trustworthy" could contribute support to opposite kinds of policies. (Just to be clear, I'm not taking a position on which one of those policies is better nor saying that they are equally good, just noting that there are emotional justifications which could drive support for either one.)

That might be one of the reasons why you don't see "I know that X is correct, but can't bring myself to support it" in politics so much. For things like "will you be hated if you speak up", there's much more of a consensus position; most people accept on an intellectual level that speaking up doesn't make people hated, because there's no big narrative saying the opposite. But for political issues, people have developed narratives to support all kinds of positions. In that case, if you have a felt position which feels true, you can often find a well-developed intellectual argument which has been produced by other people with the same felt position, so it resonates strongly with your intuitions and tells you that they are right.

This could also be related to the well-known thing where people in cities tend to become more liberal: different living conditions give rise to different kinds of implicit learning, changing the kinds of ideologies that feel plausible.

Comment by kaj_sotala on Mental Mountains · 2019-11-27T10:42:11.254Z · score: 11 (7 votes) · LW · GW
Kaj Sotala has an outstanding review of Unlocking The Emotional Brain; I read the book, and Kaj’s review is better.

^_^ <3 ^_^

Richard might be able to say “I know people won’t hate me for speaking, but for some reason I can’t make myself speak”, whereas I’ve never heard someone say “I know climate change is real, but for some reason I can’t make myself vote to prevent it.” I’m not sure how seriously to take this discrepancy.

I haven't heard this either, but I have heard (and experienced) "I know that eating meat is wrong, but for some reason I can't make myself become a vegetarian". Jonathan Haidt uses this as an example of an emotional-rational valley in The Happiness Hypothesis:

During my first year of graduate school at the University of Pennsylvania, I discovered the weakness of moral reasoning in myself. I read a wonderful book—Practical Ethics—by the Princeton philosopher Peter Singer. Singer, a humane consequentialist, shows how we can apply a consistent concern for the welfare of others to resolve many ethical problems of daily life. Singer's approach to the ethics of killing animals changed forever my thinking about my food choices. Singer proposes and justifies a few guiding principles: First, it is wrong to cause pain and suffering to any sentient creature, therefore current factory farming methods are unethical. Second, it is wrong to take the life of a sentient being that has some sense of identity and attachments, therefore killing animals with large brains and highly developed social lives (such as other primates and most other mammals ) is wrong, even if they could be raised in an environment they enjoyed and were then killed painlessly. Singer's clear and compelling arguments convinced me on the spot, and since that day I have been morally opposed to all forms of factory farming. Morally opposed, but not behaviorally opposed. I love the tast e of meat, and the only thing that changed in the first six months after reading Singer is that I thought about my hypocrisy each time I ordered a hamburger.

But then, during my second year of graduate school, I began to study the emotion of disgust, and I worked with Paul Rozin, one of the foremost authorities on the psychology of eating. Rozin and I were trying to find video clips to elicit disgust in the experiments we were planning, and we met one morning with a research assistant who showed us some videos he had found. One of them was Faces of Death, a compilation of real and fake video footage of people being killed. (These scenes were so disturbing that we could not ethically use them.) Along with the videotaped suicides and executions, there was a long sequence shot inside a slaughterhouse. I watched in horror as cows, moving down a dripping disassembly line, were bludgeoned, hooked, and sliced up. Afterwards, Rozin and I went to lunch to talk about the project. We both ordered vegetarian meals. For days afterwards, the sight of red meat made me queasy. My visceral feelings now matched the beliefs Singer had given me. The elephant now agreed with the rider, and I became a vegetarian. For about three weeks. Gradually, as the disgust faded, fish and chicken reentered my diet. Then red meat did, too, although even now, eighteen years later, I still eat less red meat and choose nonfactory-farmed meats when they are available.

That experience taught me an important lesson. I think of myself as a fairly rational person. I found Singer's arguments persuasive. But, to paraphrase Medea's lament (from chapter 1): I saw the right way and approved it, but followed the wrong, until an emotion came along to provide some force.
Comment by kaj_sotala on Kaj's shortform feed · 2019-11-26T13:01:38.893Z · score: 6 (3 votes) · LW · GW

Recent papers relevant to earlier posts in my multiagent sequence:

Understanding the Higher-Order Approach to Consciousness. Richard Brown, Hakwan Lau, Joseph E.LeDoux. Trends in Cognitive Sciences, Volume 23, Issue 9, September 2019, Pages 754-768.

Reviews higher-order theories (HOT) of consciousness and their relation to global workspace theories (GWT) of consciousness, suggesting that HOT and GWT are complementary. Consciousness and the Brain, of course, is a GWT theory; whereas HOT theories suggest that some higher-order representation is (also) necessary for us to be conscious of something. I read the HOT models as being closely connected to introspective awareness; e.g. the authors suggest a connection between alexityhmia (unawareness of your emotions) and abnormalities in brain regions related to higher-order representation.

While the HOT theories seem to suggest that you need higher-order representation of something to be conscious of a thing, I would say that you need higher-order representation of something in order to be conscious of having been conscious of something. (Whether being conscious of something without being conscious of being conscious of it can count as being conscious of it, is of course an interesting philosophical question.)

Bridging Motor and Cognitive Control: It’s About Time! Harrison Ritz, Romy Frömer, Amitai Shenhav. Trends in Cognitive Sciences, in press.

I have suggested that control of thought and control of behavior operate on similar principles; this paper argues the same.

We often describe our mental states through analogy to physical actions. We hold something in mind or push it out of our thoughts. An emerging question in cognitive control is whether this relationship runs deeper than metaphor, with similar cognitive architectures underpinning our ability to control our physical actions and our mental states. For instance, recent work has shown that analogous control processes serve to optimize performance and regulate brain dynamics for both motor and cognitive actions [1,2]. A new study by Egger and colleagues [3] provides important new clues that the mechanisms supporting motor and cognitive control are more similar than previously shown.

These researchers tested whether the control of internal states exhibits a signature property of the motor system: the reliance on an internal model to guide adjustments of control [4]. To control one’s actions, a person needs to maintain an internal model of their environment (e.g., potential changes in terrain or atmosphere) and of their own motor system (e.g., how successful they are at executing a motor command [5]). This model can be used to generate online predictions about the outcome of an action and to course-- correct when there is a mismatch between that prediction and the actual outcome. This process is thought to be implemented via interactions between: (i) a simulator that makes predictions, (ii) an estimator that learns the current state, and (iii) a controller that implements actions. This new study investigated whether neural activity during the control of cognitive processes reflected this same three-part architecture.

To answer this question, Egger and colleagues recorded neural activity while monkeys performed an interval reproduction task (Figure 1). The monkeys observed two samples of a time interval and then timed a saccade to reproduce this interval. Previous work has shown that population-level neural activity in the dorsomedial frontal cortex (DMFC) during similar tasks systematically scales with the timing of an action [6]. If action timing in this task depends on an internal model, then this temporal scaling should already be present in DMFC activity prior to receiving a cue to respond. If the monkeys were not relying on an internal model, and the activity instead reflected the passive measurement of time (‘open-loop’ control), then DMFC activity during the second interval should not exhibit such temporal scaling.

The monkeys’ behavior and neural activity demonstrated that they combined prior knowledge about the average interval duration with their perception of the current interval duration [7]. This behavior was well-captured by a nearoptimal Bayesian algorithm that updated predictions in a way that was biased towards the average interval. By independently varying the duration of the two sample intervals, the authors were further able to show that the monkeys incorporated both samples into their duration estimate.

Signatures of this biased updating process were also observed in DMFC neural activity. Replicating previous studies, individual neurons in the DMFC demonstrated ramping activity during the reproduction of an interval, with faster ramping when the monkey reproduced shorter intervals [6]. Critically, neural activity during the second sample interval exhibited the predicted simulation profile: neurons demonstrated interval-dependent ramping during this epoch, prior to the response cue.

Further support for an internal model hypothesis was found across different measures of neural activity, and in their relationship with subsequent behavior. Temporal scaling was evident not only at the level of DMFC single neurons but also in the population-level neural dynamics across this region. Unlike the transient single-unit responses, the rate of change in these population dynamics scaled consistently with interval length throughout the second sample interval. These dynamics reflected the same Bayesian biases observed in monkeys’ behavior: an initial bias towards the average interval duration that became less biased with more samples. Critically, these population dynamics also predicted when the monkey would saccade on the upcoming response interval, and did so above and beyond what would be predicted by the lengths of the sampled time intervals alone. Collectively, these findings are consistent with the DMFC implementing an internal model to optimize the learning of task goals and the control of neural population dynamics.

This study provides evidence that DMFC mediates the influence of prior predictions and incoming sensory evidence on planned actions, and lays the groundwork for critical tests of this proposed mechanism using causal manipulations (i.e., stimulation or inactivation). Such causal tests can also help to rule out alternative accounts of neural dynamics during the sample intervals, for instance, whether they reflect a simulated motor plan (as the authors infer) or an interval expectation (e.g., predicting the onset of the interval cue [8]). Nevertheless, by elaborating on the neuronal dynamics within DMFC during a task that requires online adjustments of learning and control, this study builds on a growing literature that implicates regions along this dorsomedial wall in the control of motor and cognitive commands [9,10].

More generally, this research provides compelling new evidence that motor and cognitive control share a common computational toolbox. Past work has suggested that both forms of control serve similar objectives (achieving a goal state within a dynamic, uncertain, and noisy environment) and that they are also both constrained by some underlying cost, limiting the amount of control that individuals can engage at a given time. As a consequence, decisions about how to allocate one’s control are sensitive to whether the reward for goal achievement outweighs these costs [10]. To the extent computational and neural architecture for motor and cognitive control allocation mirror one another, the behavior and neural dynamics observed in the current task should demonstrate sensitivity to performance incentives for both forms of control.

In spite of their abundant bodies of research, the obstacle to bridging our understanding of motor and cognitive control have been similarly abundant, including limitations of tasks, measurement tools, and model organisms. This study demonstrates how a combination of computational modeling and measures of neural dynamics in the monkey can be leveraged towards this goal and, in doing so, provides a valuable path forward in mapping the joints between these two domains of control.

From Knowing to Remembering: The Semantic–Episodic Distinction. Louis Renoult, Muireann Irish, Morris Moscovitch, and Michael D. Rugg. Trends in Cognitive Sciences, in press.

In Book summary: Unlocking the Emotional Brain and Building up to an Internal Family Systems model, I referenced models under which a particular event in a person's life gives rise to a generalized belief schema, and situations which re-activate that belief schema may also partially re-activate recollection of the original event, and vice versa; if something reminds you of a situation you experienced as a child, you may also to some extent reason in the kinds of terms that you did when you were a child and in that situation. This paper discusses connections between episodic memories (e.g., "I remember reading 1984 in Hyde Park yesterday") and semantic memories (e.g. "1984 was written by George Orwell"), and how activation of one may activate another.

What underlies the overlap between the semantic and recollection networks? We propose that the answer lies in the fact that the content of an episodic memory typically comprises a conjunction of familiar concepts and episode-specific information (such as sensory and spatial context), much as the episodic interpretation of concept cells suggests. Thus, recollection of a prior episode entails the reinstatement not only of contextual information unique to the episode, but also of the conceptual processing that was engaged when the recollected event was experienced (see also [66]). From this perspective, ‘recollection success effects’ in cortical members of the core recollection network do not reflect processing that supports episodic memory per se, but rather, the reinstatement of the conceptual processing that invariably underpins our interactions with the world in real-time (e.g., [10,67,68]). [...]

Although the proposal that recollection success effects in the core network reflect the reinstatement of conceptual processing is both parsimonious and, we contend, consistent with the available evidence, it lacks direct support. fMRI studies examining the neural correlates of successful recollection have invariably used meaningful experimental items, such as concrete words, or pictures of objects, and have typically done so in the context of study tasks that require or encourage semantic elaboration. To our knowledge, with the exception of [89], there are no published studies in which recollection effects were contrasted according to the amount of semantic or conceptual processing engaged during encoding (although see [90] for a study in which encoding was manipulated but the subsequent memory test did not allow identification of items recognized on the basis of recollection rather than on familiarity). In [89], the memory test required a discrimination between unstudied items and items subjected to semantic or nonsemantic study. Retrieval effects in the core network were not fully explored, but intriguingly, one member of the network (left parahippocampal cortex) was reported to demonstrate a greater recollection effect (operationalized as greater activity for correct than incorrect source judgments) for semantically than nonsemantically studied items. This finding is consistent with the present proposal, but it remains to be established whether, as predicted by the proposal, recollection-related activity within the core network as a whole covaries with the amount of semantic processing accorded a recollected episode when it was first experienced. [...]

Thus far, we have discussed episodic and semantic memories without reference to the possibility that their content and neural underpinnings might vary over time. However, there is a long-standing literature documenting that memory representations can be highly dynamic, shifting their dependence from the hippocampus and adjacent regions of the medial temporal lobe (MTL) to other neocortical regions, a phenomenon often referred to as ‘systems consolidation’ [64,65,91–93]. In recent years, systems consolidation has become increasingly intertwined with the construct of memory ‘semanticization’ and schematization, processes by which semantic knowledge and schemas [83] emerge from episodic memory or assimilate aspects of it.

Early studies and theories of memory consolidation, beginning with Ribot and reiterated for almost a century, typically did not distinguish between episodic and semantic memory [65,94–96]. Among the first to realize the importance of the episodic–semantic distinction for theories of memory consolidation were Kinsbourne and Wood [97]. They proposed that traumatic amnesia affected only episodic memory, regardless of the age of the memory, and left semantic and schematic memory relatively preserved. Cases in which remote episodic memories appeared to be preserved were attributed to semanticization or schematization through repeated re-encoding (see remote memory), allowing them to achieve the status of personal facts [98,99].

In an important development of the ‘standard’ model of consolidation, McClelland et al. proposed that the hippocampus maintains episodic representations of an event while communicating with (‘instructing’) the neocortical system to incorporate information about the event into its knowledge structure [100]. It was argued that, to protect the cortical network from catastrophic interference, learning had to be slow, thus providing a principled explanation for the extended time period that systems consolidation was assumed to take. Of importance, the model proposes that, in the process of incorporating an episodic memory into a semantic network, the episodic component, initially dependent on the hippocampus, is lost. This represents an important point of divergence from the standard model, in which episodic information is retained in the neocortex along with semantic information (see later).

Incorporating the original idea of Kinsbourne and Wood [97] and the complementary learning perspective [100], ‘multiple trace theory’ (MTT) [101] proposed that the hippocampus supports episodic memories for as long as they exist. By contrast, the theory proposed that semantic memories depend upon the neocortex, which extracts statistical regularities across distinct episodes. Thus, hippocampal damage should have a profound effect on retention and retrieval of episodic memories of any vintage, while leaving semanticized and schematized memories relatively intact.

While receiving empirical support [64,102] (see also [65,103,104] for examples of convergent findings from studies of experimental animals), MTT has also been subjected to several critiques (e.g., [93,105–108]). However, the essence of the theory resonates with the recurring theme of the present review that episodic and semantic memory are intertwined, yet retain a measure of functional and neural distinctiveness. Since its inception, MTT has been extended [65,104,109] to propose that episodic memories can become transformed to more semantic or schematic versions with time and experience (see ‘Episodic and Semantic Memory in Neurodegenerative Disorders’ section); indeed, in some cases, both the original and the semanticized or schematic version of a memory coexist and engage in dynamic interaction with one another. According to this Trace Transformation Theory, the specific neocortical regions supporting transformed memories differ depending on the kind of information that is retained and retrieved. Correspondingly, for complex events, the transformed memories might depend either on event schemas, or on the gist of the event [110–113]. Increased activation of the vmPFC, believed to be implicated in processing schemas [83], and decreased hippocampal activation have both been reported as details are lost and memories become more gist-like and schematic [83,102,110,113], particularly for memories that are congruent with existing schemas [114,115]. Even when details of remote memories are retained, along with continuing hippocampal activation, there is increased vmPFC activation over time [116,117]. Which memory of an event (e.g., its semanticized or schematic version or the detailed episodic memory of the original event) predominates at retrieval will depend on a variety of factors, such as contextual factors and processing demands (see ‘Semantic memory: Neural Underpinnings’ and ‘Episodic Memory: Neural Underpinnings’ sections), in addition to the availability of one or the other type of information (see also [118]). Thus, retrieval of complex memories depends on the coordinated activation of different combinations of regions (‘process-specific assemblies’ [64,119,120]) belonging to neural networks underlying episodic and semantic memory.

The neuroimaging evidence reviewed to date strongly suggests that successful recollection necessitates the reinstatement not only of sensory-perceptual contextual information characteristic of the original experience, but also the semantic representations and conceptual processing that occurred during that experience. Rather than viewing episodic and semantic memory as dichotomous or mutually exclusive entities, the marked neural overlap between these forms of memory suggests that we must move towards considering the dynamic interplay of sensory-perceptual and conceptual elements during reinstatement of a recollected experience. One way in which we could test this proposal is to examine how progressive neural insult of key structures implicated in episodic and semantic memory impacts related putative functions, including event recollection and event construction.

Comment by kaj_sotala on A Sketch of Good Communication · 2019-11-26T10:26:31.713Z · score: 8 (4 votes) · LW · GW

This model of communication describes what LW would ideally be all about. Have mentally referenced this several times.

Comment by kaj_sotala on Open & Welcome Thread - November 2019 · 2019-11-25T11:36:42.849Z · score: 11 (5 votes) · LW · GW

I put together a new sequence, Rationalists on Meditation; figured it would be nice to have an easy way of finding most of the meditation discussion on the site. As the sequence description says: "A semi-curated list of LW writings on meditation, ranging from self-reports to theorizing."

Let me know if I missed anything that you think ought to be included.

Comment by kaj_sotala on How I do research · 2019-11-24T13:00:50.765Z · score: 11 (3 votes) · LW · GW

The principle of charity recommends the latter reading.

If someone has a thin skin, that almost by definition means that their subconscious is running on something opposite to the principle of charity: the principle of erring-on-the-side-of-assuming-that-people-who-say-potentially-hostile-things-are-in-fact-hostile. You can't just tell someone to apply the principle of charity in that case; that's applying a counteractive strategy to a deeply felt emotional experience, and as such ineffective.

Comment by kaj_sotala on Gears-Level Models are Capital Investments · 2019-11-24T12:24:47.099Z · score: 7 (3 votes) · LW · GW

I think that there's a sliding scale between a black-box and a gears-level model; any gears-level model has black box components, and a mostly black-box model may include gears.

E.g. if you experimentally arrive at a physics equation that correctly describes how the wheel-with-weights behaves under a wide variety of parameters, this is more gearsy than just knowing the right settings for one set of parameters. But the deeper laws of physics which generated that equation are still a black box. While you might know how to adjust the weights if the slope changes, you won't know how you should adjust them if fundamental physical constants were to change.

(Setting aside the point that fundamental physics constants changing would break your body so you couldn't adjust the weights because you would be dead anyway.)

To put it in different terms, in a black box model you take some things as axiomatic. Any kind of reasoning requires you to eventually fall back on axioms that are not justified further, so all models are at least somewhat black boxy. The difference is in whether you settle on axioms that are useful for a narrow set of circumstances, or on ones which allow for broader generalization.

Comment by kaj_sotala on Practical Guidelines for Memory Reconsolidation · 2019-11-24T11:10:19.870Z · score: 12 (2 votes) · LW · GW

While we are on the topic of sharing memory reconsolidation tricks -

Something that been useful to me recently has been remembering that according to memory reconsolidation principles, experiencing an incorrect emotional belief as true is actually necessary for revising it. Then, when I get an impulse to push the wrong-feeling belief out of my mind, I instead take the objecting part or otherwise look for counterevidence and let the counterbelief feel simultaneously true as well. That has caused rapid updates the way Unlocking the Emotional Brain describes.

I think that basically the same kind of thing (don't push any part out of your mind without giving it a say) has already been suggested in IDC, IFS etc.; but in those, I've felt like the framing has been more along the lines of "consider that the irrational-seeming belief may still have an important point", which has felt hard to apply in cases where I feel very strongly that one of the beliefs is actually just false. Thinking in terms of "even if this belief is false, letting myself experience it as true allows it to be revised" has been useful for those situations.

Comment by kaj_sotala on Practical Guidelines for Memory Reconsolidation · 2019-11-24T11:08:14.799Z · score: 9 (4 votes) · LW · GW

As a general rule, you should always start with the side that's more cognitively fused.

Thanks, this has been a useful tip.

E.g. I've had a persistent aversion to looking people in the eyes while I'm talking to them; I've made temporary progress on this issue on several occasions but it has always regenerated. Nudged by this post, I finally realized/remembered that I should also be asking why I do want to look them in the eyes. (this is a bit embarrassing since I should have picked this kind of thing up from my IFS training, but apparently didn't consistently enough)

Asking that question brought up a certain kind of desired self-image which involved being able to look at people in the eyes, together with a prediction of the ways in which being that kind of a person would make me feel good. Surfacing that assumption made it possible to investigate and reconsolidate some of the assumptions in that schema, after which I ended up at "maybe looking people in the eyes isn't that important after all". But then making that update, somehow made it easier to notice that were sources of discomfort in not-looking-at-people as well. I'm not sure what position I'll end up in, but right now the thought of looking people in the eyes when I'm talking to them is starting to feel like a genuinely enjoyable thought.

Comment by kaj_sotala on Do you get value out of contentless comments? · 2019-11-22T08:31:06.089Z · score: 13 (5 votes) · LW · GW

I've felt happy about all positive comments, regardless of how substantial they are. Making a comment always takes more effort than just clicking upvote. Of course, more specific praise that gives me a better idea of what someone liked about my post is even better.

Comparing with a strong upvote is hard though, ideally I'd prefer both.

Comment by kaj_sotala on The LessWrong 2018 Review · 2019-11-22T06:21:41.691Z · score: 3 (1 votes) · LW · GW

Google Chrome for Android, 78.0.3904.108.

Comment by kaj_sotala on The LessWrong 2018 Review · 2019-11-21T19:52:34.497Z · score: 3 (1 votes) · LW · GW

Curious to hear examples of this.

Comment by kaj_sotala on The Intelligent Social Web · 2019-11-21T14:07:37.789Z · score: 3 (1 votes) · LW · GW

This post gave me an important set of intuitions and things to be on a watch for. What stands out the most clearly in my mind are:

  • The notion of the social web acting as an agent in its own right, and being on a watchout for things which might violate existing scripts
  • The thing where, after your role is explained to you, you try to act based on that new information... while still continuing to re-enact your same role, as you don't really know how else to act.

I do think that these things were not explained with as much rigor as might have been good: but then, explaining these things in a really rigorous way is hard, and in the absence of a more legible model it can be better to just point at your observations and hope that some of your readers will try the things on.

I do think that much of what Valentine described in this post can be explained more rigorously in subagent terms. For example, the notion of scripts which people keep executing, in my view corresponds to subconscious schemas, some of which filter out all incompatible information. I have a lot to say here, but it takes a while to work my way up there; but the fact that I even realized that I should be looking in that direction, was in part due to this post pointing me the way.

Comment by kaj_sotala on The LessWrong 2018 Review · 2019-11-21T12:06:00.106Z · score: 8 (4 votes) · LW · GW

A couple of comments on nomination UI:

  • On mobile at least, the "nominate" pop-up has a "submit" button but not a "cancel" button, which is a little inconvenient in cases where I realize I'd like to go back to check some detail about the post that I'm nominating before I nominate it
  • It would be nice if existing nominations would have a button saying "endorse this nomination" or something, so if I essentially just agree with an existing nomination and don't have anything to add, I have an easy way to add another vote to it. Making a top-level comment saying just "what Raemon's nomination said" feels weird, since the social convention is that comments which only endorse another comment are posted as responses to the comment that they are endorsing.
Comment by kaj_sotala on The LessWrong 2018 Review · 2019-11-21T12:05:26.601Z · score: 3 (1 votes) · LW · GW

I got that when I first followed the links from this page, then re-opened them and then it took me to the right version. No idea what made the difference.

Comment by kaj_sotala on Decoupling vs Contextualising Norms · 2019-11-21T11:19:29.094Z · score: 3 (1 votes) · LW · GW

My nomination seconds the things that were said in the first paragraphs of Raemon's nomination.

Comment by kaj_sotala on Public Positions and Private Guts · 2019-11-21T09:40:24.125Z · score: 8 (3 votes) · LW · GW

Having separate terms for these two positions has been particularly useful to me, because I have often felt reluctant to really lay out positions which were motivated by my private guts and for which I did not have a good public position for. Reminding myself of the difference and that it's legitimate to take your time in working out the position implied by your private guts, has made it feel more legitimate to even try.

I view my work with my multi-agent minds sequence as an extended attempt to work out model for which I initially had strong intuitions but no legible elaboration, and I'm glad that I had this post encouraging me to give it a shot anyway.

Comment by kaj_sotala on Sam Harris and the Is–Ought Gap · 2019-11-21T09:30:57.075Z · score: 13 (3 votes) · LW · GW

This post not only made me understand the relevant positions better, but the two different perspectives on thinking about motivation have remained with me in general. (I often find the Harris one more useful, which is interesting by itself since he had been sold to me as "the guy who doesn't really understand philosophy".)

Comment by kaj_sotala on Spaghetti Towers · 2019-11-21T08:04:43.218Z · score: 8 (3 votes) · LW · GW

This has been a useful concept, one which I've continued referring to afterwards, e.g. in the context of my multi-agent mind posts.