Under which conditions are human beings most likely to be altruistically motivated? 2020-06-01T19:16:33.212Z · score: 16 (5 votes)
Meetup #46 - Authentic Relating games 2020-02-24T21:29:14.207Z · score: 6 (1 votes)
Post-meetup social 2020-02-04T08:43:03.669Z · score: 6 (1 votes)
Meetup #45 - Implementation Intentions 2020-02-04T08:26:31.529Z · score: 6 (1 votes)
Meetup #44 - Murphyjitsu 2020-01-21T07:37:00.317Z · score: 6 (1 votes)
"human connection" as collaborative epistemics 2020-01-12T23:16:26.487Z · score: 21 (9 votes)
Meetup #43 - New techniques 2020-01-07T07:59:31.797Z · score: 6 (1 votes)
Moloch feeds on opportunity 2019-12-12T21:05:24.030Z · score: 33 (19 votes)
Meetup #42 - Ideological Turing Test 2019-12-09T07:36:06.469Z · score: 6 (1 votes)
RAISE post-mortem 2019-11-24T16:19:05.163Z · score: 143 (57 votes)
Meetup #41 - Double Crux 2019-11-24T16:09:53.722Z · score: 6 (1 votes)
Steelmanning social justice 2019-11-17T11:52:43.771Z · score: -1 (23 votes)
Meetup #40 - Street epistemology 2019-11-08T16:37:27.873Z · score: 6 (1 votes)
Toon Alfrink's sketchpad 2019-10-31T14:56:10.205Z · score: 6 (1 votes)
Meetup #39 - Rejection game 2019-10-28T20:36:54.704Z · score: 10 (2 votes)
The first step of rationality 2019-09-29T12:01:39.932Z · score: -2 (13 votes)
Is competition good? 2019-09-10T14:01:56.297Z · score: 8 (7 votes)
Have you lost your purpose? 2019-05-30T22:35:38.295Z · score: 29 (15 votes)
What is a good moment to start writing? 2019-05-29T21:47:50.454Z · score: 24 (10 votes)
What features of people do you know of that might predict academic success? 2019-05-10T18:16:59.922Z · score: 16 (2 votes)
Experimental Open Thread April 2019: Socratic method 2019-04-01T01:29:00.664Z · score: 31 (11 votes)
Open Thread April 2019 2019-04-01T01:14:08.567Z · score: 30 (5 votes)
RAISE is launching their MVP 2019-02-26T11:45:53.647Z · score: 85 (28 votes)
What makes a good culture? 2019-02-05T13:31:57.792Z · score: 30 (10 votes)
The housekeeper 2018-12-03T20:01:57.618Z · score: 26 (16 votes)
We can all be high status 2018-10-10T16:54:19.047Z · score: 61 (28 votes)
Osmosis learning: a crucial consideration for the craft 2018-07-10T15:40:12.193Z · score: 27 (8 votes)
Open Thread July 2018 2018-07-10T14:51:12.351Z · score: 11 (4 votes)
RAISE is looking for full-time content developers 2018-07-09T17:01:38.401Z · score: 25 (7 votes)
A friendly reminder of the mission 2018-06-05T00:36:38.869Z · score: 16 (5 votes)
The league of Rationalists 2018-05-23T11:55:14.248Z · score: 26 (17 votes)
Fundamentals of Formalisation level 2: Basic Set Theory 2018-05-18T17:21:30.969Z · score: 24 (7 votes)
The reverse job 2018-05-13T13:55:35.573Z · score: 49 (16 votes)
Fundamentals of Formalisation level 1: Basic Logic 2018-05-04T13:01:50.998Z · score: 15 (7 votes)
Soon: a weekly AI Safety prerequisites module on LessWrong 2018-04-30T13:23:15.136Z · score: 84 (26 votes)
Give praise 2018-04-29T21:00:42.003Z · score: 159 (58 votes)
Raising funds to establish a new AI Safety charity 2018-03-17T00:09:30.843Z · score: 126 (42 votes)
Welcome to LW Netherlands 2018-03-16T10:13:08.360Z · score: 3 (1 votes)
Updates from Amsterdam 2017-12-16T22:14:48.767Z · score: 16 (5 votes)
Project proposal: Rationality Cookbook 2017-11-21T14:34:01.537Z · score: 55 (20 votes)
In defense of common-sense tribalism 2017-11-02T08:43:11.715Z · score: 17 (9 votes)
We need a better theory of happiness and suffering 2017-07-04T20:14:15.539Z · score: 2 (2 votes)
Announcing AASAA - Accelerating AI Safety Adoption in Academia (and elsewhere) 2017-06-15T18:55:06.306Z · score: 18 (13 votes)
Meetup : Meetup 17 - Comfort Zone Expansion (CoZE) 2017-05-10T09:59:48.318Z · score: 0 (1 votes)
Meetup : Meetup 15 (for real this time) - Trigger Action Planning 2017-04-12T12:52:40.546Z · score: 0 (1 votes)
Meetup : Meetup #15 - Trigger-Action Patterns 2017-03-29T00:55:10.507Z · score: 0 (1 votes)
Meetup : #13 - Focusing 2017-02-28T20:52:57.897Z · score: 0 (1 votes)
Meetup : Amsterdam - Meetup #12 - Friendly AI 2017-02-15T15:08:00.708Z · score: 0 (1 votes)
Meetup : #10: Making a difference 2017-01-14T17:29:32.945Z · score: 0 (1 votes)
Meetup : Meetup #9 - 2017 is prime 2017-01-01T16:06:08.061Z · score: 0 (1 votes)


Comment by toonalfrink on Paul Crowley's Shortform · 2020-06-02T20:45:17.478Z · score: 2 (1 votes) · LW · GW

Did Dominic Cummings in fact try a "Less Wrong approach" to policy making? If so, how did it fail, and how can we learn from it? (if not, ignore this)

Comment by toonalfrink on Toon Alfrink's sketchpad · 2020-06-02T20:42:41.093Z · score: 7 (4 votes) · LW · GW

I did all the epistemic virtue. I rid myself of my ingroup bias. I ventured out on my own. I generated independent answers to everything. I went and understood the outgroup. I immersed myself in lots of cultures that win at something, and I've found useful extracts everywhere.

And now I'm alone. I don't fully relate to anyone in how I see the world, and it feels like the inferential distance between me and everyone else is ever increasing. I've lost motivation for deep friendships, it just doesn't seem compatible with learning new things about the world. That sense of belonging I got from LessWrong is gone too. There are a few things that LW/EA just doesn't understand well enough, and I haven't been able to get it across.

I don't think I can bridge this gap. Even if I can put things to words, they're too provisional and complicated to be worth delving into. Most of it isn't directly actionable. I can't really prove things yet.

I've considered going back. Is lonely dissent worth it? Is there an end to this tunnel?

Comment by toonalfrink on Is competition good? · 2020-05-25T20:19:07.668Z · score: 4 (2 votes) · LW · GW

I don't recall, this is one of those concepts that you kind of assemble out of a bunch of conversations with people that already presuppose it

Comment by toonalfrink on What Money Cannot Buy · 2020-05-21T22:40:17.182Z · score: 2 (1 votes) · LW · GW

Here's another: probing into their argument structure a bit and checking if they can keep it from collapsing under its own weight.

Comment by toonalfrink on What Money Cannot Buy · 2020-05-21T22:36:14.371Z · score: 2 (1 votes) · LW · GW

Probably the skill of discerning skill would be easier to learn than... every single skill you're trying to discern.

Comment by toonalfrink on Conflict vs. mistake in non-zero-sum games · 2020-05-21T22:11:33.722Z · score: 3 (2 votes) · LW · GW
The outgroup is evil, not negotiating in good faith, and it's an error to give them an inch. Conflict theory is the correct one for this decision.

Which outgroup? Which decision? Are you saying this is universally true?

Comment by toonalfrink on "human connection" as collaborative epistemics · 2020-01-14T19:26:35.164Z · score: 2 (1 votes) · LW · GW


Comment by toonalfrink on "human connection" as collaborative epistemics · 2020-01-13T08:06:52.877Z · score: 4 (3 votes) · LW · GW

Forgive me for stating things more strongly than I mean them. It’s a bad habit of mine.

I’m coming from the assumption that people are much more like Vulcans than we give them credit for. Feelings are optimizers. People that do things that aren’t in line with their stated goals, aren’t always biased. In many cases they misstate their goals but don’t actually fail to achieve them.

See my last shortform for more on this

Comment by toonalfrink on Toon Alfrink's sketchpad · 2020-01-13T08:01:15.798Z · score: 4 (2 votes) · LW · GW

So here's two extremes. One is that human beings are a complete lookup table. The other one is that human beings are perfect agents with just one goal. Most likely both are somewhat true. We have subagents that are more like the latter, and subsystems more like the former.

But the emphasis on "we're just a bunch of hardcoded heuristics" is making us stop looking for agency where there is in fact agency. Take for example romantic feelings. People tend to regard them as completely unpredictable, but it is actually possible to predict to some extent whether you'll fall in and out of love with someone based on some criteria, like whether they're compatible with your self-narrative and whether their opinions and interests align with yours, etc. The same is true for many intuitions that we often tend to dismiss as just "my brain" or "neurotransmitter xyz" or "some knee-jerk reaction".

There tends to be a layer of agency in these things. A set of conditions that makes these things fire off, or not fire off. If we want to influence them, we should be looking for the levers, instead of just accepting these things as a given.

So sure, we're godshatter, but the shards are larger than we give them credit for.

Comment by toonalfrink on Moloch feeds on opportunity · 2019-12-13T16:09:29.844Z · score: 6 (3 votes) · LW · GW
I am aware that confessing to this in most places would be seen as a huge social faux pas, I'm hoping LW will be more understanding.

You're good. You're just confessing something that is true for most of us anyway.

Where I have a big disagreement is in the lesson to take from this. Your argument is that we should essentially try to turn off status as a motivator. I would suggest it would be wiser to try to better align status motivations with the things we actually value.

Up to a point. It is certainly true that status motivations have led to great things, and I'm personally also someone who is highly status-driven but manages to mostly align that drive with at least neutral things, but there's more.

I struggle hugely with akrasia. If I didn't have some external motivation then I'd probably just lie in bed all day watching tv.

The other great humanist psychologist besides Maslow was Adam Rogers. His thinking can be seen as an expansion on this "subagent motivation is perceived opportunity" idea. He proposed an ideal vs an actual self. The ideal self is what you imagine you could and should be. Your actual self is what you imagine you are. The difference between ideal self and actual self, he said, was the cause of suffering. I believe that Buddhism backs this up too.

I'd like to expand on that and say that the difference between ideal self (which seems like a broader class of things that includes perceived opportunity but also social standards, the conditions you're used to, biological hardwiring, etc) and your actual self is the thing that activates your subagents. The bigger the difference, the more your subagents are activated by this difference.

Furthermore, the level of activation of your subagents causes cognitive dissonance (a.k.a. akrasia), i.e. one or multiple of your subagents not getting what they want even though they're activated.

And THAT is my slightly-more-gears-level model of where suffering comes from.

So here's what I think is actually going on with you: you're torn between multiple motivations until the status subagent comes along and pulls you out of your deadlock because it's stronger than everything else. So now there's less cognitive dissonance and you're happy that this status incentive came along. It cut your gordian knot. However, I think it's also possible to resolve this dissonance in a more constructive way. I.e. untie the knot. In some sense the status incentive pushes you into a local optimum.

I realise that I'm probably hard to follow. There's too much to unpack here. I should probably try and write a sequence.

Comment by toonalfrink on Toon Alfrink's sketchpad · 2019-12-12T21:08:43.679Z · score: 2 (1 votes) · LW · GW

Right, right. So there is a correlation.

I'll just say that there is no reason to believe that this correlation is very strong.

I once won a mario kart tournament without feeling my hands.

Comment by toonalfrink on Give praise · 2019-12-12T20:03:57.273Z · score: 4 (2 votes) · LW · GW
People generally only discuss 'status' when they're feeling a lack of it

While this has been true for other posts that I wrote about the subject, this post was actually written from a very peaceful, happy, almost sage-like state of mind, so if you read it that way you'll get closer to what I was trying to say :)

Comment by toonalfrink on Give praise · 2019-12-12T19:58:00.531Z · score: 16 (6 votes) · LW · GW

I appreciate your review.

Most of your review assumes that my intent was to promote praise regardless of honesty, but quite the opposite is true. My intent was for people to pause, take a breath, think for a few moments what good things others are doing, and then thank them for it, but only if they felt compelled to do so.

Or I'll put it this way: it's not about pretending to like things, it's about putting more attention to the things about others that you already like. It's about gratefulness, good faith and recognition. It's about validating those that are already on the right track, to embolden them and secure them.

And this works to the extent that it is genuine. If you don't feel what you say, people will notice and discard your opinion. Congruency is an obvious first step that I didn't include in the post because I assumed it to be obvious.

But of course not getting that point across is all on me. I suppose I could have written a better post.

Comment by toonalfrink on Toon Alfrink's sketchpad · 2019-12-12T19:33:04.345Z · score: 9 (4 votes) · LW · GW

I have gripes with EA's that try to argue about which animals have consciousness. They assume way too readily that consciousness and valence can be inferred from behavior at all.

It seems quite obvious to me that these people equate their ability to empathize with an animal with the ability for the animal to be conscious, and it seems quite obvious to me that this is a case of mind projection fallacy. Empathy is just a simulation. You can't actually see another mind.

If you're going to make guesses about whether a species is conscious, you should first look at neural correlates of consciousness and valence and then try to find these correlates in animals. You don't look at animal behavior at all. We have absolutely no reason to believe that behavior correlates with consciousness. That's just your empathy getting in the way. The same empathy that attributes feelings to stuffed animals.

Comment by toonalfrink on Raemon's Scratchpad · 2019-12-06T21:57:10.410Z · score: 9 (4 votes) · LW · GW

I'm looking forward to a bookshelf with LW review books in my living room. If nothing else, the very least this will give us is legitimacy, and legitimacy can lead to many good things.

Comment by toonalfrink on Affordance Widths · 2019-11-30T19:15:37.824Z · score: 5 (2 votes) · LW · GW

To me, the most useful part of this post is that it introduces this idea that affordances are personal, i.e. some people are allowed to do X while others are not. I like to see this as part of the pervasive social machinery that is Omega.

I imagine people of a certain political background to want to sneer at me, as in, "why did it take someone in your in-group to tell you this?"

To which I admit that, indeed, I should have listened. But I suppose I didn't (enough), and now I did, so here we are with a post that made my worldview more empathetic. The bottom line is what matters.

Comment by toonalfrink on Transhumanism as Simplified Humanism · 2019-11-30T19:13:04.582Z · score: 12 (4 votes) · LW · GW

This post has been my go-to definition of Transhumanism ever since I first read it.

It's hard to put into words why I think it has so much merit. To me it just powerfully articulates something that I hold as self-evident, that I wish others would recognize as self-evident too.

Comment by toonalfrink on Tradition is Smarter Than You Are · 2019-11-30T19:11:08.706Z · score: 2 (1 votes) · LW · GW

To me, this is exactly what the LW community (and the broader progressive tribe surrounding it) needs to hear. This post, along with other developments of thought in the same direction, has caused a major shift in how I think about changing things.

The first quote is most important, and I find myself using it quite often if I'm met with a person my age (or even older) that dismisses a tradition as obviously dumb. Why do you think the tradition exists in the first place? If you don't know, how can you be so sure it doesn't serve some function?

If a piece of code you ran raised an error, it would be pretty dumb to just remove the statement that raises the error to fix your program. The only reason why Omega doesn't break down that easily is because naive people like us have kicked at it long enough to make it a little more robust. That doesn't mean kicking it makes it stronger.

This direction of thinking caused a major shift in my career, from trying to hack away at the margins of society to becoming a part of the establishment. I'm writing this from my office at a governmental bank. This post is the reason that it's not some indie startup instead, and I don't regret it.

Comment by toonalfrink on The Tails Coming Apart As Metaphor For Life · 2019-11-30T19:07:28.546Z · score: 2 (1 votes) · LW · GW

I still find myself using this metaphor a lot in conversations. That's a good benchmark for usefulness.

Comment by toonalfrink on The Costly Coordination Mechanism of Common Knowledge · 2019-11-30T19:05:16.626Z · score: 2 (1 votes) · LW · GW

I could write a paragraph to explain some concept underlying a decision I made. OR there could be a word for this concept, in which case I can just use the word. But I can't use that word if it's not commonly understood.

The set of things that are common knowledge in a group of people is epistemic starting point. Imagine you had to explain your niche ideas about AI without using any concepts invented past 1900. You'd be speaking to toddlers.

I needed "common knowledge" to be common knowledge. It is part of our skill of upgrading skills. It's at the core of group rationality.

Comment by toonalfrink on A voting theory primer for rationalists · 2019-11-30T18:59:01.412Z · score: 2 (1 votes) · LW · GW

This post introduced me to a whole new way of thinking about institutional/agency design. The most merit, for me, was pointing out this field existed. The subject is close to one of the core subjects of my thoughts, which is how to design institutions that align selfish behavior with altruistic outcomes on different hierarchical levels, from the international, to the cultural, national, communal, relational, and as far down as the subagent level.

Comment by toonalfrink on Is daily caffeine consumption beneficial to productivity? · 2019-11-26T20:16:30.591Z · score: 4 (2 votes) · LW · GW

I don't think this is the right question to ask. Even if the net alertness gain of a cup of coffee is 0, it is still worth consuming during those moments that alertness is worth more, and abstaining during those moments where relaxation is worth more. Net alertness is not net EV.

Comment by toonalfrink on RAISE post-mortem · 2019-11-24T20:40:46.317Z · score: 1 (3 votes) · LW · GW

Good catch, fixed it.

100x is obviously a figure of speech. I'd love to see someone do some research into this and publish the actual numbers

Comment by toonalfrink on RAISE post-mortem · 2019-11-24T18:17:33.080Z · score: 25 (18 votes) · LW · GW

I suppose I was naive about the amount of work that goes into creating an online course. I had been a student assistant where my professor would meet with me and the other assistants to plan the entirety of the course a day before it started. Of course this was different because there was already a syllabus and the topic was well understood and well demarcated.

Also, I had visited Berkeley around that time, and word was out about a new prediction that the singularity was only 15 years ahead. I felt like I had no choice but to try and do something. Start moving mountains right there and then. Looking back, I suppose I was a little bit too impressed by the fad of the day.

Third reason is that when starting out the project was supposed to be relatively simple and limited in scope, not a full-blown charity, and every step towards making the thing bigger and drawing more resources felt logical at the time.

But to be honest I'm not very good at knowing my true motivations.

Comment by toonalfrink on Steelmanning social justice · 2019-11-23T21:24:24.085Z · score: 2 (1 votes) · LW · GW

That's fair

Comment by toonalfrink on Meetup #40 - Street epistemology · 2019-11-16T11:27:35.981Z · score: 6 (1 votes) · LW · GW

Take note! We moved the location from the VU (in the south of Amsterdam) to the OBA (near central station), so plan your trip accordingly

Comment by toonalfrink on Toon Alfrink's sketchpad · 2019-11-09T20:00:47.037Z · score: 5 (2 votes) · LW · GW

Moved to a new country twice, they broke once.

But the real cause is that I didnt regard these items as my standard inventory, which I would have done if I had more of a preservation mindset.

Comment by toonalfrink on Toon Alfrink's sketchpad · 2019-11-09T12:43:44.065Z · score: 10 (3 votes) · LW · GW

Here's a faulty psychological pattern that I recently resolved for myself. It's a big one.

I want to grow. So I seek out novelty. Try new things. For example I might buy high-lumen light bulbs to increase my mood. So I buy them, feel somewhat better, celebrate the win and move on.

Problem is, I've bought high-lumen bulbs three times in my life now already, yet I sit here without any. So this pattern might happen all over again: I feel like upgrading my life, get this nice idea of buying light bulbs, buy them, celebrate my win and move on.

So here's 4 life-upgrades, but did I grow 4 times? Obviously I only grew once. From not having high lumen light bulbs to having them.

My instinct towards growth seems to think this:

growth = novelty

But in reality, it seems to be more like this:

growth = novelty - decay

which I define as equal to

growth = novelty + preservation

The tap I installed that puts this preservation mindset into practice seems to be very helpful. It's as follows: if I wonder what to do, instead of starting over ("what seems like the best upgrade to add to my life?") I first check whether I'm on track with the implementation of past good ideas ("what did my past self intend to do with this moment again?")

Funnily enough, so far the feeling I get from this mindset seems pretty similar to the feeling I get from meditation. And meditation can be seen as training yourself to put your attention on your past intentions too.

I think this one goes a lot deeper than what I've written here. I'll be revisiting this idea.

Comment by toonalfrink on Toon Alfrink's sketchpad · 2019-11-07T13:21:08.382Z · score: 2 (1 votes) · LW · GW

He does influence my thinking

Comment by toonalfrink on Toon Alfrink's sketchpad · 2019-11-07T10:12:00.983Z · score: 10 (4 votes) · LW · GW

You may have heard of the poverty trap, where you have so little money that you're not able to spend any money on the things you need to make more. Being broke is an attractor state.

You may have heard of the loneliness trap. You haven't had much social interaction lately, which makes you feel bad and anxious. This anxiety makes it harder to engage in social interaction. Being lonely is an attractor state.

I think the latter is a close cousin of something that I'd like to call the irrelevance trap:

  • Lemma 1: having responsibilities is psychologically empowering. When others depend on your decisions, it is so much easier to make the right decision.
  • Lemma 2: being psychologically empowered makes it more likely for you to take on responsibility, and for others to give you responsibility, because you're more able to handle it.

I speculate that some forms of depression (the dopaminergic type) are best understood as irrelevance traps. I'm pretty sure that that was the case for me.

How do you escape such a trap? Well you escape a loneliness trap by going against your intuition and showing up at a party. You escape an irrelevance trap by going against your intuition and taking on more responsibility than you feel you can handle.

Comment by toonalfrink on steve2152's Shortform · 2019-11-01T16:38:13.557Z · score: 4 (2 votes) · LW · GW

I'm skeptical that anyone with that level of responsibility and acumen has that kind of juvenile destructive mindset. Can you think of other explanations?

Comment by toonalfrink on Toon Alfrink's sketchpad · 2019-11-01T14:47:19.455Z · score: 12 (4 votes) · LW · GW

Today I had some insight in what social justice really seems to be trying to do. I'll use neurodiversity as an example because it's less likely to lead to bad-faith arguments.

Let's say you're in the (archetypical) position of a king. You're programming the rules that a group of people will live by, optimizing for the well-being of the group itself.

You're going to shape environments for people. For example you might be running a supermarket and deciding what music it's going to play. Let's imagine that you're trying to create the optimal environment for people.

The problem is, since there is more than one person that is affected by your decision, and these people are not exactly the same, you will not be able to make the decision that is optimal for each one of them. If only two of your customers have different favourite songs, you will not be able to play both of them. In some sense, making a decision over multiple people is inherently "aggressive".

But what you can do, is reduce the amount of damage. My understanding is that this is usually done by splitting up the people as finely as possible. You might split up your audience into stereotypes for "men", "women", "youngsters", "elders", "autistic people", "neurotypicals", etc. In this case, you can make a decision that would be okay for each of these stereotypes, giving your model a lower error rate.

The problem with this is that stereotypes are leaky generalizations. Some people might not conform to it. Your stereotypes might be mistaken. Alternatively, there might be some stereotypes that you're not aware of.

Take these 2 models. Model A knows that some people are highly sensitive to sound. Model B is not aware of it. If your model of people is A, you will play much louder music in the supermarket. As a result, people that are highly sensitive to sound will be unable to shop there. This is what social justice means with "oppression". You're not actively pushing anyone down, but you are doing so passively, because you haven't resolved your "ignorance".

So the social justice project, as I understand it, is to enrich our models of humans to make sure that as many of them as possible are taken into consideration. It is a project of group epistemics, above all.

That means that good social justice means good epistemics. How do you collaboratively figure out the truth? The same laws apply as they would to any truthseeking. Have good faith, give it some probability that you're wrong, seek out to understand their model first, don't discard your own doubts, and be proud and grateful when you change your mind.

Comment by toonalfrink on Toon Alfrink's sketchpad · 2019-11-01T14:17:57.704Z · score: 2 (1 votes) · LW · GW

Sure. In this case "what it really is" means "what does it optimize for, why did people invent it"

Comment by toonalfrink on Toon Alfrink's sketchpad · 2019-11-01T14:14:23.083Z · score: 2 (1 votes) · LW · GW

Well yes, but no, the point is that these other people are merely means, but optimally distributing your assets over time is a means that screens off the other people, in a sense. In the end, assuming that people are really just optimizing for their own value, they might trade for that but in the end their goal is their own allocation.

Comment by toonalfrink on bgaesop's Shortform · 2019-10-31T15:02:10.472Z · score: 2 (1 votes) · LW · GW

Meta-point: I noticed almost falling into the defensive, trying to refute your sentiment without even reading the paper.

I don't hold you responsible for that, but a less polemic tone would probably get you a better response at least from me.

Comment by toonalfrink on steve2152's Shortform · 2019-10-31T14:57:08.599Z · score: 3 (2 votes) · LW · GW

Why not?

Comment by toonalfrink on Toon Alfrink's sketchpad · 2019-10-31T14:56:13.507Z · score: 4 (4 votes) · LW · GW

I've been trying to figure out what finance really is.

It's not resource allocation between different people, because the intention is that these resources are paid back at some point.

It's rather resource re-allocation between different moments in one person's life.

Finance takes money from a time-slice of you that has it, and gives it to a time-slice of you that can best spend it.

Optimal finance means optimal allocation of money across your life, regardless of when you earn it.

Comment by toonalfrink on Give praise · 2019-09-29T12:13:24.733Z · score: 2 (1 votes) · LW · GW

To me this pattern-matches to something else. The thing we need isn't just interaction, but "authentic" interaction. Let me unpack that:

An interaction is authentic when there is no inhibition involved. You're not hiding your true feelings and/or thoughts. You're not playing a role, or putting on a mask. You're just allowing your system 1 to do the interaction all by itself.

Hardly any interaction is 100% authentic. Even if you don't feel like you're inhibiting yourself, you most likely are. Still, there's a very important difference between 90% and 10% inhibition. An interaction is only as valuable as it's authenticity.

(on a side note, this is why I'm worried about today's tendency for people to forbid some forms of speech)

Comment by toonalfrink on Is competition good? · 2019-09-12T18:01:31.815Z · score: 7 (3 votes) · LW · GW

Replace AMF with any organisation for which this statement becomes obviously true. If none such organisations exists, I'm curious.

Comment by toonalfrink on Is competition good? · 2019-09-12T17:55:55.645Z · score: 2 (1 votes) · LW · GW

Most likely that's where this intuition can be traced back to

Comment by toonalfrink on Is competition good? · 2019-09-10T14:41:34.794Z · score: 3 (4 votes) · LW · GW

Right. You can make up a lot of just-so stories, but the one you came up with falls neatly into the categories I'm trying to explain.

In this case, being altruistic doesn't satisfy any need at all. There's no pressure because you're not penalized in any way for a shitty restaurant. That's why I make an exception for respect, in the sense that I claim that respect can be a driving force behind altruism even if other needs (like reduced income from being outperformed) are lacking.

I suppose any need ought to be considered when building incentive structures. Just using income will not always lead to the best outcome.

Comment by toonalfrink on "Cheat to Win": Engineering Positive Social Feedback · 2019-08-05T15:37:14.494Z · score: 2 (1 votes) · LW · GW

I think the context of "haters don't matter" is one where you already decided to ignore them.

Comment by toonalfrink on The Tails Coming Apart As Metaphor For Life · 2019-08-05T13:48:26.984Z · score: 1 (2 votes) · LW · GW

To me this looks like a knockdown argument to any non-solipsistic morality. I really do just care about my qualia.

In some sense it's the same mistake the deontologists make, on a deeper level. A lot their proposed rules strike me as heavily correlated with happiness. How were these rules ever generated? Whatever process generated them must have been a consequentialist process.

If deontology is just applied consequentialism, then maybe "happiness" is just applied "0x7fff5694dc58".

Your post still leaves the possibility that "quality of life", "positive emotions" or "meaningfulness" are objectively existing variables, and people differ only in their weighting. But I think the problem might be worse than that.

I think this makes the problem less bad, because if you get people to go up their chain of justification, they will all end up at the same point. I think that point is just predictions of the valence of their qualia.

Comment by toonalfrink on What is a good moment to start writing? · 2019-06-10T12:40:06.184Z · score: 2 (1 votes) · LW · GW

Because some people might already be at this level, and I worry that I'm just adding noise to their signal.

Maybe my question is this: given that, every year, I unexpectedly learn important considerations that discredit my old beliefs, how can I tell that my models are further along this process than those written by others?

Comment by toonalfrink on Our plan for 2019-2020: consulting for AI Safety education · 2019-06-05T16:30:36.403Z · score: 10 (4 votes) · LW · GW

Thank you. We're reflecting on this and will reach out to have a conversation soon.

Comment by toonalfrink on Our plan for 2019-2020: consulting for AI Safety education · 2019-06-05T15:50:16.654Z · score: 22 (6 votes) · LW · GW

Apologies, that was a knee-jerk reply. I take it back: we did disagree about something.

We're going to take some time to let all of this criticism sink in.

Comment by toonalfrink on Our plan for 2019-2020: consulting for AI Safety education · 2019-06-03T23:14:56.955Z · score: -8 (6 votes) · LW · GW

Looks like I didn't entirely succeed in explaining our plan.

My recommendation was very concretely "try to build an internal model of what really needs to happen for AI-risk to go well" and very much not "try to tell other people what really needs to happen for AI-risk", which is almost the exact opposite.

And that's also what we meant. The goal isn't to just give advice. The goal is to give useful and true advice, and this necessarily requires a model of what really needs to happen for AI risk.

We're not just going to spin up some interesting ideas. That's not the mindset. The mindset is to generate a robust model and take it from there, if we ever get that far.

We might be talking to people in the process, but as long as we are in the dark the emphasis will be on asking questions.

EDIT: this wasn't a thoughtful reply. I take it back. See Ruby's comments below

Comment by toonalfrink on Can movement from Conflict to Mistake theorist be facilitated effectively? · 2019-06-03T17:47:21.880Z · score: 9 (6 votes) · LW · GW

To what extent are these just features of low-trust and high-trust environments?

Assuming that these dimensions are the same, here's my incomplete list of things that modulate trust levels:

  • Group size (smaller is easier to trust)
  • Average emotional intelligence
  • The quality of group memes that relate to emotions
  • Scarcity mindsets
  • The level of similarity of group members
  • Group identity

Some of these might screen off others. This model suggests that groups with healthy discourse tend to be small, affluent, emotionally mature and aligned.

Apart from social effects I get the impression that there are also psychological factors that modulate the tendency to trust, including:

  • Independence (of those that disagree)
  • Ambiguity tolerance
  • Agreeableness


Different answer: one thing that I've seen work is to meet someone offline. People tend to be a lot more considerate after that

Comment by toonalfrink on Egoism In Disguise · 2019-05-31T16:22:45.270Z · score: 3 (2 votes) · LW · GW

Still I think this line of thinking is extremely important because it means that people won't agree with any proposal for a morality that isn't useful for them, and keeping this in mind makes it a lot easier to propose moralities that will actually be adopted.

Comment by toonalfrink on Egoism In Disguise · 2019-05-31T16:20:42.090Z · score: 4 (2 votes) · LW · GW

I agree with your conclusion, but feel like there's some nuance lacking. In three ways.


It seems that indeed a lot of our moral reasoning is confused because we fall for some kind of moral essentialism, some idea that there is an objective morality that is more than just a cultural contract that was invented and refined by humans over the course of time.

But then you reinstall this essentialism into our "preferences", which you hold to be grounded in your feelings:

Human flourishing is good because the idea of human flourishing makes me smile. Kicking puppies is bad because it upsets me.

We recursively justify our values, and this recursion doesn't end at the boundary between consciousness and subconsciousness. Your feelings might appear to be your basic units of value, but they're not. This is obvious if you consider that our observation about the world often change our feelings.

Where does this chain of justifications end? I don't know, but I'm reasonably sure about two things:

1) The bedrock of our values are probably the same for any human being, and any difference between conscious values is either due to having seen different data, but more likely due to different people situationally benefitting more under different moralities. For example a strong person will have "values" that are more accepting of competition, but that will change once they become weaker.

2) While a confused ethicist is wrong to be looking for a "true" (normative) morality, this is still better than not searching at all because you hold your conscious values to be basic. The best of both worlds is an ethicist that doesn't believe in normative morality, but still knows there is something to be learned about the source of our values.


Considering our evolutionary origins, it seems very unlikely to me that we are completely selfish. It seems a lot more likely to me that the source of our values is some proxy of the survival and spread of our genes.

You're not the only one who carries your genes, and so your "selfish" preferences might not be completely selfish after all


We're a mashup of various subagents that want different things. I'd be surprised if they all had the same moral systems. Part of you might be reflective, aware of the valence of your experience, and actively (and selfishly) trying to increase it. Part of you will reflect your preferences for things that are very not-selfish. Other parts of you will just be naive deontologists.