Posts

Focus on the Hardest Part First 2023-09-11T07:53:33.188Z
Using Negative Hallucinations to Manage Sexual Desire 2023-09-10T11:56:24.906Z
Where are the people building AGI in the non-dumb way? 2023-07-09T11:39:12.692Z
Write the Worst Post on LessWrong! 2023-06-23T19:17:56.049Z
I can see how I am Dumb 2023-06-10T19:18:59.659Z
Leave an Emotional Line of Retreat 2023-06-08T18:36:31.485Z
Hallucinating Suction 2023-06-02T14:16:34.676Z
Believe in Yourself and don't stop Improving 2023-04-25T22:34:30.354Z
Being at peace with Doom 2023-04-09T14:53:22.924Z
Transparency for Generalizing Alignment from Toy Models 2023-04-02T10:47:03.742Z
Eliezer's Videos 2023-03-30T22:16:30.269Z
Computer Input Sucks - A Brain Dump 2023-03-08T11:06:37.780Z
Bite Sized Tasks 2023-03-04T03:31:30.404Z
Reading Speed Exists! 2023-02-18T15:30:52.681Z
Run Head on Towards the Falling Tears 2023-02-18T01:33:50.202Z
My Advice for Incoming SERI MATS Scholars 2023-01-03T19:25:38.678Z
Don't feed the void. She is fat enough! 2022-12-29T14:18:44.526Z
Is there any unified resource on Eliezer's fatigue? 2022-12-29T14:04:53.488Z
Working towards AI alignment is better 2022-12-09T15:39:08.348Z
Understanding goals in complex systems 2022-12-01T23:49:49.321Z
Is there an Ultimate text editor? 2022-09-11T09:19:51.436Z
[Exploratory] Becoming more Agentic 2022-09-06T00:45:43.835Z
[Exploratory] What does it mean that an experiment is high bit? 2022-09-05T03:13:10.034Z
[Exploratory] Seperate exploratory writing from public writing 2022-09-03T02:57:18.167Z
[Exploratory] Exploratory Writing Info 2022-09-03T02:50:57.795Z
How (not) to choose a research project 2022-08-09T00:26:37.045Z
Gathering Information you won't use directly is often useful 2022-07-24T21:21:54.877Z
Post hoc justifications as Compression Algorithm 2022-07-03T05:02:15.142Z
SOMA - A story about Consciousness 2022-07-03T04:46:18.291Z
Sexual self-acceptance 2022-07-03T04:26:46.801Z
Agent level parallelism 2022-06-18T20:56:12.236Z
Saying no to the Appleman 2022-04-29T10:39:48.693Z
Convincing Your Brain That Humanity is Evil is Easy 2022-04-07T21:39:14.688Z
Finding Useful Things 2022-04-07T05:57:47.058Z
Setting the Brains Difficulty-Anchor 2022-04-07T05:04:54.411Z
What Should We Optimize - A Conversation 2022-04-07T03:47:42.439Z
My Transhuman Dream 2022-04-05T15:44:46.636Z
Being the Hero is hard with the void 2022-01-17T11:27:31.020Z
The possibility of no good amazing forecasters 2022-01-03T12:57:59.362Z
Johannes C. Mayer's Shortform 2021-05-23T18:30:20.427Z

Comments

Comment by Johannes C. Mayer (johannes-c-mayer) on Where are the people building AGI in the non-dumb way? · 2023-09-23T14:35:46.114Z · LW · GW

That is an interesting analogy.

So if I have a simple AGI algorithm, then if I can predict where it will move to, and understand the final state it will move to, I am probably good, as long as I can be sure of some high-level properties of the plan. I.e. the plan should not take over the world let's say. That seems to be property you might be able to predict of a plan, because it would make the plan so much longer, than just doing the obvious thing. This isn't easy of cause, but I don't think having a system that is more complex would help with this. Having a system that is simple makes it simpler to analyze the system in all regards, all else equal (assuming you don't make it short by writing a code golf program, you still want to follow good design practices, and lay out the program in the obvious most understandable way).

As a story sidenote before I get into why I think tho Q-gap probably is wrong: That I can't predict that it will rain tomorrow if I have the perfect model of low-level dynamics in the universe, has more to do with how much compute I have available. I might be able to predict if it would rain tomorrow would I know the initial conditions of the universe and some very large but finite amount of compute, if the universe is not infinite?

I am not sure the Q-gap makes sense. I can have a 2D double pendulum. This is very easy to describe and hard to predict. I can make a chaotic system more complex, and then it becomes a bit harder to predict but not really by much. It's not analytically solvable for 2 joints already (according to Google).

That describing the functioning of complex mechanisms seems harder than saying what they do, might be an illusion. We as humans have a lot of abstractions in our heads to think about the real world. A lot of the things that we build mechanisms to do are expressible in these concepts. So they seem simple to us. This is true for most mechanisms we build that produce some observable output.

If we ask "What does this game program running of a computer do?" We can say something like "It creates the world that I see on the screen." That is a simple explanation in terms of observed effects. We care about things in the world, and for those things we normally have concepts, and then machines that manipulate the world in ways we want have interpretable output.

There is also the factor that we need complex programs for things where we have not figured out a good general solution, which would then be simple. If we have a complex program in the world, it might be complex because the creators have not figured out how to do it the right way.

So I guess I am saying that there are two properties of a program. Caoticness, and Kolmogorov complexity. Increasing one always makes the program less interpretable, if the other stays fixed, if we assume that we are only considering optimal algorithms, and not a bunch of halfhazard heuristics we use because we have not figured out the best algorithm yet.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-09-16T17:44:20.753Z · LW · GW

Something really interesting just happened to me. I was kind of depressed and could not bring myself to do anything really. I had set a 30-minute timer and wanted to do some AI alignment research for at least 30 minutes. But I could not do anything. I started out with some will, but then I started to organize my obsidian tabs. That seemed sort of required before starting.

Then I did this for 10 minutes, my will gradually decreased. Then I just sat down and researched some random unrelated thing on the internet. I managed to stop myself, and just sat there staring into nothingness. Then I decided if I couldn't do any work I guess I could eat something, maybe that would help to feel better.

I went into the kitchen. Then I thought "Alright I think I could probably do at least 1 minute of thinking about AI alignment. I looked at the clock and remembered the position of the second hand. Then I started to somewhat lethargically think about stuff. After 20 seconds I had an idea, after 40 seconds I was so excited that I had a strong urge to just sit down and write about what I thought about. I was really motivated! I also stopped feeling fatigued.

WTF? In hindsight, this is a pattern that I run into all the time. I think have an idea and then get really excited about it. It's not the case that I had a super duper out-of-distribution, for me, good idea. It's the kind of idea that I expect I can generate on demand in at most a couple of minutes, pretty consistently. So I probably can just cause this situation whenever I want!

It's crazy that I observe this pattern only now because it has been there for probably all my life. Definitely for the last 10 years. Now even better, the idea that I had was not random. It was exactly on the topic that I wanted to generate an idea on. I think I am pretty good at focusing on idea generation.

It seems like I very often fail to activate my motivation in this straightforward way, because I think I need to do some other stuff first, like sort my obsidian taps. Would I just start to do the actual thing I want to do, I would succeed.

So this clearly implies a strategy. I call this "1, 2, 3, and 60". It's simple, just pick a topic, and then make yourself think about the topic. Don't think vaguely about the topic. Don't focus on recalling facts. The goal is to generate an idea. No setup is required. You don't need to go to a whiteboard or open your laptop. Just start thinking.

Comment by Johannes C. Mayer (johannes-c-mayer) on Using Negative Hallucinations to Manage Sexual Desire · 2023-09-16T12:38:39.835Z · LW · GW

I feel like had the technique been "Imagine ice cream tastes like pure turmeric powder", it would basically be the same technique. I haven't tried this, but maybe this would work for somebody who is fantasizing about eating ice cream, which causes them to eat too much ice cream.

In that case, I predict people would not have had these (from my perspective) very weird reactions. Imagining random sex scenes feels as meaningful as eating ice cream to me. I could have explained myself much better. Apparently, I did not say precisely enough that I have the problem of having random sexual thoughts. It's not about imagining having sex with some person you love or anything like that. At least not most of the time. It's not clear to me that this would actually be better. I think it would not be.

I am not reflectively stable. If don't want to love somebody because they look a certain way, I want to love somebody for their mind and I am interacting with it.

I am honestly pretty confused by all these reactions. It makes me wonder if this just is not a problem for most people, or if most people have just not realized that this is a problem. I am pretty sure it's both.

Comment by Johannes C. Mayer (johannes-c-mayer) on Focus on the Hardest Part First · 2023-09-15T00:29:42.519Z · LW · GW

Is there any particular reason to expect this problem as a whole to be solvable at all, even in principle?

This is another very good heuristic I think, that I agree is good to do first.

If the answers are "yes", "no", "no", then I am inclined to agree that attacking C is the way to go. But also I think that combination of answers barely ever happens.

I think in alignment 2 is normally not the case, and if 2 is not the case 3 will not really help you. That's why I think it is a reasonable assumption that you need A, B, and C solved.

Your childcare example is weird because the goal is to make money, which is a continuous success metric. You can make money without solving really any of the problems you listed. I did not say it in the original article (and I should have) but this technique is for problems where the actual solution requires you to solve all the problems. It would be like making money with childcare at all requires you to have solve all the problems. You can't solve one problem a bit and then make a bit more money. If C is proven to be impossible, or way harder than some other problem set for a different way to make money, then you should switch to a different way to make money.

In alignment, if you don't solve the problem you die. You can't solve alignment 90% and then deploy an AI build with this 90% level of understanding because then the AI will still be approximately 0% aligned and kill you. We can't figure out everything about a neural network, even what objective function corresponds to human values, except whether it is deceptively misaligned and live.

Your alignment example is very strange. C is basically "Solve Alignment" whereas A and B taken together do not constitute an alignment solution at all. The idea is that you have a set of subproblems that when taken together will constitute a solution to alignment. Having "solve alignment" in this set, breaks everything. The set should be such that when we trim all the unnecessary elements (elements that are not required because some subset of our set already constitutes a solution) you don't remove anything, because all elements are necessary. Otherwise, I could add anything to the set and still end up with a set that solves alignment. The set should be (locally) minimal in size. If we trim the set that contains solve alignment, we just end up with a single element "solve alignment" and we have not simplified the problem by factoring it at all.

Or even better you make the set a tree instead, such that each node is a task, and you split nodes that are large into (ideally independent) subtasks until you can see how to solve each subpart. I guess this is superior to the original formulation. Ideally, there is not even a hardest part in the end. It should be obvious what you need to do to solve every leave node. The point where you look for the hardest part now is looking at what node to split next (the splitting might take a significant amount of time).

Thank you for telling me about the CAP problem, I did not know about it.

Comment by Johannes C. Mayer (johannes-c-mayer) on Using Negative Hallucinations to Manage Sexual Desire · 2023-09-14T23:55:23.220Z · LW · GW

I see, I agree. I guess we had different ideas about what constitutes an explanation. This probably does not satisfy your requirements for an explanation. I am also not sure how to generate such a model. It seems like I have an intuition about that this can not be dangerous, but they don't really have underlying an understanding of what exactly is going on. It's probably more of the type where I have observed certain things in the past that were not dangerous and this seems sufficiently like them. But at least to some extend that intuition has compressed the actual observed instances, such that I can't recall all of them in detail to give you the same opportunity to generate an intuition based on them (given that you would believe I report the instances accurately).

Comment by Johannes C. Mayer (johannes-c-mayer) on Focus on the Hardest Part First · 2023-09-14T18:21:48.231Z · LW · GW

Yes, that is a good heuristic too, though I feel like one that does not conflict at all with the one proposed in the OP. Seem to me like they are complementary.

I think Richard Hamming would say something like "Identify the important problems of your field and then tackle those and make sure that you can actually solve them. It wouldn't do any good thinking about the most important problems all the time that you then cannot solve." This also seems to not contradict the OP and seems to be a technique that can be combined with the one in the OP.

Comment by Johannes C. Mayer (johannes-c-mayer) on Using Negative Hallucinations to Manage Sexual Desire · 2023-09-14T18:16:45.858Z · LW · GW

Quoting myself:

You can imagine rubbing yourself against somebody else, flat skin on flat skin. But it just feels kind of pointless and so you stop.

This is a high-level explanation of why it works. I am not talking about it on the level of neurons or specific mental algorithms. But to me, it seems that this explanation captures a core part of why the technique works. You can just perform the experiment and imagine this, then you will see that it works. It is not a hard experiment to perform.

Comment by Johannes C. Mayer (johannes-c-mayer) on Focus on the Hardest Part First · 2023-09-14T18:11:10.724Z · LW · GW

When Eliezer wrote HPMOR, it was not clear to him that things would go down in the 2020s. That's what he said in I think this interview. Had Eliezer's plan worked out to create a dozen new better Eliezers through his writing (which was the plan) this would have been the best action.

Also, I agree that you should not apply the reasoning suggested in the OP blindly. I think it is a useful heuristic.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-09-14T16:37:09.465Z · LW · GW

Antiperfectionism Badness

Writing well takes a lot of time and effort. I just realized that now. Before I was trying to rush everything because according to my model, it should not take that much time and effort to write something well. I think many of the things I was writing ended up a lot worse than they could have been.

Basically, exactly the same thing happened to me recently with programming. I was mostly writing programs that were completely horrible spaghetti code because I was just optimizing to get some specific functionality implemented as fast as possible. But then I realized how long it actually takes to write a good program.

Updating your model of "what it takes" to better match reality, seems to be extremely helpful. It feels like before I was not allowing myself to put in the appropriate amount of time and effort to hit my desired quality target. And the funny thing is that, at least with programming, it will actually take longer in terms of total time spent, to get some specific functionality implemented on average, if your code is a horrible mess, and has grown beyond a few hundred lines of code.

And what made me update my model is just to stop caring about completing as fast as possible. I just allowed myself to put in the time, and then I observed how much time and effort I needed to hit a specific quality target.

It seems like I had the dual problem of perfectionism. I expect that this is a common problem (at least for adults) when learning something new, and I expect realizing that this problem exists as you run into it, will lessen it's grip on you.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-09-14T12:03:48.161Z · LW · GW

I have been prescribed Pitolisant (sold as Wakix, Ozawade), a recent (FDA approved in August 2019) H3 receptor antagonist against excessive daytime sleepiness by treated sleep apnea. It works like this:

When Histamine binds to H1 and H2 receptors, it promotes wakefulness. When histamine binds to H3 auto receptors it primarily blocks the release of Histamine. It also has a weaker blocking effect on the release of other neurotransmitters. Therefore, blocking H3 receptors can increase Histamine levels in the brain, leading to increased activity on H1 and H2 receptors, which in turn leads to increased wakefulness.

I haven't tried it yet but I found it interesting as it uses a method of action I did not know about. An advantage over other stimulants is that it does not raise blood pressure (at least that's what my doctor told me, can't seem to easily confirm with google).

Comment by johannes-c-mayer on [deleted post] 2023-09-11T22:12:43.526Z

the bits you control and the bits you can merely know

This seems interesting. There is definitely something there. I was thinking about the example of minimax. There we have our actions at one step, and at the next step, we consider the actions of the opponent. Normally we imagine optimal play, but you could really substitute in any policy to determine the opponent's behavior. Maybe this this would not be captured by your formalism by default, because it is too simple, but something close to this should probably be captured.

Comment by Johannes C. Mayer (johannes-c-mayer) on Using Negative Hallucinations to Manage Sexual Desire · 2023-09-11T21:58:27.641Z · LW · GW

You haven't explained anything about why the technique has the effect it has ...

Ehh. Quoting myself:

You can imagine rubbing yourself against somebody else, flat skin on flat skin. But it just feels kind of pointless and so you stop.

And then you said:

... or demonstrated in any way that you have the expertise to know why it has the effect it does.

I used it and it worked. What else do you want me to say?

Do you want me to send you all my time tracking data from the past 3 years, and analyze the statistical correlation between masturbation and procrastination? Maybe we need to wait a bit though such that more time passes and a change becomes unambiguously visible after I started to use the technique. I think that would show it, if I am not wrong about the effectiveness.

There are multiple different mental motions that can result in emotions being suppressed.

Yes, I agree. And seems different from any of them (that I am aware of). Also, see the second paragraph here.

The model coming from Gendlin's Focusing is that having a felt sense is key for dissolving emotions. If you hallucinate away the part of the body where the felt sense that corresponds to the emotion happens to be located that hampers dissolving.

Again, see the second paragraph here.

I think I have mistitled this post. It should not contain the negative hallucination part, because it is not required at all for the technique. Its enough to imagine a fictional scene.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-09-11T20:19:40.085Z · LW · GW

You can say "Ouch that hurt me emotionally, I would like this to not happen again." Then you can both think about how to prevent this in the future and change your behavior accordingly, such that you incrementally converge onto a good conversation norm. I think something like this is the right approach.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-09-11T14:28:25.119Z · LW · GW

Good point. Probably there is a one-paragraph version of this that would be sufficient. I think escalating can be fine, though I think it is always better to be explicit about it, and at least at some point go "Let's do Crocker's rules and its opt-out." That makes it clear that opting out is an acceptable action. I think it's also good to raise the awareness of optimizing the communication for usefulness. Sometimes I talk to people and then start out just saying nice nices of how good everything is that I am doing at a very superficial level. And that is not useful at all.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-09-11T14:22:28.381Z · LW · GW

...where you seem to be the one being hurt by what I wrote... LOL, what makes you think that? I experienced no negative emotions while engaging with your content. At least that's what it feels like retrospectively. Maybe I failed to see some subtle ones, but certainly, there were no strong/medium negative feelings. I was simply trying to understand what you were saying. In fact, I remember thinking something like "Hmm interesting let's see if this person says stuff about how I am failing such that I can do better", and that was a thought with positive valence.

I think now I understand better. My model this far has been that in the past I have been suppressing my emotions. That definitely happened. But now I have updated my model so that I probably very often was unaware of them. Being unaware and suppressing emotions seems different and independent. I can be angry and not aware that I am angry, not noticing how it changes my behavior. That is different from suppressing the anger, trying to not have it influence your behavior. Though I am pretty sure that you can suppress emotions without being aware of them. I think that is probably what happened most of the time.

To be clear I am not saying that the part of my brain that feels my emotions is atrophied. I am not sure about this. It's hard to say not having any reference frame (for interpreting the emotions of others you can get a reference frame).

Actually, now realize that a major part of how I realized that I am missing certain brain functions is that other autistic people were hunting me unintentionally because they just did not realize the emotions they were creating in me. And then I realized that I was doing the same. But this I think really did not happen here. When these autistic people hurt me on accident, it was so over the top what they were saying that people normally laugh if I tell them what they said.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-09-11T14:01:32.244Z · LW · GW

Yes, I would, because then I would need to use that social inference engine that is <atrophied/not exsistant> in my brain. I don't remember what they said, but I don't think it was very ambiguous to anyone but me.

Comment by Johannes C. Mayer (johannes-c-mayer) on Elizabeth's Shortform · 2023-09-11T12:54:29.082Z · LW · GW

I am not quite sure what the correct answer is for playing Minecraft (let's ignore the Ender Dragon, which did not exist when I played it).

I think there is a correct answer for what to do to prevent AI doom. Namely to take actions that achieve high expected value in your world model. If you care a lot about the universe then this translates to "take actions that achieve high expected value on the goal of preventing doom."

So this only works if you really care about the universe. Maybe I care an unusual amount about the universe. If there was a button I could press that would kill me, but that would save the universe, then I would press it. At least in the current world, we are in. Sadly it isn't that easy. If you don't care about the universe sufficiently compared to your own well-being, the expected value from playing video games would actually be higher, and playing video games would be the right answer.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-09-11T12:39:12.789Z · LW · GW

Do I understand correctly that you think I am ignoring my emotions and that this is a problem? I agree that it is terrible to ignore your emotions and I am trying to not do this. I definitely feel emotions and in my experience not acknowledging them makes things just a lot worse.

I can definitely feel very hurt when people say extremely negative critiques about something that I am saying. And I know that this can be pretty harmful because it uncontrollably activates some reinforcement mechanism in my brain changing me for the worse. At least I think very often for me it has been for the worse. So not being aware of this mechanism and how it interacts with emotion is not a good thing.

So I'm not sure what to take from this message as it seems like I already was aware of the problems you were pointing out. Of course, I think I'm not really as good as I could be at recognizing emotions and handling them correctly.


I'm a bit confused. Do you understand the concept of not having some hardware in your brain that other people have? Here is an interesting thing that happened to me when I was a child. Other people would sometimes bully me. However, I was unable to project a harmful intent onto their actions. And then the bullying didn't work at all. Because I failed to recognize that a piece of language was supposed to hurt me, it didn't. That is pretty funny I think.

I think the only way this can happen is if you're just missing some functionality in your brain for understanding the actions, intentions, and emotions of other people. I think that is the case for me, But I am not sure if this is the case for you. I think this is a very important distinction.

Comment by Johannes C. Mayer (johannes-c-mayer) on Focus on the Hardest Part First · 2023-09-11T09:13:03.264Z · LW · GW

Now that is the right question. There is the AGI Ruin list which talks about a lot of the hard problems.

I think a very core thing is figuring out how can we make a system robustly "want" something. There are actually a bunch more heuristics that you can use in order to determine good problems to work. One is to think about what things need to be solved because they will show up in virtually all agendas (or at least all agendas of a particular type). And how to make a system robustly "want" something probably falls into that category.

If we could just figure out this, we might be able to get away with not figuring out human values. Potentially we could make the AI perform some narrow task, that constitutes a pivotal act. However, figuring out just how to make a system robustly "want" something does not seem to be enough. We need to also figure out how to make the system "want" to perform the narrow thing that constitutes a pivotal act. And we also need to make it such that the system would not spawn misaligned subagents. And probably a bunch more problems that did not come immediately to mind.

Comment by Johannes C. Mayer (johannes-c-mayer) on Using Negative Hallucinations to Manage Sexual Desire · 2023-09-11T08:06:43.729Z · LW · GW

I guess I must have not explained myself correctly. I am unsure which part of this technique is supposed to suppress emotions. You are not suppressing anything, instead, you make the feeling of sexual desire actually disappear. Without forcing anything.

I am pretty sure of this because I know what it feels like to suppress emotions and that seems like a very different mental motion.

Imagine you are angry because had an unpleasant interaction with a clerk. I can often dissolve, not suppress, that anger by realizing that this person might have had a bad day or even a really bad life working a terrible job they don't enjoy. Or maybe that is just their personality, which they did not really choose themselves, and which likely means that they have probably zero friends. When I think like that I just experience empathy for that person. I fail to see what is wrong with doing this. And I don't think the tek in the OP is different along the suppression vs. dissolving axis.

Comment by Johannes C. Mayer (johannes-c-mayer) on Elizabeth's Shortform · 2023-09-10T19:30:55.628Z · LW · GW

I know people like this. I really don't understand people like this. Why not just take the challenge to play real live it's a videogame with crushing difficulty. Oh wait that's maybe just me who plays games on very hard difficulty most of the time (in the past when I did play video games). I guess there is probably not one reason people do this. But I don't get the reason why you are being crushed by doom. At least for me using the heuristic of just not giving up, never (at least not consciously, I probably can't muster a lot of will as I am being disassembled by nanobots, because of all the pain you know), seemed to work really well. I just ended up reasoning myself into a stable state, by enduring long enough. I wonder if the same would have happened for your fried had he endured longer.

Comment by Johannes C. Mayer (johannes-c-mayer) on the gears to ascenscion's Shortform · 2023-09-10T19:20:39.994Z · LW · GW

Depends on what you are talking about. Try to make an "explanation of how quicksort works" political (well ok that is actually easy, but the default version seems pretty unpolitical to me).

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-09-10T19:17:26.593Z · LW · GW

Here is a template (feel free to use) that you might find useful as an introductory message if you find it hard to consider how your actions make other people feel:

Do you subscribe to Crooker's rules? Did you notice that Eliezer sometimes seems inconsiderate of people's emotions, when he just shoots down one (bad) alignment idea after the other? He just says things like "No, this does not work." My guess is that there are some algorithms missing from his brain or are atrophied, just like for me. For me, it's pretty hard to take into account how other people will feel when I say something. It's just not something that comes naturally and I need to be very explicitly thinking about this in order to make what I say not come across as offensive.

Basically, I think it would be good if you model me as a person who is missing the hardware in his brain which is automatically inferring how I will make people feel with my actions. I need to repurpose some other machinery for this, which takes a lot more effort and is slower. Often people call this autism, but I think my description is more precise and useful for telling other people what is going on, such that they understand.

And it seems very clear that there is a big gap. For example, Buck once said that he does not like to shoot down the ideas of other people because he does not want them to feel bad. At that moment I realized that I could not remember that this thought ever occurred to me. It seemed very alien, but also obviously good. Of course, I don't want to make people feel bad. But even the possibility of this happening was, I noticed, missing from my world model.

It would make things easier for me if I could not worry about this too much. If you subscribe to Crooker's rules then I could optimize my messages only for content. For example, if you subscribe to Crooker's rules then I could just not optimize my messages at all for sounding nice.

If you find that my mode of communication causes you emotional trouble, we could always revert back to me optimizing more for not sounding harsh.

Some harsh things I might do are:

  • I write a thousands-of-word document describing how you're wrong on a particular topic, which makes it seem like I'm really critical of what you do and don't think it's good at all, when in fact I mean something more like, look here are some issues I think I discovered and I'm not even thinking about the implicit message that I send by sending a long document that just contains criticism.
  • I use phrases like <No/wrong...[Explanation]/This doesn't make sense/This doesn't work>
Comment by Johannes C. Mayer (johannes-c-mayer) on Using Negative Hallucinations to Manage Sexual Desire · 2023-09-10T19:09:40.655Z · LW · GW

I am pretty sure the specific thing you suggested will not work at all, but I won't even try, because I have no interest in this ability. I am pretty sure you can exaggerate how good something will feel in your imagination though. But this seems significantly harder. And one of the main reasons why I think the original tek is good is that it is very very easy to do. Keep in mind that if you exaggerate then you might be disappointed when something real happens.

Comment by Johannes C. Mayer (johannes-c-mayer) on Using Negative Hallucinations to Manage Sexual Desire · 2023-09-10T18:17:48.182Z · LW · GW

I don't think its somewhat of a stretch to put this in the dark arts category. It's not like I erase from my world model the fact that I have primary sexual organs, or that it would feel a certain way being touched a certain way. It is a temporary very shallow overwrite, that instantly gets reversed when you stop doing the technique. I think it is not required to even reference your body at all.

What I am doing seems more like what you do when you do a thought experiment. You imagine a hypothetical situation with specific constraints. You can imagine this situation and feel your body at the same time. In the thought you don't have primary sexual organs, in the real world you do. Based on one brief experiment that still seems to work. Unsurprisingly, because I am pretty sure that 25-50% of the time I was using the technique like this already.

All of this seems very different, at the very least quantitatively, from e.g. changing your terminal goals for instrumental reasons.

Comment by Johannes C. Mayer (johannes-c-mayer) on Using Negative Hallucinations to Manage Sexual Desire · 2023-09-10T17:57:29.680Z · LW · GW

I am not sure. I don't think it has damaged my sexual desire even though I used this technique maybe 20-40 times. I guess this is based on that I did get sexual thoughts after waking up. Being groggy I forget to apply the technique, and only realized after having sexual thoughts for over 20 minutes that I could use the technique to escape.

Also, that might give you an idea of why I even need to come up with a weird technique like this. Because I think it would have gone on a lot longer than 20 minutes. And I wasn't even touching anything. Just laying there imagining. I was a heavy porn addict as a teenager. That was terrible. But I don't think this problem is exclusive to porn. At points in time when I had a mate, I would just do it for hours on end most days. And I stopped most of the time because my mate wanted to stop. You might think "sounds fun", and it wasn't funless, but retrospectively it seems like an addiction. One that shares many similarities with a porn or video game addiction.

I am unsure about the long-term consequences as I have not used this technique for long enough to evaluate that I think, but my guess is that it is not dangerous. I guess that this technique can not overwrite the actual physical signals that your body sends to your brain when these are sent. I expect it to only be useful for controlling imagination.

For me, it seems that fapping is very very bad. Possibly the single worst thing for my productivity. But I feel like I might have an "ultra-sexual mind". I am pretty sure there is a strong correlation between fapping and wasting multiple days afterward by playing video games and watching series. I am not quite sure what the causal relationships are, but there is a very strong correlation.

I have time tracking data where I record everything I do throughout the entire day. I haven't run any deep analysis on it that would determine this. But on the off-chance that somebody would like to do it (I don't know why anybody would) I can give them years' worth of time tracking data. I am relatively sure that I would not care if any analysis of this data would be made public (possibly with the exception of some very minor censors like the names of people I met with).

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-09-03T13:00:30.389Z · LW · GW

I have been taking bupropion for two weeks now. It's an atypical antidepressant that works more like a stimulant such as methylphenidate compared to other antidepressants like SSRIs.

So far my experience has been very positive. Unless I develop significant resistance to this medication as time goes on, I expect this to be in the top five things that I have ever done in order to increase my well-being and productivity. It does not have any annoying side effects for me. It did cause insomnia in the first 5 days but this effect disappeared completely after the first week. It was also very easy and fast to get a prescription (in Germany). It's not available in the UK or Australia iirc.

Therefore I tentatively recommend if you are even slightly depressed sometimes, that you read this document.

For me it was especially useful because it helped in 3 ways:

  • Make me less depressed (works very well for this. That is what it is for what it is prescribed for after all)
  • It makes me less fatigued (I had some chronic fatigue before. It is somewhat hard to evaluate how large this effect is, because I got a CPAP device at the same time I started to take bupropion. But there seems to be a noticeable difference before and after I take the bupropion.)
  • It slightly lessens ADHD symptoms (this is mainly useful for me right now because it takes forever to get a prescription for ADHD medication unless I would put in a lot more time into optimizing to get one faster)

It might even make sense to think about this if you are experiencing any subset of these problems.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-09-01T16:27:46.788Z · LW · GW

That is a good point. I defeated myself. The actual measure of goodness is in terms of how many words you need to make somebody truly understand, in the shortest amount of time.

That means telling you the peano axioms would not count as having told you that the system they are defining is incomplete. Though that depends on the mind. If I tell the piano axioms to an AGI that does not know about gödel incompleteness it could probably figure it out very quickly.

Comment by Johannes C. Mayer (johannes-c-mayer) on johnswentworth's Shortform · 2023-08-28T18:11:07.836Z · LW · GW

This seems basically correct though it seems worth pointing out that even if we are able to do "Meme part 2" very very well, I expect we will still die because if you optimize hard enough to predict text well, with the right kind of architecture, the system will develop something like general intelligence simply because general intelligence is beneficial for predicting text correctly. E.g. being able to simulate the causal process that generated the text, i.e. the human, is a very complex task that would be useful if performed correctly.

This is an argument Eliezer brought forth in some recent interviews. Seems to me like another meme that would be beneficial to spread more.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-08-28T17:57:47.027Z · LW · GW

Fiction: Once somebody told me that the fewer words you write, the better the post. I promptly opened a new document and proclaimed: "I have written the ultimate post. It's the empty string."

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-08-25T19:12:06.512Z · LW · GW

Hypothesis: There are policies that are good at steering the world according to arbitrary objectives, that have low Kolmogorov complexity.

It is systems that implement these policies efficiently that we should be scared of, as systems that implement policies without low Kolmogorov complexity would be computationally intractable, and therefore we can only end up with systems that are approximating these policies. Therefore these systems would not actually be that good at steering the world according to arbitrary objectives. Shallow pattern recognition objects are of this form.

Systems that don't manage to implement the policy efficiently would probably mostly not be computationally tractable (every policy can be represented with a lookup table which definitely would be computationally intractable for the real world). Every program that can be practically run that implements the policy would basically be just as dangerous as the shortest program encoding the policy.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-08-23T19:00:28.208Z · LW · GW

LOL, what a dumb mistake. Fixed. Thanks.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-08-23T18:58:28.753Z · LW · GW

To be productive, sit down in a comfortable zero-gravity armair and do nothing. You are not allowed to watch YouTube videos or browse social media. Just relax. Do this until you naturally want to start to work. It is important that you are comfortable.

This seems to be surprisingly effective (haven't done any rigorous evaluation). Ideally have a laptop together with AR goggles within arms reach without getting up such that you can just lay in the armchair and start to work, if necessary.

I have found that even when I am very tired I can still work when laying in a comfortable armchair. It is a lot harder to bring myself to go to my standing desk (though me using an IKEA shelf as a stool might have something to do with this).

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-08-23T17:47:46.328Z · LW · GW

Here is a response I wrote to the Import AI 337

I am confused about why people are building systems in the current machine learning paradigm and trying to make them more and more capable, without realizing that this can be dangerous. I basically think the arguments that Eliezer is making seem likely and should be taken seriously, but I expect most of the people working on bleeding edge systems don't even know these arguments.

For example, the argument that if you have a training process that trains a system to perform well on a text prediction task, then that doesn't necessarily mean that the resulting system will "just do text prediction". It seems quite likely to me that, as Eliezer says, intelligence is just a useful thing to have in order to perform better on the task of predicting text from the Internet. Therefore, at some point, as the systems become more and more capable, we should expect that through this optimization pressure, general intelligence will arise even for a task that seems as energetic as predicting text.

How much permission do AI developers need to get from society before irrevocably changing society?

Right now, to me it seems, like people are steering straight towards the doom. And nobody really ever approved this. But the problem is that most people, even the people doing this, don't realize that that's what they're doing. At least that's how it seems from my perspective.

Does progress always demand heterodox strategies?

I found it weird that you thought it would be weird if we got continuous learning systems. Because it seems very likely to me that if we get really capable systems at some point, will do active learning. Clearly, gradient descent is a pretty dumb optimization process that you can improve upon. Maybe we can get to the point without continuous learning where the systems improve themselves. This could then actually also be seen as a form of active learning. But at that point we the systems can improve themselves better than humans can are probably dead very very quickly.

Related to this, the thing I am working on is trying to figure out how we can do learning without using SGD. The hope is that if we find an algorithm that can learn, which we can just write down explicitly and understand, then that would make this algorithm pretty straightforward to align, especially if during the design process of the algorithm you build it such that it would be easy to align already.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-08-23T14:10:51.188Z · LW · GW

Arrogance VS Accurate Description

I know what it feels like to be arrogant. I was arrogant in the past. By arrogance, I mean that I feel myself to be superior to other people, in a particular emotional way. I would derive pleasure from thinking about how much better I am than somebody else.

I would talk with friends about other people in a subtly derogative way. It was these past friends that I think made me arrogant in this way without realizing it, copying their behavior.

All of this seems very bad to me. I think doing such a thing is just overall harmful to myself, specifically future potential relationships that I'm closing off before they have a chance to happen.

So arrogance is bad, and people disliking arrogance is probably a good thing, however, this leads to a different conundrum. Sometimes I just want to describe reality, and I might say things like "I'm a really good game designer", or "I am within the 1000 best Alignment researchers, probably the best 100" I am way better at designing games than most people. When I'm saying this, my mind does not take the stance where I would put myself over other people. And it doesn't make me feel really good when I say it.

Now, maybe sometimes there are still hints of arrogance in my mind when making statements like that. But I genuinely think it's mostly not there. But people still interpret this in exactly the same way. They perceive this as arrogance, even though the actual internal mechanisms in my brain that make me say these things, I think, are entirely different. One is some adaptation in order to exploit social dynamics to increase your own standing, while the other is simply stating my current best guess of what reality is actually like.

Once a person told me that they think Eliezer is really arrogant. Maybe he is running into the same problem.

Comment by Johannes C. Mayer (johannes-c-mayer) on I can see how I am Dumb · 2023-08-06T18:31:16.580Z · LW · GW

You writing this message reflecting on if writing this message is procrastination is probably an indicator for that it is, at least not the worst form of procrastination, which would be about entering a mental state where you don't think and in some sense really don't want to think about if you are procrastinating because whatever procrastination you're doing makes you feel good or whatever and provides escapism and some parts of your brain wouldn't want to go away.

At least that's my experience.

The longer and harder you think about if something is procrastination and come to the conclusion that it isn't, the more evidence I would say this is that it isn't procrastination (at least especially if you're trying to correct for biases).

Comment by Johannes C. Mayer (johannes-c-mayer) on Yes, It's Subjective, But Why All The Crabs? · 2023-08-06T16:21:35.126Z · LW · GW

[...] but people in fact mean a whole slew of wildly different things when they talk about “consciousness”.

Just because you have a different name for a concept doesn't necessarily mean that the concept isn't instrumentally convergent. It might be that there is a whole set of concepts that different people label with the same word, or the same concept that people have different names for.

In one video Judea Pearl was talking about consciousness as a kind of self-model that an agent could have. This is completely different from defining consciousness as there is some subjective experience. I.e. a system is conscious if it's like something to be a particular system, as Sam Harris would say it.

This means that we can run into a situation where we have the name "Consciousness" is a pointer to "A system has a model of itself" and "It's like something to be a system". But hypothetically Sam Harris might have the same concept of "A system has a model of itself" and Judea Pearl might have the concept of "It is like something to be this system" but label them with different names. Or actually not have any labels at all.

I have noticed within myself very often that I create new concepts that I do not label, which is bad, because later on I then realize that this concept already exists and somebody has thought about it, but because I didn't label it, I had just this very vague, fuzzy object in my mind that I couldn't really put my finger on precisely and therefore it was hard to think about, making me not cash out the insights that I could have cashed out that seem pretty obvious once I read about what other people have thought about that particular concept.

So, with concepts, there might be a lot more conversions that is visible at first glance, because of different labels.

Comment by Johannes C. Mayer (johannes-c-mayer) on Where are the people building AGI in the non-dumb way? · 2023-07-12T21:40:29.440Z · LW · GW

I feel like the thing that I'm hinting is not directly related to QACI. I'm talking about a specific way to construct an AGI where we write down all of the algorithms explicitly, whereas the QACI part of QACI, is about specifying an objective that is aligned when optimized very hard. It seems like, in the thing that I'm describing, you would get the alignment properties from a different place. You get them because you understand the algorithm of intelligence that you have written down very well. Whereas in QHCI, you get the alignment properties by successfully pointing to the causal process that is the human in the world that you want to "simulate" in order to determine the "actual objective".

Just to clarify, when I say non-DUMB way, I mainly refer to using giant neural networks and just making them more capable in order to get to intelligent systems to be the DUMB way. And Tasman's thing seems to be one of the least DUMB things I have heard recently. I can't see how this obviously fails (yet), though, of course, this doesn't necessarily imply that it will succeed (though it is of cause possible).

Comment by Johannes C. Mayer (johannes-c-mayer) on Where are the people building AGI in the non-dumb way? · 2023-07-10T09:15:02.111Z · LW · GW

I think the problem with the things you mention is that they are just super vague, where you don't even know what is the thing that you are talking about. What does it mean that:

Most important of all, perhaps, is making such machines learn from their own experience.

Finally, we'll get machines that think about themselves and make up theories, good or bad, of how they, themselves might work.

Also, all of this seems to be some sort of vague stuff about imagining how AI systems could be. I'm actually interested in just making the AI systems and making them in a very specific way such that they have good alignment properties and not vaguely philosophizing about what could happen. The whole point of writing down algorithms explicitly, which is one non-dumb way to build AGI, is that you can just see what's going on in the algorithm and understand it and design the algorithm in such a way that it would think in a very particular way.

So it's not like, oh yes, these machines will think for themselves some stuff and it will be good or bad, it's more like, I make these machines think, how do I make them think, what's the actual algorithm to make them think, how can I make this algorithm such that it will actually be aligned. And I am controlling what they are thinking, I am controlling if it's good or bad, I am controlling if they are going to build a model of themselves, maybe that's dangerous for alignment purposes in some context and then I would want the algorithm to not want the system to build a model of themselves.

For, at that point, they'll probably object to being called machines. I think it's pretty accurate to say that I am a machine.

(Also, as a meta note, it would be very good, I think, if you do not break the lines as you did in this big text block because that's pretty annoying to block quote.)

Comment by Johannes C. Mayer (johannes-c-mayer) on Where are the people building AGI in the non-dumb way? · 2023-07-10T09:05:39.823Z · LW · GW

I expect that it is much more likely that most people are looking at the current state of the art and don't even know or think about other possible systems and just narrowly focus on aligning the state of the art, not considering creating a "new paradigm", because they think that would just take too long.

I would be surprised if there were a lot of people who carefully thought about the topic and used the following reasoning procedure:

"Well, we could build AGI in an understandable way, where we just discover the algorithms of intelligence. But this would be bad because then we would understand intelligence very well, which means that the system is very capable. So because we understand it so well now, it makes it easier for us to figure out how to do lots of more capability stuff with the system, like making it recursively self-improving. Also, if the system is inherently more understandable, then it would also be easier for the AI to self-modify because understanding itself would be easier. So all of this seems bad, so instead we shouldn't try to understand our systems. Instead, we should use neural networks, which we don't understand at all, and use SGD in order to optimize the parameters of the neural network such that they correspond to the algorithms of intelligence, but are represented in such a format that we have no idea what's going on at all. That is much safer because now it will be harder to understand the algorithms of intelligence, making it harder to improve and use. Also if an AI would look at itself as a neural network, it would be at least a bit harder for it to figure out how to recursively self-improve."

Obviously, alignment is a really hard problem and it is actually very helpful to understand what is going on in your system at the algorithmic level in order to figure out what's wrong with that specific algorithm. How is it not aligned? And how would we need to change it in order to make it aligned? At least, that's what I expect. I think not using an approach where the system is interpretable hurts alignment more than capabilities. People have been steadily making progress at making our systems more capable and not understanding them at all, in terms of what algorithms they run inside, doesn't seem to be much of an issue there, however for alignment that's a huge issue.

Comment by Johannes C. Mayer (johannes-c-mayer) on Where are the people building AGI in the non-dumb way? · 2023-07-10T08:12:31.837Z · LW · GW

I've talked with and interviewed a lot of software developers, and it's probably fewer than 5% that really understand QuickSort including the variance in performance on pathological lists. This is trivially simple compared to large models, but not actually easy or self-explaining.

Well, these programmers probably didn't try to understand Quicksort. I think you can see simple dynamics such as, "oh this will always return a list that is the same size as the list that I input" and "all the elements in that list will be elements from the original list in a bijective mapping. There won't be different elements and there won't be duplicated elements or something like that." That part is pretty easy to see. And now there are some pathological cases for quick search, though I don't understand the mechanics of why they arise. However, I'm pretty sure that I can, within one hour, understand very well what these pathological cases are and why they arise, and how I might change a quick search algorithm to handle a particular pathological case well. That is, I'm not saying I look at Wikipedia and just read up on the pathological cases, but I just look at the algorithm alone and then derive the pathological cases. Maybe an hour is not enough, I'm not sure. That seems like an interesting experiment to test my claim.

I am pretty sure that there is a program that you can write down that has the same structural property of being interpretable in this way, where the algorithm also happens to define an AGI.

I am pretty sure that this is not possible.

Could you explain why you think that this is not possible? Do you really think there isn't an explicit Python program that I can write down such that within the Python program, e.g. write down the step-by-step instructions that when you follow them, you will end up building an accurate model of the world. And such that the program does not use any layered optimization like SGD or something similar. Do you think these kinds of instructions don't exist? Well, if they don't exist, how does the neural network learn things like constructing a world model? How does the human brain do it?

Once you write down your algorithm explicitly like that, I just expect that it will have this structural property I'm talking about of being possible to analyze and get intuitions about the algorithm.

Comment by Johannes C. Mayer (johannes-c-mayer) on johnswentworth's Shortform · 2023-07-09T11:07:30.073Z · LW · GW

Expected Utility Maximization is Not Enough

Consider a homomorphically encrypted computation running somewhere in the cloud. The computations correspond to running an AGI. Now from the outside, you can still model the AGI based on how it behaves, as an expected utility maximizer, if you have a lot of observational data about the AGI (or at least let's take this as a reasonable assumption).

No matter how closely you look at the computations, you will not be able to figure out how to change these computations in order to make the AGI aligned if it was not aligned already (Also, let's assume that you are some sort of Cartesian agent, otherwise you would probably already be dead if you were running these kinds of computations).

So, my claim is not that modeling a system as an expected utility maximizer can't be useful. Instead, I claim that this model is incomplete. At least with regard to the task of computing an update to the system, such that when we apply this update to the system, it would become aligned.

Of course, you can model any system, as an expected utility maximizer. But just because I can use the "high level" conceptual model of expected utility maximization, to model the behavior of a system very well. But behavior is not the only thing that we care about, we actually care about being able to understand the internal workings of the system, such that it becomes much easier to think about how to align the system.

So the following seems to be beside the point unless I am <missing/misunderstanding> something:

These two claims should probably not both be true! If any system can be modeled as maximizing a utility function, and it is possible to build a corrigible system, then naively the corrigible system can be modeled as maximizing a utility function.

Maybe I have missed the fact that the claim you listed says that expected utility maximization is not very useful. And I'm saying it can be useful, it might just not be sufficient at all to actually align a particular AGI system. Even if you can do it arbitrarily well.

Comment by Johannes C. Mayer (johannes-c-mayer) on johnswentworth's Shortform · 2023-07-05T16:52:26.334Z · LW · GW

This was arguably the most useful part of the SERI MATS 2 Scholars program.

Later on, we actually did this exercise with Eliezer. It was less valuable. It seemed like John was mainly prodding the people who were presenting the ideas, such that their patterns of thought would carry them in a good direction. For example, John would point out that a person proposes a one-bit experiment and asks if there isn't a better experiment that we could do that gives us lots of information all at once.

This was very useful because when you learn what kinds of things John will say, you can say them to yourself later on, and steer your own patterns of thought in a good direction on demand. When we did this exercise with Eliezer he was mainly explaining why a particular idea would not work. Often without explaining the generator behind his criticism. This can of course still be valuable as feedback for a particular idea. However, it is much harder to extract a general reasoning pattern out of this that you can then successfully apply later in different contexts.

For example, Eliezer would criticize an idea about trying to get a really good understanding of the scientific process such that we can then give this understanding to AI alignment researchers such that they can make a lot more progress than they otherwise would. He criticized this idea as basically being too hard to execute because it is too hard to successfully communicate how to be a good scientist, even if you are a good scientist.

Assuming the assertion is correct, hearing it, doesn't necessarily tell you how to think in different contexts such that you would correctly identify if an idea would be too hard to execute or flawed in some other way. And I am not necessarily saying that you couldn't extract a reasoning algorithm out of the feedback, but that if you could do this, then it would take you a lot more effort and time, compared to extracting a reasoning algorithm from the things that John was saying.

Now, all of this might have been mainly an issue of Eliezer not having a good model on how this workshop would have a positive influence on the people attending it. I would guess that if John had spent more time thinking about how to communicate what the workshop is doing and how to achieve its goal, then Eliezer could have probably done a much better job.

Comment by Johannes C. Mayer (johannes-c-mayer) on Write the Worst Post on LessWrong! · 2023-06-28T20:56:38.266Z · LW · GW

The goal of writing the worst LessWrong post is to set the standard by which you will measure yourself in the future. You want to use it as a tool to stop handicapping your own thought processes by constantly questioning yourself: "Is this really good enough?", "Should I write about this?", "Would anyone care?" Asking these questions is not necessarily a problem, in fact, they are probably good questions to consider. But in my experience, there is a self-deprecating way that you can ask these questions, which will just be demotivating, which I think is better to avoid.

The point of this post is to argue that you should lower your standards and just push out some posts when you start out writing. Nothing makes you better at writing than writing a lot. Don't worry about the quality of your posts too much in the beginning. Putting out many posts is more important. And there is some merit in them being bad because once you start to measure yourself against your past self, it will be easy to see how you improved and count that as a success.

Comment by Johannes C. Mayer (johannes-c-mayer) on Writing to think · 2023-06-23T06:40:55.921Z · LW · GW

Don't cringe when Looking at old Work

Actually, I think it leads to a second catch-22 as well. When I look back at my old posts, I'm horrified by a lot of them, despite the fact that I tried to hold myself to this high standard for publishing.

You know, the funny thing is that to get good at writing, writing a lot is actually a good strategy. Holding yourself to a very high can be actively harmful to the goal of writing a lot.

(I don't expect that you need this advice anymore, but maybe somebody else sees it.)

I think one way to frame this productively might be to have the realization that it is necessary to write a bunch of bad blog posts, in order to get good. Each of these bad blog posts, that you can look back upon, should be seen as a success. They were a necessary step along the way. It might not be apparent how you got better at writing from one blog post to the next, but ultimately the skill that you have reached now is the result of very many of these (at least most of the time) tiny steps.

Comment by Johannes C. Mayer (johannes-c-mayer) on Writing to think · 2023-06-23T06:28:44.666Z · LW · GW

A Stream model of Content Publishing

Writing to think makes sense. But what if the end result still turns out crappy? What if it's meh? What if it's good but not great? Should you publish it to the world? I'm someone who leans towards saying no. I like to make sure it's pretty refined and high quality.

But that leads me to a catch-22: most thoughts I want to explore don't seem promising enough where I'd end up publishing them. Or, rather, they usually seem like they'd take way too much time to refine. And if I'm not going to publish them, well, why write them up in the first place?

I feel like there is a conceptually (but possibly technically challenging) solution to this. You don't want to push away readers by writing bad posts, and you do not want them to update towards you being dumb. You also don't want to push away readers by writing about topics that they are not interested in. This would likely happen by default if you write about a wide range of topics.

IIRC somewhere in the Arbital postmortem there is a simple solution to this. Instead of having only one channel to publish your content, create multiple ones.

You could devise a system where you can assign a tag to a post. Readers can then decide for which tags they want to get notifications. This also gives you dynamic filtering of posts. I expect that to be good for exploring the content, e.g. finding older posts that you are likely interested in.

Additionally, you could, for each post, attach the amount of time that you have invested into the post per word written. This is probably a reasonable proxy for quality.

You could also sort each post into 3 bins based on how good you think that post turned out. Or have categories that classify how exploratory some piece of writing is. Is this writing the result of you trying to get a better understanding of something, or does the writing aim to provide the best possible explanation for an important topic that you understand well?

And I am sure if there is a lot more that you could do. It would probably be important to have a few good default configurations that people can choose between, to not overwhelm them with the available options. E.g. there could be one stream that is about topics you expect LessWrongers to be interested in, from the high-quality post tag category.

Comment by Johannes C. Mayer (johannes-c-mayer) on Writing to think · 2023-06-23T06:17:11.073Z · LW · GW

There is a relevant factor here to consider. When writing for yourself, you need to force yourself less to make things explicit. Things only need to make sense to you. You can often leave some concepts fuzzy, than if you were to write, such that another person can understand the writing.

This has the advantage that you can cover more ground in the same amount of time.

It has the disadvantage, in my model, writing forcing you to make things explicit is one of its major benefits. Making things explicit makes it easier to spot holes in your models.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-06-22T18:06:56.187Z · LW · GW

It's interesting to explore the limits of this intuition. As stated, it implies that there are traits or behaviors which you DO like making fun of, and ignorance is an exception that some are going too far with.

Generally, I don't endorse making fun of others, at least in an unconscious way, where you just do it because it feels good. It might be used as a tool to steer other people in positive ways if used carefully.

Personally, I sometimes engage in status games. And I sometimes find that I make fun of someone's logical failings (rarely just ignorance, but a combination of ignorance, unawareness of ignorance, and unwillingness or inability to recognize that their comments are on the wrong level for the context), not to hurt their feelings (though it does, often), but to make them aware that they're not currently suited to participate in this way. Ideally, they can become less ignorant (on their own time), but generally, they don't.

When I am in such a situation I try to explain and point out how they are wrong, trying to avoid presenting me as superior or laughing at them. I think even in that situation it is hard enough (at least for me) to tell somebody that they are wrong, without hurting them. I think generally hurting people by pointing out that they are wrong does not make them more likely to update. Rather the opposite. They get defensive, or even angry. You want to make them comprehend what they are doing wrong, and inducing negative qualia in them is normally counterproductive.

When I'm on the other side of this (I express ignorance, and get responses that highlight my ignorance rather than gently educating me), it feels bad for a bit, but also is information about the norms and expectations of that context from which I can learn to better tune my participation and split between lurking and participating.

Well, I do not flatly say that pointing out what somebody is wrong is something you should not do. It seems necessary to do this to communicate effectively. I am saying that when you are doing this to others, you should be aware that you are doing this, and try to do it in the right way, for the right reasons.

Comment by Johannes C. Mayer (johannes-c-mayer) on Johannes C. Mayer's Shortform · 2023-06-22T06:57:38.973Z · LW · GW

I dislike making fun of somebodies ignorance

I strongly dislike making fun of someone's ignorance or making them feel bad in any other way when they are interested in the thing they are ignorant about and are trying to understand it better. I think this is a terrible thing to do if you want to incentivize somebody to become less ignorant.

In fact, making somebody feel bad in this way, incentivizes the opposite. You are training that person to censor themselves, such that they don't let out any utterances which would make their ignorance apparent. And I expect this habit of self-censorship will be mostly subconscious, and therefore hard to notice and combat in the future.

Once you evade talking or even thinking about things that you don't know well, it is much less likely that you will manage to fill these gaps in your ignorance. Talking about your ignorance is usually a good way to destroy it. Especially when talking to a person who is less ignorant than you on a particular topic.

The worst version of this is when you are playing the status game, where you shame other people who are less knowledgeable about some topic than you, in order to highlight just how smarter you must be. Don't let this evil unbidden impulse sneak up on you. Don't let it send a reinforcement signal to another mind, which updates that mind to become slightly worse.

Comment by Johannes C. Mayer (johannes-c-mayer) on I can see how I am Dumb · 2023-06-22T06:22:00.085Z · LW · GW

Thinking about these abilities gives me the impression that highly automated and reinforced sub-conscious routines might not be easily changeable to the more effective or efficient by themselves, but they might be integrated into some higher-order routines, changing their eventual output. These could be more and more automated themselves, thereby achieving an increase in fluid intelligence.

I definitely think that one can become better at understanding and steering the world, by improving their cognitive algorithms. I am just saying that there are some low-level ones that can't be changed. So improvement needs to happen at a higher level. This then puts some hard limits on how much smarter you can get, and how much effort it takes to gain one unit of smartness.

On the point that you are not sure what you could even do, I just want to say: Did you try? It seems like the most common failure case is to not even try. Another common failure mode to avoid would be to have the wrong expectations about how hard something is, and then give up, because it is so much harder than expected. The hardness is I guess some indication of intelligence. Some people will find doing math just much easier than others, just because they are smarter.

But if you are trying to do something very hard, it might make sense to consider how somebody smart would feel doing this. Would they also struggle and find it difficult, because the problem before you is just intrinsically difficult? I think if you don't think this thought explicitly, the default implicit assumption is always that what you are doing is easy for everybody else who does it. "Writing a book is a struggle", is what I heard a professional author once say. Authors are authors, not necessarily because writing is a cakewalk for them. I would think more often than not, it is because they have managed to tune their expectations to reality, such that they no longer feel bad for taking the actually required time for completing whatever task they are working on.

I found what you said about the pre-conscious feeling interesting. It made me slightly improve my model about how to avoid procrastination and depression. Normally I only procrastinate when I feel pretty down (at least the kind of "hardcore" procrastination where you do something that definitely is not productive at all, such as watching a movie or playing a videogame). The problem is that once I am in the state of feeling down it is hard to execute a strategy that actually will make me feel better. For example, doing regular sports and meditation seems to help enormously with changing my general mood for the better. But once I feel really down these things are really hard to do. So what you need to do is develop the habit of carefully paying attention to your experience and notice when you are on a downward spiral, before you have fallen so low that interventions become difficult to implement. And then of cause you still need to actually implement the intervention, but becoming sensitive to subtle emotional trends (which I am still not as good at as I would like) seems to be >25% of the battle.