LessWrong 2.0 Reader

This describes the LessWrong team, and some other startups I've known of. The sense I've gotten is that most people either don't see the appeal or aren't willing to make that large a commitment? (Living with your teammates is a pretty big commitment, especially if you have other parts of your life, like a partner or kids.) Or, well, I think for the most part, people are working within the standard view of what an organization looks like and just don't think of this option.

Fixed. Thanks.

Thanks!

Or do you mean literally that the world-model uses row-hammer on the computer it runs, to make the supervisory signal positive?

Yes!

If row-hammering (or whatever) improves the loss, then the gradient will push in that direction.

I don't think this is true in the situation I'm talking about ("literally that the world-model uses row-hammer on the computer it runs, to make the supervisory signal positive").

Let's say we have weights θ, and loss is nominally the function f(θ), but the actual calculated loss is F(θ). Normally f(θ)=F(θ), but there are certain values of θ for which merely running the trained model corrupts the CPU, and thus the bits in the loss register are not what they're supposed to be according to the nominal algorithm. In those cases f(θ)≠F(θ).

Anyway, when the computer does symbolic differentiation / backprop, it's calculating ∇f, not ∇F. So it won't necessarily walk its way towards the minimum of F.

I can't imagine any processing step without 4th-wall-breaking consequences

Oh yeah, for sure. My idea was: sometimes the 4th-wall-breaking consequences are part of the reason that the processing step is there in the first place, and sometimes the 4th-wall-breaking consequences are just an incidental unintended side-effect, sorta an "externality".

Like, as the saying goes, maybe a butterfly flapping its wings in Mexico will cause a tornado in Kansas three months later. But that's not why the butterfly flapped its wings. If I'm working on the project of understanding the butterfly—why does it do the things it does? why is it built the way it's built?—knowing that there was a tornado in Kansas is entirely unhelpful. It contributes literally nothing whatsoever to my success in this butterfly-explanation project.

So by the same token, I think it's possible that we can work on the project of understanding a postdictively-trained model—why does it do the things it does? why is it built the way it's built?—and find that thinking about the 4th-wall-breaking consequences of the processing steps is entirely unhelpful for this project.

I don't see any reason why a sufficiently advance postdictive learner with a general enough modality (like text) wouldn't learn to model 4th-wall-breaking consequences: that's just the sort of thing you need to predict security exploits or AI alignment posts like this one.

Of course a good postdictive learner will learn that other algorithms can be manipulative, and it could even watch itself in a mirror and understand the full range of things that it could do (see the part of this post "Let’s take a postdictive learner, and grant it “self-awareness”…"). Hmm, maybe the alleged mental block I have in mind is something like "treating one's own processing steps as being part of the physical universe, as opposed to taking the stance where you're trying to the universe from outside it". I think an algorithm could predict that security researchers can find security exploits, and predict that AI alignment researchers could write comments like this one, while nevertheless "trying to understand the universe from outside it".

there's still the risk of spinning agents early in training

Oh yeah, for sure, in fact I think there's a lot of areas where we need to develop safety-compatible motivations as soon as possible, and where there's some kind of race to do so (see "Fraught Valley" section here [AF · GW]). I mean, "hacking into the training environment" is in that category too—you want to install the safety-compatible motivation (where the model doesn't want to hack into the training environment) sooner than the model becomes a superintelligent adversary trying to hack into the training environment. I don't like those kinds of races and wish I had better ideas for avoiding them.

I listened to David Goggins' account of Navy SEAL training last year. They encourage you to push yourself so hard that you are at genuine risk of death or permanent disability. The first two times Goggins tried to get through he failed out because of injuries, even though he was willing to — and did — run many miles on literally broken legs. He only made it through the third time because hell week got cut short due to someone in their cohort DYING (from participating in an exercise to tie and untie knots at the bottom of a swimming pool while very sick with pneumonia).

I actually found the book incredibly inspiring, though it did not make me think anyone should model themselves after the Navy SEALs in particular. I also don't think someone should run 100 miles in 24 hours with zero training and continue despite the fact that their legs are breaking and they're shitting and pissing blood while they run, which is another thing that Goggins did.

One training exercise in the book that seemed more reasonable to me (more like an exercise and less like abject torture) was an orienteering-type thing (for I think the Army Rangers?), where the terrain was treacherous and unfamiliar and the weather dangerously cold at night. I think it's a good test of rationality to put yourself in a genuinely high-stakes situation like that — as long as one of the choices you're allowed to make is to call for help if you are genuinely afraid for your life. That was an option in the case of the Rangers orienteering challenge, but my point is that the thing that's bad about SEAL hell week is that you're considered a pussy if you quit, even if it's out of genuine and reasonable fear for your life.

The book overall is about the idea that your limits are fake, and humans can accomplish things that seem like they should be physically impossible as long as they just don't give up. I think that's a concept we could work with. 

I think there are quite a few rationalists who challenge themselves to do fairly hard things, like founding a successful startup, putting together a large conference on short notice at the age of 18, or publishing a good post on rationality every day for a month, things kind of like that. I think I've challenged myself a lot more than I would have if I weren't in the rationalist community, but I don't think I've ever tried to do something that I felt was impossible. (I think a precious few rationalists have faced the impossible — probably Holden and Eliezer, to name any at all — but they're very much the exception rather than the rule.)

Here are some things that feel impossible:

• Write something as groundbreaking as the sequences, starting in one week (that's your planning period) and posting every day for at least a year
• Cause the public collective consciousness and ~all significant intellectuals in the US to take x-risk (and especially AI x-risk) seriously, within the year
  • Note that I very much do not suggest that people throw themselves at this task!
• Make a novel discovery in particle physics (or a similar well-established field that you've never studied before), within six months
• Without piggybacking on any existing space exploration project, put a spacecraft of your own design / owned by you on the moon within five years
• Found a new country that gets recognized by the UN

And here are some things where I can see a path to accomplishing them, but where that path feels incredibly hard and scary — these examples are specific to me:

• Become fluent in Mandarin, both speaking/listening AND reading/writing, in the next three months
  • I have a lifetime of failure to learn Mandarin behind me, including one academic year when I really actually tried, also Mandarin is just really fucking hard
• Run a marathon within the next year 
  • I have a chronic leg injury that makes running essentially impossible, that feels insurmountable but probably in reality is not
• Make a million dollars in the next six months just via investing/betting
  • I am a very risk-averse person and was raised to fear the stock market
• Permanently fix my depression and anxiety
  • It's probably not impossible but jeeeeeeeeeeeeeeezzzz
• Found and run a company, like, one with actual employees and investors and a goal of growth (not just a one-person LLC, that's cheating)
  • This just sounds awful in every way; I hate dealing with people and money and feel super unqualified for all of this

Again these will be different for different people. I think Eliezer's quest to lose weight qualifies somewhere around here. I think things in this class are probably better candidates for serious rationality training exercises than the first list, though, maybe that's wrong.

Anyway the goal is not to teach object-level skills, but to cause people to change their outlook on tasks that seem impossible. I think that's one really important skill for rationalists/EAs to have, though not the only important skill. In any given quest you will probably learn additional useful object-level skills.

So idk those are some thoughts on one aspect of the thing. Didn't properly feel like an answer so here it is as a comment instead.

I hadn't heard of mirror life before and the wiki page is unhelpful in explaining why it might be a risk. Have any resources on this? 

Open hypnothesis:

I had my first vaccination with AstraZeneca after which I spent a day in bed. For me second vaccination I had Biontech where I had less energy the next day but not so strong effects as the first one.

It's quite odd that the side-effects of the vaccines are so different from person to person.

One thing that might explain how the side effects are so different is that maybe for some people the vaccine stays mostly at the point where it was injected and for others it travels more through the body which causes side effects.

I had those ideas before my second vaccination and as a result I made sure to do nothing to relax the side of injection for the 36 hours after injection while I did do so after the first vaccination. I wanted the vaccine in my arm to get immunity but not in my brain getting my body to attack brain cells even if it only kills a nonsignificant amount of them.

This hypothesis is very much guesswork at this point, but the experiment it suggests is very cheap. Just leave the side of injection in the tension it has after injection till the second day after the vaccination. 

I'd be curious if other people would try it when they get additional vaccinations while having a lot of side effects with the first vaccination and whether the approach could also reduce side effects for other people. 

D&D website estimates 13.7m active players and rising.

I don't think the two closed answers of "Have you stopped beating your wife ?" have such a well-defined meaning. Since this is natural language, and I understand a no as meaning "I'm still beating her." and I expect most people to interpret a no the same way as I, then it's not from obvious why this interpretation is incorrect (if we ignore that the sentence is typically used as an example that has no good answer. Use "Will you stop smoking soon ?" which is less standard for the sake of the argument.)

(on the other hand, the inability to condition yourself seems relevant here. It seems like the brain might be not be controlling for whether something is reasonable, but only for whether something is produced by yourself. So maybe exercise counts because it’s under your control, but waterboarding doesn’t count because it isn’t. I wonder if anyone has ever tried letting someone waterboard themselves and giving them the on-off switch for the waterboarding device. Was Hitchens’ experience close enough to this to count? Why would this be different from letting someone hold their breath, which doesn’t produce the same level of panic?)

Hypothesis: the difference is in the failure mode.

If you hold your breath, you can always choose (assuming you are not underwater or in some other environment that prevents proper breathing) to stop holding your breath and save yourself from suffocating. If you are being waterboarded by friendly demonstrators, you can say the safe word and save yourself from "drowning". These may seem the same, but are not. In both cases the longer you hold the weaker you get - the way that weakening affects your ability to stop the ordeal is very different.

The longer you hold your breath the harder it gets to keep holding your breath - until at some point you are no longer able to hold your breath and are forced to breath. Even if you can keep holding your breath past that point - you are just going to pass out, and then you'll just switch to autopilot and breath automatically. Unless you suffer from Ondine's curse, failure will not kill you.

With waterboarding, if you become too weak to properly signal the "torturers" to stop - they won't stop. Sure, in Hitchens' case they noticed that he passed out and stopped it. Because they are professionals. But this is probably too high level for your subconsciousness and your body to rely on - as far as they care, failure can mean death. Holding as long as you can is no longer a safe option - so your body will try to scream at you to stop it as soon as possible.

You say "these are very wide error bars", in reference to the plot of Ct value versus vaccination status, hence casting doubt on the idea that the vaccinated and unvaccinated have similar Ct values.

But I think the boxplots shown represent the distribution of values, NOT the uncertainty in the estimated mean.  With n=84 and n=127 for the two groups, the standard error in the difference in the two means should be around 0.5 (an exact figure isn't obtainable from the displayed data, which shows quantiles, not standard deviations). Whether that's "small" or not is of course a subjective judgement, but it's a lot smaller than the size of the boxes.

However, a serious problem with this plot is that the unvaccinated are lumped together with "unknown" and "not fully vaccinated".  Since unknowns may well be vaccinated, and the partially-vaccinated have a large part of the immunity of the fully-vaccinated, the plot is basically useless.